René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"XCP-ng 8.2 - Live migration fails","articleBody":"##### ISSUE TYPE\r\n * Bug Report\r\n\r\n##### COMPONENT NAME\r\n~~~\r\nAPI, UI\r\n~~~\r\n\r\n##### CLOUDSTACK VERSION\r\n~~~\r\n4.18 RC3\r\n~~~\r\n\r\n##### CONFIGURATION\r\nAdvanced networking, 1 cluster, 2 XCP- ng hosts in the cluster, 2 storage pools\r\n\r\n\r\n##### OS / ENVIRONMENT\r\nN/A\r\n\r\n\r\n##### SUMMARY\r\nLive migration of user VMs fail\r\n\r\n\r\n##### STEPS TO REPRODUCE\r\n~~~\r\nCreate VM on a host, start VM = true\r\nAttempt a live migration of the VM to the other host\r\n~~~\r\n\r\n##### EXPECTED RESULTS\r\n~~~\r\nVM migration succeeds\r\n~~~\r\n\r\n##### ACTUAL RESULTS\r\n~~~\r\n2023-03-15 02:09:01,054 DEBUG [c.c.a.t.Request] (Work-Job-Executor-11:ctx-4d6cfe4b job-48/job-49 ctx-8797bea1) (logid:603f76bc) Seq 2-3130846165952627095: Executing:  { Cmd , MgmtId: 167781012, via: 2(ref-trl-4655-x-Mr8-nicolas-vazquez-xs1), Ver: v1, Flags: 100011, [{\"com.cloud.agent.api.MigrateCommand\":{\"vmName\":\"i-2-5-VM\",\"destIp\":\"10.0.32.148\",\"migrateStorageManaged\":\"false\",\"migrateNonSharedInc\":\"false\",\"autoConvergence\":\"false\",\"hostGuid\":\"0239ad4f-31e8-4e38-9e41-99edf0df35cf\",\"isWindows\":\"false\",\"vmTO\":{\"id\":\"5\",\"name\":\"i-2-5-VM\",\"bootloader\":\"PyGrub\",\"state\":\"Running\",\"type\":\"User\",\"cpus\":\"1\",\"minSpeed\":\"500\",\"maxSpeed\":\"500\",\"minRam\":\"(512.00 MB) 536870912\",\"maxRam\":\"(512.00 MB) 536870912\",\"arch\":\"x86_64\",\"os\":\"CentOS 5.6 (64-bit)\",\"platformEmulator\":\"CentOS 7\",\"bootArgs\":\"\",\"enableHA\":\"false\",\"limitCpuUse\":\"false\",\"enableDynamicallyScaleVm\":\"false\",\"params\":{\"cpuOvercommitRatio\":\"2.0\",\"memoryOvercommitRatio\":\"1.0\"},\"uuid\":\"d06072e4-7a60-4c00-9794-cd392bf9c3ac\",\"enterHardwareSetup\":\"false\",\"disks\":[{\"data\":{\"org.apache.cloudstack.storage.to.VolumeObjectTO\":{\"uuid\":\"d57788a0-0667-475c-8c58-64f65461d121\",\"volumeType\":\"ROOT\",\"dataStore\":{\"org.apache.cloudstack.storage.to.PrimaryDataStoreTO\":{\"uuid\":\"a39e31cd-cd6b-35a1-8b60-025dde80927a\",\"name\":\"ref-trl-4655-x-Mr8-nicolas-vazquez-xs-pri2\",\"id\":\"2\",\"poolType\":\"NetworkFilesystem\",\"host\":\"10.0.32.4\",\"path\":\"/acs/primary/ref-trl-4655-x-Mr8-nicolas-vazquez/ref-trl-4655-x-Mr8-nicolas-vazquez-xs-pri2\",\"port\":\"2049\",\"url\":\"NetworkFilesystem://10.0.32.4/acs/primary/ref-trl-4655-x-Mr8-nicolas-vazquez/ref-trl-4655-x-Mr8-nicolas-vazquez-xs-pri2/?ROLE=Primary\u0026STOREUUID=a39e31cd-cd6b-35a1-8b60-025dde80927a\",\"isManaged\":\"false\"}},\"name\":\"ROOT-5\",\"size\":\"(20.00 GB) 21474836480\",\"path\":\"a5cf9ac8-ca32-4d8d-9a1c-f4cd4e3d56f2\",\"volumeId\":\"6\",\"vmName\":\"i-2-5-VM\",\"accountId\":\"2\",\"format\":\"VHD\",\"provisioningType\":\"THIN\",\"poolId\":\"2\",\"id\":\"6\",\"deviceId\":\"0\",\"bytesReadRate\":\"(0 bytes) 0\",\"bytesWriteRate\":\"(0 bytes) 0\",\"iopsReadRate\":\"(0 bytes) 0\",\"iopsWriteRate\":\"(0 bytes) 0\",\"hypervisorType\":\"XenServer\",\"directDownload\":\"false\",\"deployAsIs\":\"false\"}},\"diskSeq\":\"0\",\"path\":\"a5cf9ac8-ca32-4d8d-9a1c-f4cd4e3d56f2\",\"type\":\"ROOT\",\"_details\":{\"storageHost\":\"10.0.32.4\",\"managed\":\"false\",\"storagePort\":\"2049\",\"storage.pool.disk.wait\":\"60\",\"volumeSize\":\"(20.00 GB) 21474836480\"}},{\"data\":{\"org.apache.cloudstack.storage.to.TemplateObjectTO\":{\"id\":\"0\",\"format\":\"ISO\",\"accountId\":\"0\",\"hvm\":\"false\",\"bootable\":\"false\",\"directDownload\":\"false\",\"deployAsIs\":\"false\"}},\"diskSeq\":\"3\",\"type\":\"ISO\"}],\"nics\":[{\"deviceId\":\"0\",\"networkRateMbps\":\"200\",\"defaultNic\":\"true\",\"pxeDisable\":\"false\",\"nicUuid\":\"c0528515-a991-44b9-a445-35b9b7e13943\",\"details\":{\"PromiscuousMode\":\"false\",\"MacAddressChanges\":\"true\",\"ForgedTransmits\":\"true\",\"MacLearning\":\"false\"},\"dpdkEnabled\":\"false\",\"uuid\":\"21980820-3162-43d0-a760-933203f24752\",\"ip\":\"10.1.1.152\",\"netmask\":\"255.255.255.0\",\"gateway\":\"10.1.1.1\",\"mac\":\"02:00:4b:fb:00:03\",\"dns1\":\"10.0.32.1\",\"dns2\":\"8.8.8.8\",\"broadcastType\":\"Vlan\",\"type\":\"Guest\",\"broadcastUri\":\"vlan://1024\",\"isolationUri\":\"vlan://1024\",\"isSecurityGroupEnabled\":\"false\",\"name\":\"Network1\"}],\"vcpuMaxLimit\":\"3\",\"configDriveLabel\":\"config-2\",\"configDriveLocation\":\"SECONDARY\",\"guestOsDetails\":{\"xenserver.dynamicMin\":\"536870912\",\"xenserver.dynamicMax\":\"17179869184\"},\"extraConfig\":{}},\"executeInSequence\":\"false\",\"migrateDiskInfoList\":[],\"dpdkInterfaceMapping\":{},\"vlanToPersistenceMap\":{\"1024\":\"true\"},\"wait\":\"0\",\"bypassHostMaintenance\":\"false\"}}] }\r\n2023-03-15 02:09:01,054 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-394:ctx-043826c7) (logid:4e4e66ee) Seq 2-3130846165952627095: Executing request\r\n2023-03-15 02:09:01,095 WARN  [c.c.h.x.r.CitrixResourceBase] (DirectAgent-394:ctx-043826c7) (logid:603f76bc) Task failed! Task record:                 uuid: 5865769c-39d0-abbd-6463-fbe8a69e1322\r\n           nameLabel: Async.VM.pool_migrate\r\n     nameDescription: \r\n   allowedOperations: []\r\n   currentOperations: {}\r\n             created: Wed Mar 15 02:09:01 UTC 2023\r\n            finished: Wed Mar 15 02:09:01 UTC 2023\r\n              status: failure\r\n          residentOn: com.xensource.xenapi.Host@1c19dce\r\n            progress: 1.0\r\n                type: \u003cnone/\u003e\r\n              result: \r\n           errorInfo: [VM_LACKS_FEATURE, OpaqueRef:9a1b9861-3dff-418f-89ba-ce31f75a3528]\r\n         otherConfig: {}\r\n           subtaskOf: com.xensource.xenapi.Task@aaf13f6f\r\n            subtasks: []\r\n\r\n2023-03-15 02:09:01,095 DEBUG [c.c.a.ApiServlet] (qtp1955920234-22:ctx-f292c428) (logid:9830e96f) ===START===  10.0.3.251 -- GET  jobId=603f76bc-0fc9-4d79-a98f-2433fc5be9e7\u0026command=queryAsyncJobResult\u0026response=json\r\n2023-03-15 02:09:01,096 DEBUG [c.c.a.ApiServlet] (qtp1955920234-22:ctx-f292c428) (logid:9830e96f) Two factor authentication is already verified for the user 2, so skipping\r\n2023-03-15 02:09:01,101 WARN  [c.c.h.x.r.CitrixResourceBase] (DirectAgent-394:ctx-043826c7) (logid:603f76bc) Unable to migrate VM(i-2-5-VM) from host(a7cd7897-f5be-4a94-bafb-600683304577)\r\nTask failed! Task record:                 uuid: 5865769c-39d0-abbd-6463-fbe8a69e1322\r\n           nameLabel: Async.VM.pool_migrate\r\n     nameDescription: \r\n   allowedOperations: []\r\n   currentOperations: {}\r\n             created: Wed Mar 15 02:09:01 UTC 2023\r\n            finished: Wed Mar 15 02:09:01 UTC 2023\r\n              status: failure\r\n          residentOn: com.xensource.xenapi.Host@1c19dce\r\n            progress: 1.0\r\n                type: \u003cnone/\u003e\r\n              result: \r\n           errorInfo: [VM_LACKS_FEATURE, OpaqueRef:9a1b9861-3dff-418f-89ba-ce31f75a3528]\r\n         otherConfig: {}\r\n           subtaskOf: com.xensource.xenapi.Task@aaf13f6f\r\n            subtasks: []\r\n\r\n\tat com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.checkForSuccess(CitrixResourceBase.java:508)\r\n\tat com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.migrateVM(CitrixResourceBase.java:3968)\r\n\tat com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixMigrateCommandWrapper.execute(CitrixMigrateCommandWrapper.java:86)\r\n\tat com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixMigrateCommandWrapper.execute(CitrixMigrateCommandWrapper.java:42)\r\n\tat com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixRequestWrapper.execute(CitrixRequestWrapper.java:122)\r\n\tat com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:1782)\r\n\tat com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:315)\r\n\tat org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)\r\n\tat org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)\r\n\tat org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)\r\n\tat org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)\r\n\tat org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)\r\n\tat java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)\r\n\tat java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)\r\n\tat java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)\r\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)\r\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)\r\n\tat java.base/java.lang.Thread.run(Thread.java:829)\r\n2023-03-15 02:09:01,105 DEBUG [c.c.a.ApiServer] (qtp1955920234-22:ctx-f292c428 ctx-2c8cdcdb) (logid:9830e96f) CIDRs from which account 'Account [{\"accountName\":\"admin\",\"id\":2,\"uuid\":\"513b92a7-c2c9-11ed-bc47-1e000e000327\"}]' is allowed to perform API calls: 0.0.0.0/0,::/0\r\n2023-03-15 02:09:01,105 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-394:ctx-043826c7) (logid:603f76bc) unable to destroy task(com.xensource.xenapi.Task@7c4c8878) on host(a7cd7897-f5be-4a94-bafb-600683304577) due to You gave an invalid object reference.  The object may have recently been deleted.  The class parameter gives the type of reference given, and the handle parameter echoes the bad value given.\r\n2023-03-15 02:09:01,105 WARN  [c.c.h.x.r.w.x.CitrixMigrateCommandWrapper] (DirectAgent-394:ctx-043826c7) (logid:603f76bc) Unable to migrate VM(i-2-5-VM) from host(a7cd7897-f5be-4a94-bafb-600683304577)\r\ncom.cloud.utils.exception.CloudRuntimeException: Unable to migrate VM(i-2-5-VM) from host(a7cd7897-f5be-4a94-bafb-600683304577)\r\n\tat com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.migrateVM(CitrixResourceBase.java:3979)\r\n\tat com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixMigrateCommandWrapper.execute(CitrixMigrateCommandWrapper.java:86)\r\n\tat com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixMigrateCommandWrapper.execute(CitrixMigrateCommandWrapper.java:42)\r\n\tat com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixRequestWrapper.execute(CitrixRequestWrapper.java:122)\r\n\tat com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:1782)\r\n\tat com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:315)\r\n\tat org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)\r\n\tat org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)\r\n\tat org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)\r\n\tat org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)\r\n\tat org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)\r\n\tat java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)\r\n\tat java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)\r\n\tat java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)\r\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)\r\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)\r\n\tat java.base/java.lang.Thread.run(Thread.java:829)\r\n2023-03-15 02:09:01,106 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-394:ctx-043826c7) (logid:603f76bc) Seq 2-3130846165952627095: Response Received: \r\n2023-03-15 02:09:01,106 DEBUG [c.c.a.t.Request] (DirectAgent-394:ctx-043826c7) (logid:603f76bc) Seq 2-3130846165952627095: Processing:  { Ans: , MgmtId: 167781012, via: 2(ref-trl-4655-x-Mr8-nicolas-vazquez-xs1), Ver: v1, Flags: 10, [{\"com.cloud.agent.api.MigrateAnswer\":{\"result\":\"false\",\"details\":\"Unable to migrate VM(i-2-5-VM) from host(a7cd7897-f5be-4a94-bafb-600683304577)\",\"wait\":\"0\",\"bypassHostMaintenance\":\"false\"}}] }\r\n2023-03-15 02:09:01,107 DEBUG [c.c.a.t.Request] (Work-Job-Executor-11:ctx-4d6cfe4b job-48/job-49 ctx-8797bea1) (logid:603f76bc) Seq 2-3130846165952627095: Received:  { Ans: , MgmtId: 167781012, via: 2(ref-trl-4655-x-Mr8-nicolas-vazquez-xs1), Ver: v1, Flags: 10, { MigrateAnswer } }\r\n~~~\r\n","author":{"url":"https://github.com/nvazquez","@type":"Person","name":"nvazquez"},"datePublished":"2023-03-15T02:17:44.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":12},"url":"https://github.com/7340/cloudstack/issues/7340"}