René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"VM migration with disks fails between local storage hypervisors if the VM template is deleted","articleBody":"### problem\n\nWhen trying to migrate a virtual machine between two local storage hypervisors, the migration fails if the VM template is removed.\n\nI did some reasearch, and have figured out the possible root cause of it. \n\nFirst, the template is not found here: [StorageSystemDataMotionStrategy.java:2158](https://github.com/apache/cloudstack/blob/4.20/engine/storage/datamotion/src/main/java/org/apache/cloudstack/storage/motion/StorageSystemDataMotionStrategy.java#L2158)\nTherefore, the system thinks this VM does not have a template.\n\nAs the result, the system thinks this VM does not have any backing files, and tries to perform full clone migration: [MigrateKVMAsync.java:128](https://github.com/apache/cloudstack/blob/4.20/plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/resource/MigrateKVMAsync.java#L128). This seems to be the right migration option, the documentation says it will perform the migration of all the VMs disks: [VIR_MIGRATE_NON_SHARED_INC](https://libvirt.org/html/libvirt-libvirt-domain.html#VIR_MIGRATE_NON_SHARED_INC)\nBut there is a catch: the QCOW2 images have to be precreated on the destination machine with the correct size.\n\nThere is a method that would copy the template from the secondary storage to the destination host: [KvmNonManagedStorageDataMotionStrategy.java:220](https://github.com/apache/cloudstack/blob/4.20/engine/storage/datamotion/src/main/java/org/apache/cloudstack/storage/motion/KvmNonManagedStorageDataMotionStrategy.java#L220-L235), but there is no method to just create a QCOW2 file with the specific size at the specific path.\n\n\n### versions\n\n4.20\n\n### The steps to reproduce the bug\n\n1. Take two KVM hypervisors with local storage pools.\n2. Create a virtual machine from a template on one of them.\n3. Delete this template after the test virtual machine was started.\n4. Try to migrate the virtual machine with its storage to the other hypervisor.\n5. It will fail with an error telling that the backing file does not exist on the target machine:\n```\n2025-01-22 12:41:08,662 ERROR [c.c.a.ApiAsyncJobDispatcher] (API-Job-Executor-1:[ctx-9e293509, job-226]) (logid:61e8bd24) Unexpected exception while executing org.apache.cloudstack.api.command.admin.vm.MigrateVirtualMachineWithVolumeCmd com.cloud.utils.exception.CloudRuntimeException: Failed to migrate VM [VM instance {\"id\":18,\"instanceName\":\"i-2-18-VM\",\"type\":\"User\",\"uuid\":\"d3b043ff-716a-4bc8-b20e-639853b0babe\"}] along with its volumes due to [com.cloud.utils.exception.CloudRuntimeException: Copy volume(s) to storage(s) [{volume: \"33\", from: \"3\", to:\"4\"}] and VM to host [{vm: \"18\", from: \"2\", to:\"4\"}] failed in StorageSystemDataMotionStrategy.copyAsync. Error message: [Exception during migrate: org.libvirt.LibvirtException: Path '/var/lib/libvirt/images/bb51c27d-ca94-4cda-a74e-cec07295ce26' is not accessible: No such file or directory].].\n\tat org.apache.cloudstack.engine.orchestration.VolumeOrchestrator.migrateVolumes(VolumeOrchestrator.java:1444)\n\tat java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)\n\tat java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n....\n```\n\n### What to do about it?\n\nIt would be nice to fix the storage migration when the VM template was removed.\n\nI would like to try to develop a fix myself but my knowledge of Cloudstack internals is not enough to know where to look.\nI'd need some guidance how to properly develop the fix, for example point me to the right files where I can look. ","author":{"url":"https://github.com/phsm","@type":"Person","name":"phsm"},"datePublished":"2025-01-23T08:13:11.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":11},"url":"https://github.com/10246/cloudstack/issues/10246"}