René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Timeout snapshot","articleBody":"### problem\n\nWhen creating snapshots of large volumes (approximately larger than 200 GB), the task fails to complete successfully. The snapshot command reaches a timeout of 3600 seconds, which is not enough time to copy the entire disk.\n\nWe attempted to modify various timeout-related parameters in CloudStack (including those for snapshot and asynchronous tasks), but none of them appear to extend or affect this timeout limit.\n\nIt is unclear whether this is a bug or a missing feature that should allow increasing the timeout duration for snapshot operations.\n\nThe corresponding logs are attached for further analysis.\n```\nManagement:\n\n2025-10-21 02:01:57,863 DEBUG [c.c.s.s.SnapshotSchedulerImpl] (SnapshotPollTask:ctx-93846872) (logid:f246f83a) Snapshot [455b227d-797f-45e6-aa03-1dc7609ac030] for volume [{\"name\":\"ROOT-392\",\"uuid\":\"8508204f-24ba-454a-8bd1-f56b274088ae\"}] can be executed.\n2025-10-21 02:01:59,126 DEBUG [c.c.s.s.SnapshotSchedulerImpl] (SnapshotPollTask:ctx-93846872) (logid:f246f83a) Scheduling snapshot [455b227d-797f-45e6-aa03-1dc7609ac030] for volume [{\"name\":\"ROOT-392\",\"uuid\":\"8508204f-24ba-454a-8bd1-f56b274088ae\"}] at [2025-10-21 05:00:00 GMT].\n2025-10-21 02:01:59,179 DEBUG [c.c.s.s.SnapshotSchedulerImpl] (SnapshotPollTask:ctx-93846872) (logid:f246f83a) Scheduled snapshot [455b227d-797f-45e6-aa03-1dc7609ac030] for volume [{\"name\":\"ROOT-392\",\"uuid\":\"8508204f-24ba-454a-8bd1-f56b274088ae\"}] as job [d0e089bb-d2cb-4e81-8fb4-5e6a23eee556].\n2025-10-21 02:02:01,607 DEBUG [o.a.c.s.s.StorPoolSnapshotStrategy] (Work-Job-Executor-28:ctx-a246b76c job-33232/job-33233 ctx-6f646c29) (logid:d0e089bb) StorpoolSnapshotStrategy.canHandle: snapshot=server18063_ROOT-392_20251021050158, uuid=70decb5c-6f4d-405a-b7e1-f2eb2ace4de0, op=TAKE\n\n\n\n2025-10-21 03:02:01,993 DEBUG [o.a.c.s.s.SnapshotServiceImpl] (Work-Job-Executor-28:ctx-a246b76c job-33232/job-33233 ctx-6f646c29) (logid:d0e089bb) create snapshot server18063_ROOT-392_20251021050158 failed: com.cloud.utils.exception.CloudRuntimeException: Failed to send command, due to Agent:59, com.cloud.exception.OperationTimedoutException: Commands 545217029888601412 to Host 59 timed out after 3600\n\n\nHost:\n\n\n2025-10-21 03:06:26,176 WARN  [utils.script.Script] (Script-10:null) (logid:) Interrupting script.\n2025-10-21 03:06:26,176 WARN  [utils.script.Script] (agentRequest-Handler-2:null) (logid:d0e089bb) Process [3934352] for command [qemu-img convert -O qcow2 -U --imag\ne-opts driver=qcow2,file.filename=/mnt/4c524bab-25a3-3db5-a665-d37159b81f11/11806790-0ad0-4a4a-b389-2f7ab41b4e87 /mnt/4c524bab-25a3-3db5-a665-d37159b81f11/snapshots/\n3ae09c67-c072-48b4-a8a7-e3f2dbaf2687 ] timed out. Output is [].\n2025-10-21 03:06:43,053 ERROR [kvm.storage.KVMStorageProcessor] (agentRequest-Handler-2:null) (logid:d0e089bb) Failed take snapshot for volume [volumeTO[uuid=8508204\nf-24ba-454a-8bd1-f56b274088ae|path=11806790-0ad0-4a4a-b389-2f7ab41b4e87|datastore=PrimaryDataStoreTO[uuid=4c524bab-25a3-3db5-a665-d37159b81f11|name=Pool-01|id=18|poo\nltype=NetworkFilesystem]]], in VM [i-52-392-VM], due to [Failed to convert volumeTO[uuid=8508204f-24ba-454a-8bd1-f56b274088ae|path=11806790-0ad0-4a4a-b389-2f7ab41b4e\n87|datastore=PrimaryDataStoreTO[uuid=4c524bab-25a3-3db5-a665-d37159b81f11|name=Pool-01|id=18|pooltype=NetworkFilesystem]] snapshot of volume [KVMPhysicalDisk {\"dispN\name\":null,\"format\":\"qcow2\",\"name\":\"11806790-0ad0-4a4a-b389-2f7ab41b4e87\",\"path\":\"\\/mnt\\/4c524bab-25a3-3db5-a665-d37159b81f11\\/11806790-0ad0-4a4a-b389-2f7ab41b4e87\",\"\npool\":{\"uuid\":\"4c524bab-25a3-3db5-a665-d37159b81f11\",\"path\":\"\\/mnt\\/4c524bab-25a3-3db5-a665-d37159b81f11\"},\"size\":122363461632,\"virtualSize\":161061273600,\"vmName\":nu\nll}] to [/mnt/4c524bab-25a3-3db5-a665-d37159b81f11/snapshots/3ae09c67-c072-48b4-a8a7-e3f2dbaf2687] due to [timeout].].\ncom.cloud.utils.exception.CloudRuntimeException: Failed to convert volumeTO[uuid=8508204f-24ba-454a-8bd1-f56b274088ae|path=11806790-0ad0-4a4a-b389-2f7ab41b4e87|datas\ntore=PrimaryDataStoreTO[uuid=4c524bab-25a3-3db5-a665-d37159b81f11|name=Pool-01|id=18|pooltype=NetworkFilesystem]] snapshot of volume [KVMPhysicalDisk {\"dispName\":null,\"format\":\"qcow2\",\"name\":\"11806790-0ad0-4a4a-b389-2f7ab41b4e87\",\"path\":\"\\/mnt\\/4c524bab-25a3-3db5-a665-d37159b81f11\\/11806790-0ad0-4a4a-b389-2f7ab41b4e87\",\"pool\":{\"uuid\":\"4c524bab-25a3-3db5-a665-d37159b81f11\",\"path\":\"\\/mnt\\/4c524bab-25a3-3db5-a665-d37159b81f11\"},\"size\":122363461632,\"virtualSize\":161061273600,\"vmName\":null}] to [/mnt/4c524bab-25a3-3db5-a665-d37159b81f11/snapshots/3ae09c67-c072-48b4-a8a7-e3f2dbaf2687] due to [timeout].\n        at com.cloud.hypervisor.kvm.storage.KVMStorageProcessor.validateConvertResult(KVMStorageProcessor.java:1915)\n        at com.cloud.hypervisor.kvm.storage.KVMStorageProcessor.createSnapshot(KVMStorageProcessor.java:1790)\n        at com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.execute(StorageSubsystemCommandHandlerBase.java:140)\n        at com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.handleStorageCommands(StorageSubsystemCommandHandlerBase.java:66)\n        at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtStorageSubSystemCommandWrapper.execute(LibvirtStorageSubSystemCommandWrapper.java:36)\n        at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtStorageSubSystemCommandWrapper.execute(LibvirtStorageSubSystemCommandWrapper.java:30)\n        at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78)\n        at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1930)\n        at com.cloud.agent.Agent.processRequest(Agent.java:683)\n        at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1106)\n        at com.cloud.utils.nio.Task.call(Task.java:83)\n        at com.cloud.utils.nio.Task.call(Task.java:29)\n        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)\n        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)\n        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)\n        at java.base/java.lang.Thread.run(Thread.java:829)\n\n### versions\n\nCloudstack v4.20 / KVM\nS.O ubuntu 24\n\n\n### The steps to reproduce the bug\n\n1. Start a snapshot task for a volume \u003e 200GB\n2. Wait one hour\n3. Error\n\n\n### What to do about it?\n\nComplete configuration to be able to modify this time or fix a bug if necessary.","author":{"url":"https://github.com/Lauta11","@type":"Person","name":"Lauta11"},"datePublished":"2025-10-22T18:01:09.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":8},"url":"https://github.com/11893/cloudstack/issues/11893"}