René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Intermittent Template Download Failure  (NOT_DOWNLOADED) in CloudStack 4.20.1.0 with Linstor Primary Storage","articleBody":"Hi all,\n\nI’m encountering an intermittent issue when uploading templates to CloudStack 4.20.1.0 where Linstor is used as Primary Storage. Here’s the detailed breakdown:\n\n\nCloudStack Version: 4.20.1.0\nPrimary Storage:\nLinstor-controller 1.32.3-1ppa1-noble1\nLinstor-satellite 1.32.3-1ppa1-noble1\nDRBD_KERNEL_VERSION=9.2.14\nSystem: Ubuntu 24.04.3 LTS\nHypervisor: KVM\n\nWhen uploading a template (e.g., an Ubuntu 22.04 qcow2 image), the process intermittently fails with NOT_DOWNLOADED in Primary Storage. In my tests, 2 out of 3 upload attempts fail, while 1 succeeds randomly.\n\n\n\nTwo critical symptoms:\n1. qemu-img Binary Not Found: The CloudStack agent not  execute qemu-img convert (verified via strace ):\nps -ef | grep cloudstack-agent\nstrace -f -s 256 -p 1318808 -o /tmp/agent_trace.log\n\ncan't find execute qemu-img convert like this:\n   ```\nexecve(\"/usr/local/sbin/qemu-img\", [\"qemu-img\", \"convert\", \"-n\", \"--target-is-zero\", \"-W\", \"-S\", \"1M\", \"-O\", \"raw\", \"-t\", \"none\", \"-U\", \"--image-opts\", \"driver=qcow2,file.filename=/mnt/.../xxx.qcow2\", \"/dev/drbd1105\"], ...) = -1 ENOENT (No such file or directory)\n   ```\n\n\nDRBD Devices Stay “Unused”: Linstor resource listings show DRBD devices as Unused (instead of one of device InUse):\n\n   ```\n   root@NODE76:~# linstor v l |grep  80bc7788-638e-4211-9b31-d76d037b50b3\n   | cs-80bc7788-638e-4211-9b31-d76d037b50b3    | NODE158 | DfltDisklessStorPool |     0 |    1105 | /dev/drbd1105 |            | Unused |   Diskless | Established(2) |\n   | cs-80bc7788-638e-4211-9b31-d76d037b50b3    | NODE76  | x86_pool_zfs_ssd     |     0 |    1105 | /dev/drbd1105 |  20.00 GiB | Unused |   UpToDate | Established(2) |\n   | cs-80bc7788-638e-4211-9b31-d76d037b50b3    | NODE83  | x86_pool_zfs_ssd     |     0 |    1105 | /dev/drbd1105 |  20.00 GiB | Unused |   UpToDate | Established(2) |\n   ```\n\n\nSteps to Reproduce:\n1. Upload a qcow2 template (e.g., `ubuntu-22.04.4-amd64-base-UEFI.qcow2`) to CloudStack.\n2. Monitor the Primary Storage (Linstor) download status.\n3. Repeat the upload 3+ times and observe intermittent `NOT_DOWNLOADED` failures.\n\n\nExpected Behavior:\nThe template should download to Primary Storage successfully every time, with `qemu-img convert` executing properly and DRBD devices showing `InUse`.\n\n\nActual Behavior:\n- Intermittent failures (2 out of 3 attempts fail).\n- DRBD resources remain `Unused` during failed attempts.\n\n\nQuestions:\n- What configuration or troubleshooting steps should I take to resolve this intermittent behavior?\n\n\nAny insights or guidance would be greatly appreciated!\n\nThanks,\n\n\u003cimg width=\"1598\" height=\"865\" alt=\"Image\" src=\"https://github.com/user-attachments/assets/12b26478-f480-4201-90ab-407f5fd1522b\" /\u003e","author":{"url":"https://github.com/gw769","@type":"Person","name":"gw769"},"datePublished":"2025-10-24T07:59:45.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":16},"url":"https://github.com/11905/cloudstack/issues/11905"}