René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Local Template/ISO Upload Fails - SSVM receives SIGNATURE=null","articleBody":"### problem\n\n**Related Issue**\nThis is related to #12486 but represents a distinct root cause.\n\n**Summary**\nWhen uploading templates/ISOs from local file via the UI, the SSVM sometimes receives requests with SIGNATURE=null instead of the valid signature. This causes immediate validation failure with HTTP 432 error, and the upload never completes.\nThe root cause (whether the browser fails to send the header, or something strips it in transit) requires further investigation.\n\n\n\n### versions\n\n- Management Server: Rocky Linux 8 / RHEL based\n- SSVM: Debian-based SystemVM template 4.20.2.0\n- Browser: Chrome Version 144.0.7559.59 (Official Build) (64-bit)\n\n\n### The steps to reproduce the bug\n\n1.  Navigate to Images → Templates (or ISOs)\n2.  Click \"Upload Template from local\"\n4.  Fill in template details (name, hypervisor=KVM, format=QCOW2, etc.)\n5.  Select a file and submit\n6.  Observe HTTP 432 error in browser console\n7.  Check SSVM logs - may show SIGNATURE=null\n\nNote: This issue is intermittent and difficult to reproduce consistently.\n\n\u003cimg width=\"3132\" height=\"1861\" alt=\"Image\" src=\"https://github.com/user-attachments/assets/52e44d33-c3b5-4bf3-98bb-badc8e95ab3b\" /\u003e\n\n**Expected Behavior**\nThe SSVM should receive valid SIGNATURE, METADATA, and EXPIRES headers with every upload request.\n\n**Actual Behavior**\nThe SSVM sometimes receives SIGNATURE=null, causing immediate validation failure.\n\n**Workaround**\n\n- Retry the upload (may work on subsequent attempts)\n- Use URL-based template registration instead of local upload\n\n**Analysis**\nThe SSVM definitively received requests with SIGNATURE=null. Further investigation is needed to determine where in the chain the signature is lost:\n\n- Browser fails to attach the header\n- Proxy/load balancer strips the header\n- Apache on SSVM fails to forward the header\n- Race condition in UI JavaScript code\n\nThe issue is intermittent, which suggests a timing or race condition.\n\n**Evidence**\n\n- SSVM Logs (/var/log/cloud.log) - Shows SIGNATURE=null received\n\n```\n2026-01-21T15:35:12,754 INFO  [storage.resource.HttpUploadServerHandler] (nioEventLoopGroup-3-2:[]) HEADER: SIGNATURE=null\n2026-01-21T15:35:12,779 ERROR [storage.resource.HttpUploadServerHandler] (nioEventLoopGroup-3-2:[]) post request validation failed com.cloud.exception.InvalidParameterValueException: signature, metadata and expires are compulsory fields.\n2026-01-21T15:35:18,548 INFO  [storage.resource.HttpUploadServerHandler] (nioEventLoopGroup-3-3:[]) HEADER: SIGNATURE=null\n2026-01-21T15:35:18,554 ERROR [storage.resource.HttpUploadServerHandler] (nioEventLoopGroup-3-3:[]) post request validation failed com.cloud.exception.InvalidParameterValueException: signature, metadata and expires are compulsory fields.\n2026-01-21T15:35:49,917 INFO  [storage.resource.HttpUploadServerHandler] (nioEventLoopGroup-3-4:[]) HEADER: SIGNATURE=null\n2026-01-21T15:35:49,923 ERROR [storage.resource.HttpUploadServerHandler] (nioEventLoopGroup-3-4:[]) post request validation failed com.cloud.exception.InvalidParameterValueException: signature, metadata and expires are compulsory fields.\n```\n\n- Browser Console Error\n\n```\nFailed to load resource: the server responded with a status of 432 (432)\nError: Request failed with status code 432\n```\n\n- Management Server Logs - Misleading Error Message\n\nThe management server shows a misleading \"Couldn't find ssvm\" error even though the SSVM is running and healthy:\n\n```\n2026-01-21 15:22:16,171 DEBUG [o.a.c.s.e.DefaultEndPointSelector] (Upload-Monitor-1:[ctx-e41f2a9e]) (logid:0531e27c) Couldn't find ssvm for urlhttps://10.0.56.181/upload/78e32075-e9e2-4d91-b40b-6b6f65e59165\n2026-01-21 15:23:16,255 DEBUG [o.a.c.s.e.DefaultEndPointSelector] (Upload-Monitor-1:[ctx-0c36c949]) (logid:1e37cd87) Couldn't find ssvm for urlhttps://10.0.56.181/upload/78e32075-e9e2-4d91-b40b-6b6f65e59165\n2026-01-21 15:24:16,316 DEBUG [o.a.c.s.e.DefaultEndPointSelector] (Upload-Monitor-1:[ctx-b52f89e1]) (logid:0587fffd) Couldn't find ssvm for urlhttps://10.0.56.181/upload/78e32075-e9e2-4d91-b40b-6b6f65e59165\n```\n\n- Database Verification - SSVM was Running and Up\n\n```\nmysql\u003e SELECT id, uuid, name, public_ip_address, private_ip_address, status FROM host WHERE type='SecondaryStorageVM';\n+----+--------------------------------------+--------+-------------------+--------------------+--------+\n| id | uuid                                 | name   | public_ip_address | private_ip_address | status |\n+----+--------------------------------------+--------+-------------------+--------------------+--------+\n|  4 | 2cf2f703-cb23-4ed1-909e-4ad47ed22919 | s-1-VM | 10.0.56.181       | 10.0.40.156        | Up     |\n+----+--------------------------------------+--------+-------------------+--------------------+--------+\n```\n\n- Template State After Failed Upload\n\n```\n{\n  \"id\": \"78e32075-e9e2-4d91-b40b-6b6f65e59165\",\n  \"name\": \"Linux Debian 12\",\n  \"isready\": false,\n  \"status\": \"NotUploaded\",\n  \"downloaddetails\": []\n}\n```\n\n- No Files Created on SSVM:\n\n```\nroot@s-1-VM:~# ls -la /mnt/SecStorage/5f75abe3-3803-3595-89ad-52d23b3dea97/template/tmpl/2/202/\nls: cannot access '/mnt/SecStorage/5f75abe3-3803-3595-89ad-52d23b3dea97/template/tmpl/2/202/': No such file or directory\n\nroot@s-1-VM:~# grep -i \"78e32075-e9e2-4d91-b40b-6b6f65e59165\" /var/log/cloud.log \n(no output - template UUID never reached SSVM)\n```\n\n### What to do about it?\n\n_No response_","author":{"url":"https://github.com/RosiKyu","@type":"Person","name":"RosiKyu"},"datePublished":"2026-01-21T18:06:04.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":1},"url":"https://github.com/12488/cloudstack/issues/12488"}