René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"naturalsize np.int32 multiplication overflow","articleBody":"### What did you do?\r\n\r\nI was using the torchio library, which relies on humanize to return the memory size of image arrays. The images I processed had dimensions 512x512x166, with each pixel being a 32-bit (4-byte) integer.\r\n### What did you expect to happen?\r\n\r\ntorchio calls the naturalsize function with the occupied memory size in bytes and sets binary=True. Manually calculating the expected value:\r\n512×512×166×4÷(1024×1024)=166 MiB\r\n\r\nSo, I expected the function to return approximately 166 MiB.\r\n### What actually happened?\r\n\r\nInstead, the returned value was -2 MiB, accompanied by this warning:\r\n```\r\nRuntimeWarning: overflow encountered in scalar multiply\r\nret: str = format % ((base * bytes_ / unit)) + s\r\n``` \r\n\r\nThis overflow occurs because the input to the function was of type np.int32 instead of Python's native int. Since np.int32 has a maximum value of 2^31−1, the multiplication of base and bytes_ results in an overflow.\r\n### Steps to Reproduce\r\n\r\n```python\r\n\r\nimport humanize\r\nimport numpy as np\r\n\r\nprint(humanize.naturalsize(512*512*166*4, binary=True))  \r\n# Expected: 166.0 MiB\r\n# Works as expected with Python’s built-in int type\r\n\r\nprint(humanize.naturalsize(np.int32(512*512*166*4), binary=True))\r\n# Returns: -2.0 MiB\r\n# RuntimeWarning: overflow encountered in scalar multiply ret: str = format % ((base * bytes_ / unit)) + s\r\n```\r\n\r\n### Proposed Solutions\r\n\r\n1) Change the Order of Operations\r\nAdjusting the order of operations can avoid overflow. In the current line:\r\n\r\n```python\r\n\r\nret: str = format % ((base * bytes_ / unit)) + s\r\n```\r\n\r\nwhen bytes_ is np.int32, multiplying base and bytes_ produces an np.int32 result, which overflows before it’s divided by unit. By dividing either base or bytes_ by unit before the multiplication, each sub-operation remains a float:\r\n\r\n```python\r\n\r\nret: str = format % ((base * (bytes_ / unit))) + s\r\n# or\r\nret: str = format % ((base / unit * bytes_)) + s\r\n```\r\n2) Convert Input to Float\r\n Alternatively, cast value to float without checking its type. Currently, the casting applies only if value is a string:\r\n\r\n```python\r\n\r\n# Current approach\r\nif isinstance(value, str):\r\n    bytes_ = float(value)\r\nelse:\r\n    bytes_ = value\r\n```\r\nUpdating it to cast all inputs to float could resolve the issue:\r\n\r\n```python\r\n\r\nbytes_ = float(value)\r\n```\r\nI haven’t created a pull request since I’m unsure if you’d prefer developers to ensure input compatibility or handle this within the function.\r\nEnvironment\r\n\r\n    OS: Windows 11\r\n    Python: 3.12.1\r\n    Humanize: 4.11.0\r\n    Numpy: 1.26.3","author":{"url":"https://github.com/Toprak2","@type":"Person","name":"Toprak2"},"datePublished":"2024-11-05T19:01:52.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":3},"url":"https://github.com/217/humanize/issues/217"}