René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"FEAT: Overall progress bar for multi-file downloads   ","articleBody":" # Feature Request                                                                                                         \n                                                                                                                              \n    **Describe the feature you'd like**                                                                                       \n    When downloading a group, artifact, or collection — which can contain dozens or                                           \n    hundreds of files — the CLI currently shows an individual `tqdm` progress bar for                                         \n    each file but gives no indication of overall progress.                                                                    \n                                                                                                                              \n    Add a second, outer progress bar that tracks overall batch progress:                                                      \n                                                                                                                              \n                                                                                                                              \n  Downloading group: dbpedia/mappings                                                                                         \n  Overall  [=====\u003e          ]  12/47 files  |  340 MiB / 1.2 GiB                                                              \n  Current  [=========\u003e      ]  file.ttl.bz2  87 MiB  [12.3 MiB/s]                                                             \n                                                                                                                              \n                                                                                                                              \n    **Why is this feature important?**                                                                                        \n    - When downloading a large collection (e.g. `dbpedia-snapshot-2022-12` with 100+                                          \n      files and several GiB of data), the user has no way to know how much is left or                                         \n      how long it will take.                                                                                                  \n    - The current experience shows one bar per file that disappears after each download,                                      \n      making it feel like an endless sequence with no end in sight.                                                           \n    - An overall counter (e.g. `12/47 files`) gives users enough context to leave the                                         \n      process running with confidence.                                                                                        \n                                                                                                                              \n    **Describe the proposed implementation**                                                                                  \n    The `_download_files()` function in `download.py` iterates over a list of URLs.                                           \n    Wrapping that loop with a `tqdm` outer bar requires minimal change:                                                       \n                                                                                                                              \n    ```python                                                                                                                 \n    from tqdm import tqdm                                                                                                     \n                                                                                                                              \n    def _download_files(urls, localDir, ...):                                                                                 \n        with tqdm(total=len(urls), desc=\"Overall\", unit=\"file\", position=0) as overall:                                       \n            for url in urls:                                                                                                  \n                _download_file(url, localDir, ...)                                                                            \n                overall.update(1)                                                                                             \n                                                                                                                              \n  The per-file  tqdm  bar inside  _download_file  would use  position=1  to render                                            \n  below the overall bar without overwriting it.                                                                               \n                                                                                                                              \n  Optionally, if file sizes are known upfront (e.g. from JSON-LD  byteSize  fields),                                          \n  the overall bar can also show bytes:                                                                                        \n                                                                                                                              \n    Overall  [=====\u003e    ]  12/47 files  |  340 MiB / 1.2 GiB                                                                  \n                                                                                                                              \n  Describe alternatives you've considered                                                                                     \n                                                                                                                              \n  • Printing  \"Downloading file N of M: ...\"  as plain text before each file — simple                                         \n  to implement but less visually clear than a progress bar.                                                                   \n  • Suppressing per-file bars and only showing the overall bar — loses per-file speed                                         \n  and ETA information which is useful for large individual files.                                                             \n                                                                                                                              \n  Additional context                                                                                                          \n                                                                                                                              \n  •  tqdm  is already a dependency ( tqdm = \"^4.42.1\"  in  pyproject.toml ) so no                                             \n  new dependencies are needed.                                                                                                \n  • Affected functions:  _download_files() ,  _download_collection() ,                                                        \n   _download_artifact() ,  _download_group()  — all in  download.py .                                                         \n  • The  tqdm   position  parameter supports nested bars natively.                                                            \n                                                       ","author":{"url":"https://github.com/Prakhar54-byte","@type":"Person","name":"Prakhar54-byte"},"datePublished":"2026-07-05T20:09:50.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":1},"url":"https://github.com/71/databus-python-client/issues/71"}