René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"[REQUEST] Refactor CLI and main modules","articleBody":"**Type of Request**\r\n  - [ ] Functionality\r\n  - [ ] UX/UI\r\n  - [ ] Documentation\r\n  - [x] Quality of Life\r\n  - [ ] Other\r\n\r\nFor context, the run-time argument parsing is pretty tightly associated with the actual modules. For example, as is, it's passing around an `args` argument for most function calls. I picked this habit up from a project in the Baxter lab. It's not awful and it makes it a lot neater when you have a lot of shared parameters (e.g., `verbose` flag).\r\n\r\nPros:\r\n* Fewer parameters for function definitions and calls\r\n\r\nCons:\r\n* It's effectively a global state object without actually being one\r\n* It's difficult to directly call or work with the core functionality without making a dummy argument parser\r\n* It's difficult to use modules in a standalone way (e.g., `python -m rawtools.raw2img`)\r\n\r\nFor these reasons, I think the CLI module and individual modules should be refactored to pull the core of the CLI into the module themselves. It's probably easier to explain with an example.\r\n\r\nLet's look at the current argparser in cli.py for raw2img:\r\nhttps://github.com/Topp-Roots-Lab/python-rawtools/blob/2a042b160e50173b87c6b20269118501b50bfaee/rawtools/cli.py#L110-L132\r\n\r\nThis should be moved into the separate function within `raw2img.py`, probably named `main`.\r\nSo the entry point for raw2img should change from `raw2img=rawtools.cli:raw_image` to `raw2img=rawtools.raw2img:main`.\r\n\r\nThis `raw2img.main` would look something along the lines of the following\r\n```python\r\ndef main():\r\n    start_time = time()\r\n    \r\n    description='Convert .raw 3d volume file to typical image format slices'\r\n    parser = argparse.ArgumentParser(description=description,formatter_class=argparse.ArgumentDefaultsHelpFormatter)\r\n    parser.add_argument(\"-v\", \"--verbose\", action=\"store_true\", help=\"Increase output verbosity\")\r\n    parser.add_argument(\"-V\", \"--version\", action=\"version\", version=f'%(prog)s {__version__}')\r\n    parser.add_argument(\"-t\", \"--threads\", type=int, default=cpu_count(), help=f\"Maximum number of threads dedicated to processing.\")\r\n    parser.add_argument(\"-f\", '--force', action=\"store_true\", help=\"Force file creation. Overwrite any existing files.\")\r\n    parser.add_argument(\"-n\", '--dry-run', dest='dryrun', action=\"store_true\", help=\"Perform a trial run. Do not create image files, but logs will be updated.\")\r\n    parser.add_argument(\"--format\", default='png', help=\"Set image filetype. Available options: ['png', 'tif']\")\r\n    parser.add_argument(\"path\", metavar='PATH', type=str, nargs=1, help='Input directory to process')\r\n    args = parser.parse_args()\r\n\r\n    # Make sure user does not request more CPUs can available\r\n    if args.threads \u003e cpu_count():\r\n        args.threads = cpu_count()\r\n\r\n    # Change format to always be lowercase\r\n    args.format = args.format.lower()\r\n    args.path = list(set(args.path)) # remove any duplicates\r\n\r\n    args.module_name = 'raw2img'\r\n    log.configure(args)\r\n\r\n    # Collect all volumes and validate their metadata\r\n    try:\r\n        # Gather all files\r\n        args.files = []\r\n        for p in args.path:\r\n            for root, dirs, files in os.walk(p):\r\n                for filename in files:\r\n                    args.files.append(os.path.join(root, filename))\r\n\r\n        # Append any loose, explicitly defined paths to .RAW files\r\n        args.files.extend([f for f in args.path if f.endswith('.raw')])\r\n\r\n        # Get all RAW files\r\n        args.files = [f for f in args.files if f.endswith('.raw')]\r\n        logging.debug(f\"All files: {args.files}\")\r\n        args.files = list(set(args.files))  # remove duplicates\r\n        logging.info(f\"Found {len(args.files)} volume(s).\")\r\n        logging.debug(f\"Unique files: {args.files}\")\r\n\r\n        # Validate that a DAT file exists for each volume\r\n        for fp in args.files:\r\n            dat_fp = f\"{os.path.splitext(fp)[0]}.dat\"  # .DAT filepath\r\n            logging.debug(f\"Validating DAT file: '{dat_fp}'\")\r\n            # Try to extract the dimensions to make sure that the file exists\r\n            dat.read(dat_fp)\r\n    except Exception as err:\r\n        logging.error(err)\r\n    else:\r\n        # For each provided volume...\r\n        pbar = tqdm(total=len(args.files), desc=f\"Overall progress\")\r\n        for fp in args.files:\r\n            logging.debug(f\"Processing '{fp}'\")\r\n            # Extract slices for all volumes in provided folder\r\n            extract_slices(args, fp)\r\n            pbar.update()\r\n        pbar.close()\r\n\r\n    logging.debug(f'Total execution time: {time() - start_time} seconds')\r\n\r\nif __name__ == \"__main__\":\r\n    main()\r\n```\r\n\r\n\u003e**Note**\r\n\u003e I added in an _if-main-name_ block to call the newly defined `main` function.\r\n\r\nA side benefit of this is fewer imports to run each module, so it should be much quicker to execute the command at run-time.","author":{"url":"https://github.com/tparkerd","@type":"Person","name":"tparkerd"},"datePublished":"2022-10-16T18:12:30.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":0},"url":"https://github.com/31/python-rawtools/issues/31"}