René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Improve the error message when `build_multivariate_dataframe` has the list of stat_vars more than the batch_size","articleBody":"cc: @shifucun \r\n\r\nI was using a script to `build_multivariate_dataframe` for a stat_var list of length more than 50 and got the following error:\r\n\r\n```shell\r\nTraceback (most recent call last):\r\n  File \"/home/sharadshriram/accessible_charts/datasets/datacommons/get_data.py\", line 88, in \u003cmodule\u003e\r\n    save_statvar_to_csv(place, 'data.csv')\r\n  File \"/home/sharadshriram/accessible_charts/datasets/datacommons/get_data.py\", line 67, in save_statvar_to_csv\r\n    df = dpd.build_multivariate_dataframe([place], stat_vars)\r\n  File \"/home/sharadshriram/env/lib/python3.10/site-packages/datacommons_pandas/df_builder.py\", line 314, in build_multivariate_dataframe\r\n    df = pd.DataFrame.from_records(_multivariate_pd_input(places, stat_vars))\r\n  File \"/home/sharadshriram/env/lib/python3.10/site-packages/datacommons_pandas/df_builder.py\", line 238, in _multivariate_pd_input\r\n    rows_dict = _group_stat_all_by_obs_options(places,\r\n  File \"/home/sharadshriram/env/lib/python3.10/site-packages/datacommons_pandas/df_builder.py\", line 88, in _group_stat_all_by_obs_options\r\n    stat_all = dc.get_stat_all(places, stat_vars)\r\n  File \"/home/sharadshriram/env/lib/python3.10/site-packages/datacommons_pandas/stat_vars.py\", line 226, in get_stat_all\r\n    batches = -(-len(places) // places_per_batch)\r\nZeroDivisionError: integer division or modulo by zero\r\n```\r\n\r\nHowever, `ZeroDivisionError: integer division or modulo by zero` did not help me understand what caused the ZeroDivisionError. After backtracking, I observed the error was caused not because of batching, but because the `len(stat_var)` passed to `dc.get_stat_all(places, stat_vars)` was greater than 50.\r\n\r\nIs it possible for the error message to read out that the length of `stat_var` list passed is more than the batch_size limit of 50?\r\n\r\nI also wonder whether, we can extend the `get_stat_all()` method to chunk long lists of `stat_var` to length 50, and do the API query. Would like to hear your thoughts?","author":{"url":"https://github.com/sharadshriram","@type":"Person","name":"sharadshriram"},"datePublished":"2022-10-13T04:40:12.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":0},"url":"https://github.com/184/api-python/issues/184"}