René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Run queries concurrently","articleBody":"```\n\tasync def executeQueriesAsync(self, queries):\n\t\t\"\"\"Executes multiple StackQL queries asynchronously using the current StackQL instance.\n\n\t\tThis method utilizes an asyncio event loop to concurrently run a list of provided \n\t\tStackQL queries. Each query is executed independently, and the combined results of \n\t\tall the queries are returned as a list of JSON objects if 'dict' output mode is selected,\n\t\tor as a concatenated DataFrame if 'pandas' output mode is selected.\n\n\t\tThe order of the results in the returned list or DataFrame may not necessarily \n\t\tcorrespond to the order of the queries in the input list due to the asynchronous nature \n\t\tof execution.\n\n\t\t:param queries: A list of StackQL query strings to be executed concurrently.\n\t\t:type queries: list[str], required\n\t\t:return: A list of results corresponding to each query. Each result is a JSON object or a DataFrame.\n\t\t:rtype: list[dict] or pd.DataFrame\n\t\t:raises ValueError: If method is used in `server_mode` on an unsupported OS (anything other than Linux).\n\t\t:raises ValueError: If an unsupported output mode is selected (anything other than 'dict' or 'pandas').\n\n\t\tExample:\n\t\t\t\u003e\u003e\u003e from pystackql import StackQL\n\t\t\t\u003e\u003e\u003e stackql = StackQL()\t\t\n\t\t\t\u003e\u003e\u003e queries = [\n\t\t\t\u003e\u003e\u003e \\\"\\\"\\\"SELECT '%s' as region, instanceType, COUNT(*) as num_instances \n\t\t\t... FROM aws.ec2.instances \n\t\t\t... WHERE region = '%s' \n\t\t\t... GROUP BY instanceType\\\"\\\"\\\" % (region, region)\n\t\t\t\u003e\u003e\u003e for region in regions ]\n\t\t\t\u003e\u003e\u003e result = stackql.executeQueriesAsync(queries)\n\n\t\tNote:\n\t\t\t- When operating in `server_mode`, this method is not supported.\n\t\t\"\"\"\n\t\tif self.server_mode:\n\t\t\traise ValueError(\"executeQueriesAsync are not supported in sever_mode.\")\n\t\tif self.output not in ['dict', 'pandas']:\n\t\t\traise ValueError(\"executeQueriesAsync supports only 'dict' or 'pandas' output modes.\")\n\t\tasync def main():\n\t\t\twith ThreadPoolExecutor() as executor:\n\t\t\t\t# New connection is created for each query in server_mode, reused otherwise.\n\t\t\t\tnew_connection = self.server_mode\n\t\t\t\t# Gather results from all the async calls.\n\t\t\t\tloop = asyncio.get_event_loop()\n\t\t\t\tfutures = [loop.run_in_executor(executor, self._sync_query, query, new_connection) for query in queries]\n\t\t\t\tresults = await asyncio.gather(*futures)\n\t\t\t# Concatenate DataFrames if output mode is 'pandas'.\n\t\t\tif self.output == 'pandas':\n\t\t\t\treturn pd.concat(results, ignore_index=True)\n\t\t\telse:\n\t\t\t\treturn [item for sublist in results for item in sublist]\n\t\t# Running the async function\n\t\treturn await main()\n```","author":{"url":"https://github.com/yunchengyang515","@type":"Person","name":"yunchengyang515"},"datePublished":"2023-12-09T12:57:12.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":0},"url":"https://github.com/19/stackqljs/issues/19"}