René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"get_graph() RecursionError: maximum recursion depth exceeded","articleBody":"my conf:\r\nubuntu v 22. lts\r\nDataiku On Premise v 13.1.2\r\n\r\n\r\n### Exemple\r\n\r\nThe `get_graph` method in the `DSSProjectFlow` class does not support flows that have loops, causing a `RecursionError` when trying to get the flow graph for such projects.\r\n\r\nfunction's reference : https://developer.dataiku.com/latest/api-reference/python/flow.html#dataikuapi.dss.flow.DSSProjectFlow.get_graph\r\n\r\n### Steps to Reproduce\r\n\r\n1. Create a project in Dataiku where a recipe has a dataset as output (last stage in the flow) but used as input of another recipe. Both input and output, forming a loop.\r\n2. Use the API to get the flow graph:\r\n   ```python\r\n   import dataiku\r\n \r\n   client = dataiku.api_client()\r\n   project = client.get_project(\"YOUR_PROJECT_KEY\")\r\n   flow = project.get_flow()\r\n   graph = flow.get_graph()  # This line causes the RecursionError below\r\n\r\nDSSProjectFlowGraph.get_items_in_traversal_order.\u003clocals\u003e.add_from(graph_node)\r\n    776     predecessor_node = self.nodes[predecessor_ref]\r\n    777     if not in_set(predecessor_node):\r\n--\u003e 778         add_from(predecessor_node)\r\n    780 # Then add ourselves\r\n    781 if not in_set(graph_node):\r\n\r\nDSSProjectFlowGraph.get_items_in_traversal_order.\u003clocals\u003e.add_from(graph_node)\r\n    775 for predecessor_ref in graph_node[\"predecessors\"]:\r\n    776     predecessor_node = self.nodes[predecessor_ref]\r\n--\u003e 777     if not in_set(predecessor_node):\r\n    778         add_from(predecessor_node)\r\n    780 # Then add ourselves\r\n\r\nDSSProjectFlowGraph.get_items_in_traversal_order.\u003clocals\u003e.in_set(obj)\r\n    767 def in_set(obj):\r\n    768     for candidate in ret:\r\n--\u003e 769         if candidate[\"type\"] == obj[\"type\"] and candidate[\"ref\"] == obj[\"ref\"]:\r\n    770             return True\r\n    771     return False\r\n\r\n**RecursionError: maximum recursion depth exceeded in comparison**\r\n\r\nAs a result, flows using data to rewrite the same directory become problematic only in certain cases\r\noutput dataset \u003e used as intermediate dataset of a recipe upstream of this output dataset. \r\nbut not problematic in other cases, such as;\r\ninput folder \u003e used as output folder at the end of the flow.","author":{"url":"https://github.com/MFE-DSS","@type":"Person","name":"MFE-DSS"},"datePublished":"2024-11-11T14:58:13.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":0},"url":"https://github.com/310/dataiku-api-client-python/issues/310"}