René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Fatal Python error: GC object already tracked","articleBody":"When using the library by running up and down the parse tree and extracting used columns and attributes of the parsed query I frequently ran into inexplicable errors and eventually into segmentation faults that crashed my python program. When going for a proof of concept I ended with the following code that only works for me with iPython. However, I am pretty sure that the problem is not iPython but sqlparser as segmentation faults occurred on multiple occasions when using the library and when doing the proof of concept I was able to eliminate everything but sqlparser related code - however the overall error changed to the GC problem.\r\n\r\nI am using:\r\n- an Ubuntu 14.04\r\n- Python 2.7.6 (default, Oct 26 2016, 20:30:19)\r\n- IPython 1.2.1\r\n\r\nThe code:\r\n```\r\nimport sqlparser\r\n\r\ndef analyze_result_column_list(node):\r\n    ret = []\r\n    for subnode in node.list:\r\n        if subnode.expr.leftOperand is not None:\r\n            operand = subnode.expr.leftOperand\r\n            for item in operand.caseExpression.whenClauseItemList:\r\n                ret.append(item.comparison_expr.leftOperand.objectOperand.get_text())\r\n    return ret\r\n\r\n\r\ndef analyze_select_statement(root):\r\n    result_attributes = analyze_result_column_list(root.resultColumnList)\r\n    return [], []\r\n\r\n\r\nclass SqlQuery():\r\n    _query_string = None\r\n    _defines = None\r\n    _uses = None\r\n\r\n    def __init__(self, stmt, vendor=sqlparser.gsp_dbvendor.dbvmysql):\r\n        root = stmt.get_root()\r\n        self._uses, self._defines = analyze_select_statement(root)\r\n\r\n\r\nparser = sqlparser.Parser()\r\nquery = \"SELECT (CASE WHEN password = MD5( 'bitnami ') THEN 1 ELSE 0 END) AS zend_auth_credential_match FROM si_user WHERE (email = 'user@example.com ')\"\r\nparser.check_syntax(query)\r\nstmt = parser.get_statement(0)\r\npq = SqlQuery(stmt)\r\npq._defines\r\npq._uses\r\nprint \"This was a triump,h \\nI make a note here: \\nHUGE SUCCESS.\"\r\n```\r\nRemoving `self._uses, self._defines = analyze_select_statement(root)` leads to the error not manifesting.\r\n\r\nThe bug can be recreated using a fresh virtual environment and then installing nothing but the sqlparser ,as described in the readme. Then do the following steps:\r\n1. activate the virtenv\r\n2. start IPython\r\n3. mark and copy the above code\r\n4. type %paste\r\n```\r\nPython 2.7.6 (default, Oct 26 2016, 20:30:19) \r\nType \"copyright\", \"credits\" or \"license\" for more information.\r\n\r\nIPython 1.2.1 -- An enhanced Interactive Python.\r\n?         -\u003e Introduction and overview of IPython's features.\r\n%quickref -\u003e Quick reference.\r\nhelp      -\u003e Python's own help system.\r\nobject?   -\u003e Details about 'object', use 'object??' for extra details.\r\n\r\nIn [1]: %paste\r\nimport sqlparser\r\n\r\ndef analyze_result_column_list(node):\r\n    ret = []\r\n    for subnode in node.list:\r\n        if subnode.expr.leftOperand is not None:\r\n            operand = subnode.expr.leftOperand\r\n            for item in operand.caseExpression.whenClauseItemList:\r\n                ret.append(item.comparison_expr.leftOperand.objectOperand.get_text())\r\n    return ret\r\n\r\n\r\ndef analyze_select_statement(root):\r\n    result_attributes = analyze_result_column_list(root.resultColumnList)\r\n    return [], []\r\n\r\n\r\nclass SqlQuery():\r\n    _query_string = None\r\n    _defines = None\r\n    _uses = None\r\n\r\n    def __init__(self, stmt, vendor=sqlparser.gsp_dbvendor.dbvmysql):\r\n        root = stmt.get_root()\r\n        self._uses, self._defines = analyze_select_statement(root)\r\n\r\n\r\nparser = sqlparser.Parser()\r\nquery = \"SELECT (CASE WHEN password = MD5( 'bitnami ') THEN 1 ELSE 0 END) AS zend_auth_credential_match FROM si_user WHERE (email = 'user@example.com ')\"\r\nparser.check_syntax(query)\r\nstmt = parser.get_statement(0)\r\npq = SqlQuery(stmt)\r\npq._defines\r\npq._uses\r\n\r\n## -- End pasted text --\r\nFatal Python error: GC object already tracked\r\nAborted (core dumped)\r\n```\r\n\r\nIf any additional feedback/versions/etc. are needed please notify me as I am happy to help in any way possible to get to the root of this issue.","author":{"url":"https://github.com/simkoc","@type":"Person","name":"simkoc"},"datePublished":"2017-05-30T18:46:44.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":0},"url":"https://github.com/11/python-sqlparser/issues/11"}