René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"C API: Consider adding a public PyList_Extend() function","articleBody":"# Feature or enhancement\n\nThe private `_PyList_Extend()` function has been removed in Python 3.13: see PR #108451.\n\n@scoder [asked](https://github.com/python/cpython/pull/108451#issuecomment-1772792843) what is the replacement for this removed function.\n\nThe obvious replacement is ``PyObject_CallMethod(list, \"extend\", \"O\", arg)``: call the `list.extend()` method. But it's slower, `PyObject_CallMethod()` has to get the method and decode the \"O\" format string.\n\nAnother replacement is ``PyList_SetSlice(L, PY_SSIZE_T_MAX, PY_SSIZE_T_MAX, arg)`` which is less straightforward, but it's efficient.\n\nI propose adding a public PyList_Extend() function. The list type is commonly used in C extensions, it's a convenient API to create a collection when the length is not known is advance.\n\nI don't think that performance is really the problem here since ``PyList_SetSlice(L, PY_SSIZE_T_MAX, PY_SSIZE_T_MAX, arg)`` is available. The problem is more to more the function easier to discover and make the API easier to use. Also, it should help people to migrate away from the removed function.\n\nThere is already a public **PyDict_Update()** API for the dict type, it is part of the limited C API.\n\nIf we add a function for the list type, should we also add **PySet_Update()** \"for completeness\"? The private `_PySet_Update()` was removed in Python 3.13.\n\ncc @serhiy-storchaka @encukou \n\n---\n\nA code search on ``_PyList_Extend`` in PyPI top 5,000 projects (2023-07-04) found 4 projects using this function:\n\n* Cython (0.29.36)\n* catboost (1.2)\n* mypy (1.4.1)\n* pyrsistent (0.19.3)\n\nLogs:\n\n\u003cdetails\u003e\n\n```\nPYPI-2023-07-04/catboost-1.2.tar.gz: catboost-1.2/catboost_all_src/contrib/tools/cython/Cython/Compiler/Builtin.py: BuiltinMethod(\"extend\",  \"TO\",   \"r\", \"__Pyx_PyList_Extend\",\nPYPI-2023-07-04/catboost-1.2.tar.gz: catboost-1.2/catboost_all_src/contrib/tools/cython/Cython/Compiler/ExprNodes.py: extend_func = \"__Pyx_PyList_Extend\"\nPYPI-2023-07-04/catboost-1.2.tar.gz: catboost-1.2/catboost_all_src/contrib/tools/cython/Cython/Utility/Optimize.c: static CYTHON_INLINE int __Pyx_PyList_Extend(PyObject* L, PyObject* v) {\nPYPI-2023-07-04/catboost-1.2.tar.gz: catboost-1.2/catboost_all_src/contrib/tools/cython/Cython/Utility/Optimize.c: PyObject* none = _PyList_Extend((PyListObject*)L, v);\n\nPYPI-2023-07-04/Cython-0.29.36.tar.gz: Cython-0.29.36/Cython/Compiler/Builtin.py: BuiltinMethod(\"extend\",  \"TO\",   \"r\", \"__Pyx_PyList_Extend\",\nPYPI-2023-07-04/Cython-0.29.36.tar.gz: Cython-0.29.36/Cython/Compiler/ExprNodes.py: extend_func = \"__Pyx_PyList_Extend\"\nPYPI-2023-07-04/Cython-0.29.36.tar.gz: Cython-0.29.36/Cython/Utility/Optimize.c: static CYTHON_INLINE int __Pyx_PyList_Extend(PyObject* L, PyObject* v) {\nPYPI-2023-07-04/Cython-0.29.36.tar.gz: Cython-0.29.36/Cython/Utility/Optimize.c: PyObject* none = _PyList_Extend((PyListObject*)L, v);\n\nPYPI-2023-07-04/mypy-1.4.1.tar.gz: mypy-1.4.1/mypyc/lib-rt/dict_ops.c: PyObject *res = _PyList_Extend((PyListObject *)list, view);\nPYPI-2023-07-04/mypy-1.4.1.tar.gz: mypy-1.4.1/mypyc/lib-rt/dict_ops.c: PyObject *res = _PyList_Extend((PyListObject *)list, view);\nPYPI-2023-07-04/mypy-1.4.1.tar.gz: mypy-1.4.1/mypyc/lib-rt/dict_ops.c: PyObject *res = _PyList_Extend((PyListObject *)list, view);\nPYPI-2023-07-04/mypy-1.4.1.tar.gz: mypy-1.4.1/mypyc/lib-rt/list_ops.c: return _PyList_Extend((PyListObject *)o1, o2);\n\nPYPI-2023-07-04/pyrsistent-0.19.3.tar.gz: pyrsistent-0.19.3/pvectorcmodule.c: PyObject *retVal = _PyList_Extend((PyListObject *)self-\u003eappendList, args);\n```\n\n\u003c/details\u003e\n\nCython uses the function to implement its own __Pyx_PyList_Extend() API:\n\n```c\nstatic CYTHON_INLINE int __Pyx_PyList_Extend(PyObject* L, PyObject* v) {\n#if CYTHON_COMPILING_IN_CPYTHON\n    PyObject* none = _PyList_Extend((PyListObject*)L, v);\n    if (unlikely(!none))\n        return -1;\n    Py_DECREF(none);\n    return 0;\n#else\n    return PyList_SetSlice(L, PY_SSIZE_T_MAX, PY_SSIZE_T_MAX, v);\n#endif\n}\n```\n\n\u003c!-- gh-linked-prs --\u003e\n### Linked PRs\n* gh-111582\n* gh-111862\n\u003c!-- /gh-linked-prs --\u003e\n","author":{"url":"https://github.com/vstinner","@type":"Person","name":"vstinner"},"datePublished":"2023-10-20T21:13:41.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":8},"url":"https://github.com/111138/cpython/issues/111138"}