René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Add serialized XML interface to avoid libxml2 ABI conflicts with lxml","articleBody":"`python-xmlsec` currently relies on passing raw `xmlNodePtr` objects between `lxml` (which builds on libxml2) and `xmlsec1` (which also uses libxml2). This creates a fragile situation where different versions of `libxml2` may be loaded into the same process, leading to:\n\n-  Segfaults or memory corruption due to incompatible struct layouts  \n-  Invalid memory free errors (e.g., double-free or mismatched allocators)  \n-  Signature verification failures caused by inconsistent parser state  \n-  Undefined behavior from mismatched `libxml2` global configuration  \n\nThis occurs because:\n\n- `lxml` **bundles its own `libxml2` and `libxslt`** (especially in binary wheels) to ease installation for users on Windows, macOS, and some Linux platforms.\n- `python-xmlsec` binds to **`xmlsec1`**, which in turn links to the **system's `libxml2`**.\n- Pointers like `xmlNodePtr` created by `lxml` are then passed to `python-xmlsec` functions like `tree.find_node()` or `SignatureContext.sign()`.\n\nIf the `libxml2` versions are not ABI-compatible, this can easily lead to crashes, unpredictable behavior, or memory corruption.\n\n---\n\n## Proposed Solution: Decoupling via Canonicalized XML\n\nInstead of passing `xmlNodePtr` from `lxml` to `python-xmlsec`, we should support passing **serialized XML (as `bytes`)**, ideally using [Canonical XML (C14N)](https://www.w3.org/TR/xml-c14n) where appropriate. This isolates the XML parsing and memory management between the two libraries.\n\n### Example Usage\n\n```python\nfrom lxml import etree\nimport xmlsec\n\ndoc = etree.fromstring(\"\u003cRoot\u003e\u003cSignature/\u003e\u003c/Root\u003e\")\nc14n_bytes = etree.tostring(doc, method=\"c14n\", exclusive=True)\n\n# Proposed new API:\nsigned_bytes = xmlsec.sign_serialized(c14n_bytes, key_file=\"key.pem\")\n\n# Parse back with lxml if needed\nsigned_doc = etree.fromstring(signed_bytes)","author":{"url":"https://github.com/mxamin","@type":"Person","name":"mxamin"},"datePublished":"2025-07-10T19:06:31.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":2},"url":"https://github.com/356/python-xmlsec/issues/356"}