René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"How to properly hold onto an object in embedded mode?","articleBody":"Hello! First, I'd like to thank you for the amazing work on this library. It's a mind-bending level of inter-language interaction, IMO. 👏\r\n\r\nI was experimenting with it in the embedded mode. Not sure if I was using it in the intended way, but... I took a considerably sized web-app and tried to add some kind of a debugging/introspection interface to it with clj \u0026 cljs.\r\n\r\nThe app uses gevent internally for non-blocking IO operations, so the setup was a bit involving, but in the end it worked out. I'll explain it to get a better picture:\r\n\r\n* app starts as a Python process\r\n* gevent is called at the earliest to do its monkey-patching of IO\r\n* python app continues it's startup\r\n* separate python thread (real OS thread) is started, which does all JVM/clojure initialization (one more thread I guess) and than calls `-main` of the clojure's part of the app\r\n* in the main thread of the python app a queue is created and worker greenlet (fake thread or coroutine) is started to watch it\r\n* when clojure world needs to call something from python world, that involves gevent, it puts a task into a queue for the worker to execute it on the main thread\r\n* if task needs to return something to clojure world, a one-off queue is created to transfer the return value\r\n\r\nIt worked fine, I've even made a little introspection tool, that can peer into a running python app vars and call functions, toggle stuff and so on.\r\n\r\nThen, I've tried to implement a `tap\u003e` like system (just putting python objects into an atom for later inspection), to send info from python to clojure... And was abruptly stopped by a SIGSEGV. :(\r\n\r\n```\r\n#\r\n# A fatal error has been detected by the Java Runtime Environment:\r\n#\r\n#  SIGSEGV (0xb) at pc=0x0000aaaac541ef74, pid=1685, tid=1869\r\n#\r\n# JRE version: OpenJDK Runtime Environment (17.0.12+7) (build 17.0.12+7-Ubuntu-1ubuntu222.04)\r\n# Java VM: OpenJDK 64-Bit Server VM (17.0.12+7-Ubuntu-1ubuntu222.04, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, g1 gc, linux-aarch64)\r\n# Problematic frame:\r\n# C  [python3+0xcef74]\r\n#\r\n# No core dump will be written. Core dumps have been disabled. To enable core dumping, try \"ulimit -c unlimited\" before starting Java again\r\n#\r\n# An error report file with more information is saved as:\r\n# /var/www/hs_err_pid1685.log\r\n#\r\n# If you would like to submit a bug report, please visit:\r\n#   https://bugs.launchpad.net/ubuntu/+source/openjdk-17\r\n# The crash happened outside the Java Virtual Machine in native code.\r\n# See problematic frame for where to report the bug.\r\n#\r\n```\r\n\r\nIt seems that objects I send through \"tap\" got GC'ed on python part. I've tried to find something about holding to objects in the documentation, but nothing was particularly fitting to my case. Digging through the code, I've stumbled upon `track-pyobject` and `incref-and-track`. Using the latter on the objects I send to clojure world did not help though... I still got a SIGSEGV, but in different internal python function. To validate my assumption about objects being GD'ed, I've implemented a dumb \"borrowing\" mechanics on python side (putting objects, to be sent to clojure, in a dict) an it worked okay.\r\n\r\nI'd prefer not to do `-\u003ejvm` on objects, because the idea is to use them again in a python app to replay some actions that it performs.\r\n\r\nMy question: Is there a way to hold onto python objects from clojure in embedded mode so that refcounts will be decreased when references to the pyobjs are GC'ed on the JVM?\r\n\r\nThank you for your time :)\r\n\r\nPython 3.10.12\r\nClojure 1.12\r\nopenjdk 17.0.12 2024-07-16\r\nOpenJDK Runtime Environment (build 17.0.12+7-Ubuntu-1ubuntu222.04)\r\nOpenJDK 64-Bit Server VM (build 17.0.12+7-Ubuntu-1ubuntu222.04, mixed mode, sharing)","author":{"url":"https://github.com/residentsummer","@type":"Person","name":"residentsummer"},"datePublished":"2024-11-10T16:09:30.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":19},"url":"https://github.com/267/libpython-clj/issues/267"}