René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"A way to override Python.NET .NET \u003c-\u003e Python wrapping for custom classes","articleBody":"### Environment\r\n\r\n-   Pythonnet version: 2.3.0\r\n-   Python version: 3.6\r\n-   Operating System: Windows 10 x64\r\n\r\n### Details\r\n\r\nI am trying to generate statically-typed wrappers for Python libraries, particularly TensorFlow (see [Gradient](https://github.com/losttech/Gradient)).\r\n\r\nSince I am automatically generating .NET-side types for wrappers, I can make their methods to use my custom converter written in C# to convert function arguments to `PyObject` instances. And then for any result values, I can convert `PyObject` instances back into my wrapper types.\r\n\r\nProblem I am facing is when a user of my wrapper wants to inherit from one of the Python-derived classes, and override and/or extend its behavior.\r\n\r\nAn artificial sample:\r\n```csharp\r\n// this part of my library is auto-generated\r\n// represents tensorflow module\r\nclass tf {\r\n  static dynamic tf = Py.Import(\"tensorflow\");\r\n  // set_default_session - made up method\r\n  static void SetDefaultSession(Session session) =\u003e tf.set_default_session(session.underlyingSession);\r\n\r\n  // made up method\r\n  static void ComputeUsingDefaultSession() =\u003e tf.compute_using_default_session();\r\n}\r\n\r\n// represents tf.Tensor\r\nclass Tensor { PyObject underlyingTensor; }\r\n// represents tf.Session\r\nclass Session {\r\n  PyObject underlyingSession;\r\n  virtual object Run(Tensor tensor) {\r\n    var pyTensor = ConvertToPyObject(tensor); // simply does tensor.underlyingTensor here\r\n    dynamic pySession = this.underlyingSession;\r\n    var result = pySession.Run(pyTensor);\r\n    return ConvertFromPyObject(result); // wraps any returned object into one of generated classes\r\n  }\r\n}\r\n\r\n// this code is what user of my library wants to do:\r\nclass MyBetterSession : Session {\r\n  override object Run(Tensor tensor) {\r\n    ... here he writes custom code to run a Tensor ...\r\n  }\r\n}\r\n\r\n// an attempts to use the above (functions made up):\r\nvar simpleSession = new Session();\r\ntf.SetDefaultSession(simpleSession); // OK\r\ntf.ComputeUsingDefaultSession(); // SUCCEEDS\r\n\r\nvar betterSession = new MyBetterSession();\r\ntf.SetDefaultSession(betterSession); // OK\r\ntf.ComputeUsingDefaultSession(); // FAILS\r\n```\r\n\r\nThe last line will fail, because Python will attempt to call `MyBetterSession.Run` with Python's class `tf.Tensor`, but the method actually expects wrapped class `Tensor`. I need to somehow tell Python or rather Python.NET to invoke my `ConvertFromPyObject` on the argument, before trying to find a matching overload.\r\n\r\nI looked into Python.NET source, and the corresponding objects are `MethodBinding` and `MethodObject`, however, neither seem to provide any extensibility.\r\n\r\nNow I am considering several approaches to the problem, and I just wanted to discuss them with Python.NET team, as some of them involve modification of Python.NET.\r\n\r\n1. Do not change anything in Python.NET, and for every user class like `MyBetterSession` generate a pythonic wrapper with the same set of methods, but replacing all parameter and return types with `PyObject`. It would require one of the following:\r\n- `System.Reflection.Emit`, which is a very heavy dependency, and also not very pleasant to work with\r\n- Roslyn, which is much heavier, but moderately OK to work with\r\n\r\n2. I noticed, that while Python.NET supports representing any Python object as `dynamic`, it does not actually support passing `dynamic` objects (e.g. `IDynamicMetaObjectProvider` instances) back to Python.  I mean, it would pass them, but if Python would try to access a dynamic attribute, it would not attempt to call `TryGetMember`. I could potentially implement something similar to `ClassBase` specifically for wrapping `IDynamicMetaObjectProvider` instances. Then it is much easier to implement its members, that would simply wrap PyObject arguments before forwarding them to an instance of `MyBetterSession`.\r\n\r\n3. Have Python.NET directly expose some low-level interface, that would enable hooking into Python.NET's marshaling and, possibly, also method binding processes.","author":{"url":"https://github.com/lostmsu","@type":"Person","name":"lostmsu"},"datePublished":"2019-03-05T02:17:32.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":3},"url":"https://github.com/823/pythonnet/issues/823"}