René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"ClassManager Cache Performance Impact","articleBody":"### Environment\r\n\r\n-   Pythonnet version: Latest Dev 3.0.0\r\n-   Python version: 3.6.8\r\n-   Operating System: Windows 10\r\n\r\n### Details\r\nI was doing some performance testing on the latest PythonNet and noticed a significant slow down in the conversion of CLR objects to Python. After some digging I believe I found where the slow down is coming from. \r\n\r\nIn #1287 , the addition of the `MaybeType` changed the `ClassManager` dictionary cache, which ended up storing `ClassBases` with `MaybeType` as the key. But because the `ClassManager` 's `GetClass` function attempts to fetch a value out of the cache using a `Type`, the object is implicitly converted to a MaybeType first. This conversion and then comparison (finding obj in dictionary) leads to a major difference in performance when converting large quantities of an object to Python since we have to initialize a new object every time the cache is used. \r\n\r\n\r\n\r\n-   What commands did you run to trigger this issue? If you can provide a\r\n    [Minimal, Complete, and Verifiable example](http://stackoverflow.com/help/mcve)\r\n    this will help us understand the issue.\r\n\r\nHere is a unit test without any pass/fail that I was using to debug this issue. I paired it with dotTrace to see the difference in performance.\r\n```\r\n        [Test]\r\n        public void Test()\r\n        {\r\n            for (int i = 0; i \u003c 1000000; i++)\r\n            {\r\n                var testSlice = new Slice();\r\n                testSlice.ToPython();\r\n            }\r\n        }\r\n\r\n        public class Slice\r\n        {\r\n            public Dictionary\u003cchar, int\u003e intDict = new Dictionary\u003cchar, int\u003e();\r\n            public List\u003cint\u003e intList = new List\u003cint\u003e();\r\n            public string test = \"pepe\";\r\n            public decimal testDecimal = 5.5m;\r\n\r\n            public Slice()\r\n            {\r\n                for (int i = 0; i \u003c 20; i++)\r\n                {\r\n                    var key = 'A' + i;\r\n                    intDict.Add((char)key, i);\r\n                    intList.Add(i);\r\n                }\r\n\r\n            }\r\n        }\r\n```\r\n\r\n#### Current Branch Performance on 1 million calls of same object:\r\n![image](https://user-images.githubusercontent.com/41277998/109896485-d4d13480-7c45-11eb-8031-065b0db03997.png)\r\n\r\n#### Without MaybeType in ClassManager (Cache dict is \u003cType, ClassBase\u003e) performance:\r\n![image](https://user-images.githubusercontent.com/41277998/109898073-7eb1c080-7c48-11eb-88a1-5613ef566e2c.png)\r\n\r\n\r\n\r\n\r\n### Attempt at a fix while maintaining `MaybeType`\r\n\r\n\r\n\r\nI did attempt to improve the performance by overriding the `Equals()` and `GetHashCode()` functions for `MaybeType` which did show a little improvement and a bit more visibility into the slowdown. But unfortunately the object is already implicit converted before `Equals()` because the ClassManager cache dictionary is \u003cMaybeType, ClassBase\u003e\r\n\r\n`public static implicit operator MaybeType (Type ob) =\u003e new MaybeType(ob);`\r\n\r\nHere was my attempt.\r\n```\r\n        /// \u003csummary\u003e\r\n        /// Determines whether the specified \u003csee cref=\"T:System.Object\"/\u003e is equal to the current \u003csee cref=\"T:System.Object\"/\u003e.\r\n        /// \u003c/summary\u003e\r\n        /// \u003creturns\u003e\r\n        /// true if the specified object  is equal to the current object; otherwise, false.\r\n        /// \u003c/returns\u003e\r\n        /// \u003cparam name=\"obj\"\u003eThe object to compare with the current object. \u003c/param\u003e\u003cfilterpriority\u003e2\u003c/filterpriority\u003e\r\n        public override bool Equals(object obj)\r\n        {\r\n            if (ReferenceEquals(null, obj)) return false;\r\n\r\n            // Compare as a Type\r\n            if (obj is Type typeObj)\r\n            {\r\n                return this.Value == typeObj;\r\n            }\r\n\r\n            // If MaybeType just compare Types\r\n            if (obj is MaybeType maybeObj)\r\n            {\r\n                return Value == maybeObj.Value;\r\n            }\r\n\r\n            // It must be false\r\n            return false;\r\n        }\r\n\r\n        /// \u003csummary\u003e\r\n        /// Serves as a hash function for a particular type.\r\n        /// \u003c/summary\u003e\r\n        /// \u003creturns\u003e\r\n        /// A hash code for the current \u003csee cref=\"T:System.Object\"/\u003e.\r\n        /// \u003c/returns\u003e\r\n        /// \u003cfilterpriority\u003e2\u003c/filterpriority\u003e\r\n        public override int GetHashCode()\r\n        {\r\n            // Use Type HashCode\r\n            unchecked { return Value.GetHashCode(); }\r\n        }\r\n```\r\n\r\nAnd results:\r\n![image](https://user-images.githubusercontent.com/41277998/109897607-d4399d80-7c47-11eb-8539-bcd577843212.png)\r\n\r\n\r\nIt appears that the slowest part is actually the fetching of the fully qualified name, Which could be something that is changed to be lazily fetched when requested instead of during construction.\r\n\r\n\r\n","author":{"url":"https://github.com/C-SELLERS","@type":"Person","name":"C-SELLERS"},"datePublished":"2021-03-04T01:52:11.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":2},"url":"https://github.com/1407/pythonnet/issues/1407"}