René's URL Explorer Experiment

Title: When IterableWrapper iterates up to the middle of list, System.AccessViolationException occurs. by JakeJP · Pull Request #2280 · pythonnet/pythonnet · GitHub

Open Graph Title: When IterableWrapper iterates up to the middle of list, System.AccessViolationException occurs. by JakeJP · Pull Request #2280 · pythonnet/pythonnet

X Title: When IterableWrapper iterates up to the middle of list, System.AccessViolationException occurs. by JakeJP · Pull Request #2280 · pythonnet/pythonnet

Description: System.AccessViolationException occurs when iteration quits in the middle of list before reaching the end. Environment Pythonnet version: 3.0.3 Python version: 3.12.0 Operating System: Windows 11 .NET Runtime: .NET Framework 4.8 Reproduce the problem I'm trying to pass a list from python to .NET method via List Codec. mylist = ["a","b","c"] dotnet.method( mylist ) Conversion(decode) is successful and mylist is wrapped by ListWrapper. .NET code receives the list like this. void method( IList mylist ) { mylist.First( s => s == "b" ); // <- here the crash with System.AccessViolationException } Using Enumerator of mylist, if enumerated to the end of the list, nothing bad happens. However if iteration didn't go to the end of the list, AccessViolationException is thrown. This case happens in the following code: mylist.First( s => s == "b" ); // iteration stops at the 2nd item similarly foreach( var item in mylist ) { if( item == "b" ) break; // iteration stops at the 2nd item } mylist.Contains("b"); // iteration stops at the 2nd item mylist.Any( s => s == "b"); // iteration stops at the 2nd item The following code doesn't throw exception becuase it always iterates to the end of the list. mylist.ToList(); mylist.ToArray(); Exception occurs because PyIter(ator) object is Dispose(d) outside of GIL protection only when iteration ends before reaching the end ( before MoveNext() returns null ). In such a case, code process just slips out of GIL enclosure as the nature of C# scope and yield return and break. solution for now original code is like this: public IEnumerator GetEnumerator() { PyIter iterObject; using (Py.GIL()) { iterObject = PyIter.GetIter(pyObject); } using var _ = iterObject; while (true) { using var GIL = Py.GIL(); if (!iterObject.MoveNext()) { iterObject.Dispose(); break; } yield return iterObject.Current.As()!; } } using var _ = iterObject; above the while loop is the problematic line. Before iteration reaches the end, the executing process is somewhere out of the using scope. When c# caller process stops iteration(Enumeration) for some reason, c# suddely calls using var _ = iterObject; 's closing process which is iterObject.Dispose(). But here iterObject is not inside of GIL() protection. This is why exception occurs. Solution is to make sure iterObject.Dispose() is always inside of GIL protection and always executed whatever the caller process abandons the enumeration in the middle. My suggestion is to use try catch pattern. Without catching any exception it may look odd. But this is one solution to make sure the execution of Dispose and GIL scope surrounding always. public IEnumerator GetEnumerator() { PyIter iterObject; using (Py.GIL()) { iterObject = PyIter.GetIter(pyObject); } try { while (true) { using var _ = Py.GIL(); if (!iterObject.MoveNext()) { break; } yield return iterObject.Current.As()!; } } finally { using var _ = Py.GIL(); iterObject.Dispose(); } }

Open Graph Description: System.AccessViolationException occurs when iteration quits in the middle of list before reaching the end. Environment Pythonnet version: 3.0.3 Python version: 3.12.0 Operating System: Windows 11 ...

X Description: System.AccessViolationException occurs when iteration quits in the middle of list before reaching the end. Environment Pythonnet version: 3.0.3 Python version: 3.12.0 Operating System: Windows 11 ...

Opengraph URL: https://github.com/pythonnet/pythonnet/pull/2280

X: @github

direct link

Domain: github.com

Links:

Viewport: width=device-width