René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Change in OSX Catalina makes matplotlib + multiprocessing crash","articleBody":"I'm getting a crash on macOS Catalina when running some multiprocessing code after doing some plotting with matplotlib.\r\n\r\nLooks like this:\r\n```\r\nlibc++abi.dylib: terminating with uncaught exception of type std::runtime_error: Couldn't close file\r\nFatal Python error: Aborted\r\n```\r\nand using `print(psutil.Process().open_files())` I figured out that likely the issue is that this MPL TTF font file is opened multiple times and (I guess) for some reason closing the file occurs only later, from my multiprocessing code, and that fails on macOS Catalina.\r\n```\r\nopen files: [popenfile(path='/Users/deil/anaconda3/envs/gammapy-dev/lib/python3.7/site-packages/matplotlib/mpl-data/fonts/ttf/DejaVuSans.ttf', fd=5), popenfile(path='/Users/deil/anaconda3/envs/gammapy-dev/lib/python3.7/site-packages/matplotlib/mpl-data/fonts/ttf/DejaVuSans.ttf', fd=6), popenfile(path='/Users/deil/anaconda3/envs/gammapy-dev/lib/python3.7/site-packages/matplotlib/mpl-data/fonts/ttf/DejaVuSansDisplay.ttf', fd=7), popenfile(path='/Users/deil/anaconda3/envs/gammapy-dev/lib/python3.7/site-packages/matplotlib/mpl-data/fonts/ttf/DejaVuSansDisplay.ttf', fd=8)]\r\n```\r\n\r\nI tried to extract a minimal test case that doesn't involve our software, but couldn't so far, sorry.\r\nBut you should be able to reproduce the crash by copy \u0026 pasting these commands:\r\n```\r\ngit clone https://github.com/gammapy/gammapy.git\r\ncd gammapy\r\nconda env create -f environment-dev.yml\r\npip install -e .\r\npytest -s -v gammapy/cube\r\n```\r\nLog: https://gist.github.com/cdeil/a75211856a3bcab751ead707df9708c9#file-gistfile1-txt-L88\r\n\r\nI know MPL isn't thread-safe. But it should be possible to do some plotting, and then later after plotting is done to run some multiprocessing code, no?\r\n\r\nIs it normal that the file handle for `DejaVuSansDisplay.ttf` remains open?\r\nShouldn't MPL load the file content and close the file handle directly?\r\nMaybe that is a bug?\r\n\r\nI don't think it matters for this issue, but in case it does -- we are calling `plt.close()` after all plotting from our tests, because previously we had a problem with too many open figures (see [here](\r\nhttps://github.com/gammapy/gammapy/blob/d4db11d559b210c0e0d03bba75ddcff9c03e0511/gammapy/utils/testing.py#L209-L230\r\n\r\n```\r\n(gammapy-dev) hfm-1804a:gammapy deil$ python --version\r\nPython 3.7.0\r\n(gammapy-dev) hfm-1804a:gammapy deil$ python -c 'import matplotlib; print(matplotlib.__version__)'\r\n3.1.1\r\n(gammapy-dev) hfm-1804a:gammapy deil$ python -c 'import matplotlib; print(matplotlib.get_backend())'\r\nMacOSX\r\n```","author":{"url":"https://github.com/cdeil","@type":"Person","name":"cdeil"},"datePublished":"2019-10-12T09:49:41.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":38},"url":"https://github.com/15410/matplotlib/issues/15410"}