René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Bad interaction between kerning and non-latin1 characters in pdf output","articleBody":"\u003c!--To help us understand and resolve your issue, please fill out the form to the best of your ability.--\u003e\r\n\u003c!--You can feel free to delete the sections that do not apply.--\u003e\r\n\r\n### Bug report\r\n\r\n**Bug summary**\r\n\r\nNon-latin1 character occuring after strings that involve large kerning corrections are misplaced in the pdf output.\r\n\r\n**Code for reproduction**\r\n\r\n\u003c!--A minimum code snippet required to reproduce the bug.\r\nPlease make sure to minimize the number of dependencies required, and provide\r\nany necessary plotted data.\r\nAvoid using threads, as Matplotlib is (explicitly) not thread-safe.--\u003e\r\n\r\n```python\r\nfrom pylab import *\r\nfigtext(.5, .5, \"AVAVAVAVAVAVAVA€\")\r\nsavefig(\"/tmp/test.pdf\")\r\n```\r\n\r\n**Actual outcome**\r\n\r\n![test](https://user-images.githubusercontent.com/1322974/90509778-2b6b9b00-e15a-11ea-824d-dfa93d6b1de9.png)\r\n(screenshot of the pdf)\r\n\r\n**Expected outcome**\r\n\r\nThe € sign should come at the end.\r\n\r\nThis bisects to #14940, before which kerning was reduced by a factor of 64 and thus the effect would have been still present but unnoticeable except for extreeeemely long strings.\r\n\r\nMy analysis of the bug is as follows:\r\nFirst, note that for Type3 output (the default), non-latin1 characters are written to the pdf file separately from the rest of the string.  I believe(?) that's a fundamental limitation of the pdf spec (see pdf spec. section 9.4.3 Text Showing Operators).  (We could generate a custom encoding to only cover the glyphs we want, and split the font into multiple independent Type3 subsets if the string has more than 255 different glyphs, I guess...)  So we measure the widths of the latin1 substrings (using standard kerning) and position the other glyphs relative to these.\r\nHowever, when subsetting the font as a Type3 font (the default), kerning information is lost.  Therefore, the pdf software (okular/etc.) draws the string \"AVAVAVAVA\" without kerning; in this case it is much wider than expected by Matplotlib and overflows the position where the € sign is drawn.\r\nI believe the solutions are either to\r\n- Add an unkerned mode to `_text_layout.layout` and use that to measure the width of the latin1 substrings in this specific case.  This isn't really worse than pre-#14940, where we effectively *never* kerned.\r\n- Use the showkern (TJ) operator instead of show (Tj) to include the kerning values in the outputted string in the pdf file (we already do this for usetex+pdf, in fact).  I guess this is technically more correct.\r\n\r\n**Matplotlib version**\r\n\u003c!--Please specify your platform and versions of the relevant libraries you are using:--\u003e\r\n  * Operating system: fedora\r\n  * Matplotlib version: master, bisects to #14940\r\n  * Matplotlib backend (`print(matplotlib.get_backend())`): pdf\r\n  * Python version: 38\r\n  * Jupyter version (if applicable):\r\n  * Other libraries: \r\n\r\n\u003c!--Please tell us how you installed matplotlib and python e.g., from source, pip, conda--\u003e\r\n\u003c!--If you installed from conda, please specify which channel you used if not the default--\u003e\r\n\r\n","author":{"url":"https://github.com/anntzer","@type":"Person","name":"anntzer"},"datePublished":"2020-08-18T12:09:26.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":2},"url":"https://github.com/18282/matplotlib/issues/18282"}