René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Line normalization before diff","articleBody":"**Description**\n\nThe `DiffRowGenerator` class offers the `lineNormalizer` property. By default, it is used to replace `\u003c` and `\u003e` by their escaped versions `\u0026lt;` and `\u0026gt;`.\n\nThe `lineNormalizer` is applied to the input texts before the diff is calculated. While I see this is as a useful feature, in case of the default settings it might be surprising that the resulting text might not have HTML escaping anymore:\n\n```java\nfinal var generator = DiffRowGenerator.create() //\n        .mergeOriginalRevised(true) //\n        .showInlineDiffs(true) //\n        .inlineDiffByWord(true) //\n        .build();\n\nfinal var rows = generator.generateDiffRows(List.of(\"hello \u003cworld\u003e\"), List.of(\"bye \u003eworld\u003c\"));\n\nfinal var resultingText = rows.stream() //\n        .map(DiffRow::getOldLine) //\n        .collect(Collectors.joining(StringUtils.LF));\n``` \n\nThe resulting text is\n```\n\u003cspan class=\"editOldInline\"\u003ehello\u003c/span\u003e\u003cspan class=\"editNewInline\"\u003ebye\u003c/span\u003e \u0026\u003cspan class=\"editOldInline\"\u003elt\u003c/span\u003e\u003cspan class=\"editNewInline\"\u003egt\u003c/span\u003e;world\u0026\u003cspan class=\"editOldInline\"\u003egt\u003c/span\u003e\u003cspan class=\"editNewInline\"\u003elt\u003c/span\u003e;\n``` \n\nNote the part ` \u0026` is considered as an equal text part because both replacements `\u0026lt;` and `\u0026gt;` start with an ampersand. This resulting text is therefore no valid HTML anymore.\n\nIn order for this behaviour to be a problem, the following conditions must all be true:\n\n1. The `inlineDiffByWord` must be used\n2. The default `lineNormalizer` must be used\n3. The two provided texts must differ at a position which starts with a character that is replaced by the `lineNormalizer`\n4. A release \u003e= 4.15 must be used.\n\n**Workaround**\nOverride the `lineNormalizer`. E.g., by using the `SPLIT_BY_WORD_PATTERN` of release 4.12, in which [the ampersand was not considered a character that splits words](https://github.com/java-diff-utils/java-diff-utils/blob/0fd3bd8e061eed09dbb937c8ab9ba0969ba12264/java-diff-utils/src/main/java/com/github/difflib/text/DiffRowGenerator.java#L70).\n\n**Solution approaches**\nIMHO, the `SPLIT_BY_WORD_PATTERN` of release 4.15+ is fine and I do not consider it to be the problem.\n\nThe library could offer one of the following features:\n1. a parameter which defines when the 'lineNormalizer' should be applied (before diff-ing or after)\n2. a second type of line-normalizer that is applied after diff-ing\n3. an option to have the library apply the [`processDiffs` function](https://github.com/java-diff-utils/java-diff-utils/blob/637cb7b6a309d66ff5e0cec2b3ffea52f867edc7/java-diff-utils/src/main/java/com/github/difflib/text/DiffRowGenerator.java#L190) to non-diffs as well\n","author":{"url":"https://github.com/epictecch","@type":"Person","name":"epictecch"},"datePublished":"2025-09-22T14:09:33.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":3},"url":"https://github.com/219/java-diff-utils/issues/219"}