René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Improve our approach to javadoc linking","articleBody":"The [pom-scijava-base](https://github.com/scijava/pom-scijava-base) ancestor POM includes [configuration hardcoding several links](https://github.com/scijava/pom-scijava-base/blob/master/pom.xml#L412-L455) for the javadoc tool. This configuration makes classes for SciJava-based projects clickable, pointing uniformly to [javadoc.scijava.org](https://javadoc.scijava.org/) endpoints aggregating multiple components. For example, the URL https://javadoc.scijava.org/SciJava/ includes javadoc for many components in the [scijava GitHub org](https://github.com/scijava) with groupId `org.scijava`. The general pattern is \"Maven groupId = GitHub org = javadoc.scijava.org endpoint\".\r\n\r\n## Problems\r\n\r\n1. __Reproducibility.__ The javadoc.scijava.org endpoints are not reproducible. They are like SNAPSHOTs, always serving the latest javadoc. Properly, we should be linking to stable API javadoc corresponding to the versions depended upon by each built project. Otherwise, rebuilding the javadoc later will produce a different result, and potentially javadoc build errors if e.g. a class was later removed.\r\n\r\n2. __Performance.__ The more `\u003clink\u003e` entries hardcoded by pom-scijava-base, the longer everyone's javadoc builds take, because the javadoc tool scrapes the index from each linked URL.\r\n\r\n3. __Extensibility.__ Projects wanting to extend their javadoc with links other than what pom-scijava-base hardcodes must add those links manually to their project POMs.\r\n\r\n4. __Separation of concerns.__ The pom-scijava-base should not be hardcoding links related to components from the pom-scijava BOM. Or to put another way: the list of javadoc links should be defined here in pom-scijava, and fully correspond to all the components managed here.\r\n\r\n## Solutions\r\n\r\n### How to extend a project POM with more links\r\n\r\nThe following block adds more javadoc URL links:\r\n\r\n```xml\r\n\u003cbuild\u003e\r\n  \u003cpluginManagement\u003e\r\n    \u003cplugins\u003e\r\n      \u003cplugin\u003e\r\n        \u003cartifactId\u003emaven-javadoc-plugin\u003c/artifactId\u003e\r\n        \u003cconfiguration\u003e\r\n          \u003clinks combine.children=\"append\"\u003e\r\n            \u003clink\u003ehttps://javadoc.io/static/commons-io/commons-io/${commons-io.version}\u003c/link\u003e\r\n            \u003clink\u003ehttps://javadoc.io/static/org.ojalgo/ojalgo/${ojalgo.version}\u003c/link\u003e\r\n          \u003c/links\u003e\r\n        \u003c/configuration\u003e\r\n      \u003c/plugin\u003e\r\n    \u003c/plugins\u003e\r\n  \u003c/pluginManagement\u003e\r\n\u003c/build\u003e\r\n```\r\n\r\n### How to achieve link reproducibility\r\n\r\nNote the use of the wonderful `javadoc.io` service to link _reproducibly_ to individual component javadoc available from Maven Central! I think `javadoc.io` is the nicest way forward for reproducible javadoc permalinks. We don't have to deploy our own javadoc infrastructure—just continue working toward publishing as many of our artifacts as possible to Maven Central as they mature.\r\n\r\n### How to balance javadoc build times with correct linking\r\n\r\nFirstly: __we shouldn't unilaterally add javadoc links for the entire component collection.__ I tried updating pom-scijava to use granular javadoc.io links instead of our fuzzy aggregating javadoc.scijava.org links. This changed the configuration from 37 unstable (javadoc.scijava.org) links to 231 stable ones (javadoc.io). Unfortunately, javadoc build time appears to scale pretty much linearly with this number: 0 links -\u003e 11s; 37 links -\u003e 48s; 231 links -\u003e 215s.\r\n\r\nTherefore, every project should include links in its javadoc configuration matching its direct dependencies. (Assuming a project's dependencies are structured correctly, with `mvn dependency:analyze` not reporting problems: transitive dependencies are not part of the project's public API, and thus no such links will be needed for that project's javadoc.)\r\n\r\nIt would be nice is the maven-javadoc-plugin had a feature that did this automatically. The [`\u003cdetectLinks\u003e` option](https://maven.apache.org/plugins/maven-javadoc-plugin/aggregate-jar-mojo.html#detectLinks) is _almost_ what we need—but because there is no way to know what remote javadoc URL you want for each given `groupId:artifactId` dependency, it uses a simple heuristic, `${project.url}/apidocs`, which will fail for the vast majority of components.\r\n\r\n## Conclusion\r\n\r\nWe could enhance the maven-javadoc-plugin to support configuration declaring a mapping from groupId:artifactId to javadoc link URL, with the `\u003cdetectLinks\u003e` feature using this mapping preferentially when discerning the links. For performance, we would need to double check that `\u003cdetectLinks\u003e` only includes links for _direct_ dependencies, not transitive ones—or else add a flag controlling this behavior.","author":{"url":"https://github.com/ctrueden","@type":"Person","name":"ctrueden"},"datePublished":"2020-06-15T15:01:52.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":3},"url":"https://github.com/130/pom-scijava/issues/130"}