René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Transfer buildscript tasks to another language","articleBody":"Currently, the Feather buildscript uses Gradle to execute all of its tasks, from running the main feather task, mapping generation, and name propagation, to downloading necessary libraries and publishing the mappings to the Ornithe maven repository. This has *mostly* worked, but Gradle has a tendency to time out or reset its connection randomly for no discernable reason. This was previously a really significant issue when we would publish every version in the same task sequentially, but #134 has significantly improved that situation by running every publish task in parallel, publishing up to 20 versions at a time concurrently. \r\n\r\nThis solved the issue of having to restart the whole publish task for every version if one of the versions failed, but the second time the new publish workflow was run, many versions failed to publish because the publish tasks would try to upload an artifact version that was already on the repository, because the `maven-metadata.xml` file didn't have those versions listed in them. We believe the concurrent nature of the workflow is causing a race condition with reading and writing the `maven-metadata.xml` file, preventing it from getting accurately updated for every version.\r\n\r\nWhen thinking of ways to resolve the first issue, I had the idea of transferring most, if not all, of Gradle's workload to Pure Java Code; I shelved this idea when @Kahzerx refactored the workflow, but this race condition issue is making me consider bringing up this ussie again.\r\n\r\nThere are a couple things to consider:\r\n- What are all the tasks that are part of the current Gradle buildscript that we would need to translate to Java? Are there tasks we can get rid of?\r\n- Is there any new functionality that would be nice to add to a potential Java buildscript that was unfeasible to have when we used Gradle?\r\n   - This might be a way that we could publish all the versions *without* updating the `maven-metadata.xml` file, until the very end.\r\n- Where are we storing the cache for these tasks, like version jars and the like? Should we try to use the same cache folder that our current Gradle buildscript uses? Should we use a different location?\r\n- Related to the previous point, should we use Gradle as a sort of \"bootstrap\" to run the core Java code part of this potential buildscript? Or should we ship with a Pure Java Code bootstrap main class that we compile on the fly that runs the rest of the buildscript?\r\n\r\nI'm getting to the point where I've written so much that I'm losing track of my thoughts, so I'll stop here, I might edit this more later. Please feel free to share your thoughts about this.","author":{"url":"https://github.com/Copetan","@type":"Person","name":"Copetan"},"datePublished":"2023-09-15T23:41:23.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":4},"url":"https://github.com/138/feather/issues/138"}