René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Automated RSS/Atom Feed Validation Workflow","articleBody":"Hi @hugovk! 👋\n\n## Summary\n\nI've been working on improving the issue templates and adding automated RSS/Atom feed validation for Planet Python feed requests. This addresses #579 by implementing automated feed validation in CI.\n\n## What I've Implemented\n\nI've created a complete GitHub Actions workflow in my fork (https://github.com/matrixise/planet) that includes:\n\n### 1. **Modernized Issue Templates**\n- Migrated from old markdown format to GitHub's YAML issue forms\n- Two templates: \"Add or Edit RSS Feed\" and \"Bug Report\"\n- Structured form fields with validation requirements\n\n### 2. **Automated Feed Validation Workflow**\nThe workflow automatically validates feeds when issues are submitted, addressing the concerns raised in #579:\n\n- ✅ Validates URL format\n- ✅ Checks feed accessibility (HTTP 200)\n- ✅ Validates RSS/Atom structure using feedparser\n- ✅ Detects duplicate feeds in config.ini\n- ✅ Analyzes Python content (keyword detection)\n- 📝 Posts detailed validation results as comments\n- 🏷️ Adds labels based on validation status\n\nThis provides **immediate CI validation** for new feed submissions, catching issues before manual review.\n\n### 3. **Validation Results**\nThe workflow posts a comprehensive comment showing:\n- All validation checks with emoji indicators (✅/⚠️/❌)\n- Python content analysis with keyword detection score\n- Duplicate detection against existing feeds\n- Sample article titles from the feed\n- Clear next steps for contributors and maintainers\n- Link to W3C Feed Validator for detailed validation\n\n### Example Output\n\nSee my test issue for a live example: https://github.com/matrixise/planet/issues/2\n\n## Benefits\n\n1. **Addresses #579**: Automatic feed validation in CI for new submissions\n2. **Reduced maintainer workload**: Automatic validation catches common issues\n3. **Faster feedback**: Contributors get immediate validation results\n4. **Better quality**: Ensures feeds are accessible, valid, and Python-relevant before review\n5. **Duplicate prevention**: Automatically detects if a feed already exists\n6. **Transparency**: Clear, detailed feedback for all submissions\n\n## Implementation Details\n\n**New Files:**\n- `.github/workflows/validate-feed-request.yml` - Main validation workflow\n- `.github/scripts/validate_feed.py` - Feed validation logic (~450 lines)\n- `.github/scripts/format_comment.py` - Comment formatting (~250 lines)\n- `.github/scripts/get_labels.py` - Label extraction helper\n\n**Dependencies:**\n- `feedparser` - RSS/Atom parsing\n- `requests` - HTTP accessibility checks\n\n**Labels Used:**\n- `feed-request` - Triggers the workflow\n- `validation-passed` - All checks passed\n- `validation-warning` - Passed with warnings\n- `validation-failed` - Critical failure\n- `duplicate-feed` - Feed already exists\n\n## How This Addresses #579\n\nWhile #579 requested periodic validation of existing feeds (cron job), this implementation provides:\n1. **Immediate validation** for new feed submissions via issues\n2. **CI-based validation** that runs automatically on GitHub Actions\n3. **Foundation for future work**: The validation scripts can easily be extended to run as a periodic cron job to check all existing feeds\n\nThe current implementation focuses on the **submission workflow** (validating new feeds), which is the most critical use case. Adding periodic validation of all existing feeds would be a natural next step.\n\n## Current Status\n\n⚠️ **Note**: This is still in early testing phase. I haven't completed all test scenarios yet, but the initial results look very promising! The workflow successfully validates feeds and provides helpful feedback. I'm opening this issue to get early feedback from maintainers before investing more time in comprehensive testing and refinement.\n\n## Initial Testing\n\nThe workflow has been tested on my fork with the following scenario:\n- ✅ Valid feed with good Python content (Real Python)\n\nThe code **includes logic** to handle:\n- HTTP errors (404, timeouts, connection errors)\n- Malformed XML/RSS feeds (via feedparser's bozo detection)  \n- Duplicate detection in config.ini\n- Python content analysis (keyword detection in titles and summaries)\n\nHowever, I haven't systematically tested all error scenarios yet. The implementation looks solid, but comprehensive testing across different feed types and failure modes is still needed.\n\n### Python Content Detection Details\n\nThe workflow analyzes up to 10 recent articles and searches for Python-related keywords in titles and article summaries:\n- Keywords: `python`, `django`, `flask`, `fastapi`, `pytest`, `pip`, `pandas`, `numpy`, `asyncio`, `pypi`, `virtualenv`, `conda`, `jupyter`, `matplotlib`, `scikit`, `tensorflow`, `pytorch`\n- Score = percentage of articles containing at least one keyword\n- Thresholds: \u003c30% = warning, 30-60% = suggestion to filter, \u003e60% = good\n\n## Next Steps\n\nI'd like to contribute this to the main python/planet repository. The workflow is:\n- Non-blocking (informational, doesn't prevent issue creation)\n- Complements manual review (doesn't replace maintainer judgment)\n- Fully automated (no maintenance required once set up)\n\nWould you be interested in this addition? I'm happy to:\n1. Complete more comprehensive testing\n2. Open a PR with the implementation\n3. Make any adjustments based on feedback\n4. Help with documentation\n5. Extend it to add periodic validation of existing feeds (to fully address #579)\n\nLet me know if you'd like me to proceed with a PR or if you'd like to see more testing first!","author":{"url":"https://github.com/matrixise","@type":"Person","name":"matrixise"},"datePublished":"2026-01-07T06:47:09.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":2},"url":"https://github.com/614/planet/issues/614"}