René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"agent: always use exact equality when testing console message output","articleBody":"## What happened\n\nWhen adding new code paths that emit console messages, the agent wrote tests using `assert \"...\" in out` (substring checks) instead of exact equality assertions. The reviewer had to request a fix via code review comment.\n\n## Root cause\n\nThe `usethis-python-test-full-coverage` skill was not invoked during test writing. The agent interpreted the AGENTS.md instruction — which says to \"ALWAYS consider the skill to be relevant\" — as permission to evaluate relevance and decide not to use it, rather than as a mandate to always apply it.\n\n## Generalised principle\n\nTwo rules must be observed for all code involving console output:\n\n1. **Always invoke the `usethis-python-test-full-coverage` skill when writing or modifying any code.** The word \"consider\" in the skill instruction means \"treat as always applicable\", not \"decide whether to apply\". There is no scenario where this skill is not relevant when code is being changed.\n2. **Never use substring checks (`assert \"...\" in out`) for console-output tests.** Always assert the exact output string with equality (`assert out == \"...\"`), and set the backend explicitly (e.g. via `usethis_config.set(backend=...)`) so the result is deterministic. Substring checks allow partial coverage to hide missing output and fail to detect unexpected extra messages.\n\n## Resolution\n\nThe `test_no_git_repo` tests were updated to use exact equality assertions with an explicit backend, and the `# noqa: RUF001` suppression was placed on the line containing the ℹ character.\n\n## Follow-up\n\nConsider strengthening the AGENTS.md wording for the `usethis-python-test-full-coverage` skill from \"ALWAYS consider the skill to be relevant\" to something unambiguous such as \"ALWAYS invoke this skill — it is mandatory for any code change, without exception.\"","author":{"url":"https://github.com/nathanjmcdougall","@type":"Person","name":"nathanjmcdougall"},"datePublished":"2026-04-07T18:01:16.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":1},"url":"https://github.com/1866/usethis-python/issues/1866"}