René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Decide what should we do with doctests in CI","articleBody":"# Bug report\n\nInitial PR and discussion: https://github.com/python/cpython/pull/111682\n\nSo, here's the problem:\n\n```rst\nExample 1::\n\n\u003e\u003e\u003e 1 + 2\n3\n\nExample 2:\n\n.. doctest::\n\n\u003e\u003e\u003e 1 + 2\n3\n```\n\nWe have two tools: `doctest` (builtin module) and `sphinx` (3rd party) that can run these samples as doctests. The problem is that they define \"doctest\" term differently:\n- `doctest` module says that both of these examples are doctests\n- `sphinx` says that only example with `.. doctest::` directive is a doctest\n\nThis is problematic, because not all example are actually executed in CI, right now we have a CI job that uses Sphinx. So, in the result of this we have broken doctest (1st meaning) examples in the docs.\n\nExamples:\n- https://github.com/python/cpython/pull/111677\n- https://github.com/python/cpython/pull/111180\n- https://github.com/python/cpython/pull/111283\n\nWhat can we do?\n\nOptions:\n1. Add `.. doctest::` directives everywhere, but I think it is too much work and people will forget to do this\n2. Try to use `doctest_test_doctest_blocks`, see docs https://www.sphinx-doc.org/en/master/usage/extensions/doctest.html This is what I plan to do locally and I will send a PR with the results. I fear that quite a lot of doctests might be broken\n3. Add doctests to `test_*` files using something like\n\n```python\ndef load_tests(loader, tests, pattern):\n    tests.addTests(doctest.DocFileSuite(\n        os.path.join(REPO_ROOT, 'Doc/library/LIBNAME.rst'),\n        module_relative=False,\n    ))\n    return tests\n```\n\nThe last option is also problematic, because right now we can only run doctests in CI, when PR only touches `Doc/` folder, I am not sure that it will be easy with this setup.\n\nCC @AlexWaygood @AA-Turner \n\n\u003c!-- gh-linked-prs --\u003e\n### Linked PRs\n* gh-111723\n* gh-136501\n\u003c!-- /gh-linked-prs --\u003e\n","author":{"url":"https://github.com/sobolevn","@type":"Person","name":"sobolevn"},"datePublished":"2023-11-03T17:17:24.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":9},"url":"https://github.com/111704/cpython/issues/111704"}