René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Metabug: Improving C-level coverage","articleBody":"*[edit by @encukou, May 2024] The coverage report and checklist are outdated. [Run coverage locally](https://devguide.python.org/testing/coverage/#measuring-coverage-of-c-code-with-gcov-and-lcov) before contributing.*\r\n\r\n---\r\n\r\nThis bug is going to be used to track work in a other bugs to improve the C-level coverage of the CPython test suite.\r\n\r\nThere is a set of ~~[baseline coverage results on main](http://droettboom.com/cpython-coverage/llvm-coverage/)~~ [edit: outdated, see above] that can be used to find coverage gaps.\r\n\r\nThe plan, [discussed on discuss.python.org](https://discuss.python.org/t/filing-c-level-test-coverage-bugs/17309) is as follows:\r\n\r\n- Read through the coverage report and record any notable gaps in the checklist below. The goal is not 100% coverage, and each area of improvement will probably require some judgement calls. For example, covering all cases where memory exhaustion can occur is probably not worth the effort. On the other hand, detailed coverage in the eval loop may be worth the effort.\r\n- When someone has \"read through\" a particular source file and added created subitems for any interesting gaps, they should check it off on the list below and add links to any issues created.\r\n\r\n**Related work:**\r\n\r\nThere is [related work to publish coverage results from CPython on a regular basis](https://github.com/python/cpython/issues/94759), but this issue is concerned with using those results to actually reduce our gaps in coverage.\r\n\r\n**List of source files:**\r\n\r\n- [x] `Include/internal/pycore_asdl.h`\r\n- [ ] `Include/internal/pycore_bitutils.h`\r\n- [ ] `Include/internal/pycore_call.h`\r\n- [ ] `Include/internal/pycore_code.h`\r\n- [ ] `Include/internal/pycore_frame.h`\r\n- [ ] `Include/internal/pycore_moduleobject.h`\r\n- [ ] `Include/internal/pycore_object.h`\r\n- [ ] `Include/internal/pycore_pymath.h`\r\n- [ ] `Include/internal/pycore_pymem.h`\r\n- [ ] `Include/internal/pycore_pystate.h`\r\n- [ ] `Include/object.h`\r\n- [ ] `Include/pydtrace.h`\r\n- [x] `Objects/abstract.c`\r\n  - [ ] Buffer related functions: `PyBuffer_FromContiguous`, `PyObject_CopyData`, `PyBuffer_FillContiguousStrides`\r\n  - [ ] `PyNumber_Check` doesn't test `complex`\r\n  - [ ] `PySequence_Repeat` and `PySequence_InPlaceRepeat` have no coverage\r\n  - [x] `PySequence_SetItem` with a negative index is untested\r\n  - [ ] `PySequence_SetSlice` and `PySequence_DelSlice` are untested\r\n  - [ ] `PyMapping_HasKey` and `PyMapping_HasKeyString` are untested\r\n- [x] `Objects/accu.c`\r\n- [x] `Objects/boolobject.c` \r\n  - [x] #94859\r\n- [x] `Objects/bytearrayobject.c`\r\n  - [x] #95802\r\n- [x] `Objects/bytes_methods.c`\r\n- [x] `Objects/bytesobject.c`\r\n  - [x] ~In `PyBytes_FromFormatV`, the special handling of `%p` isn't tested.~  It's only the case where the underlying libc is broken that isn't tested.\r\n  - [x] #95895\r\n- [ ] `Objects/call.c`\r\n  - [ ] `PyEval_CallObjectWithKeywords` has no coverage\r\n  - [ ] `_PyObject_CallMethodId_SizeT` has no coverage\r\n- [x] `Objects/capsule.c`\r\n- [x] `Objects/cellobject.c`\r\n- [x] `Objects/classobject.c`\r\n- [x] `Objects/codeobject.c` \r\n  - [x] #94814 \r\n  - [x] #96609\r\n  - [ ] #94816  \r\n- [ ] `Objects/complexobject.c`\r\n- [x] `Objects/descrobject.c`\r\n- [ ] `Objects/dictobject.c`\r\n  - [ ] In `dictresize` [convert split table into new combined table\"](http://droettboom.com/cpython-coverage/llvm-coverage/coverage/home/mdboom/Work/builds/cpython/Objects/dictobject.c.html#L1448) is uncovered.\r\n  - [ ] [`_PyDict_GetItemHint`](http://droettboom.com/cpython-coverage/llvm-coverage/coverage/home/mdboom/Work/builds/cpython/Objects/dictobject.c.html#L1689) has no coverage\r\n- [x] `Objects/enumobject.c`\r\n- [x] `Objects/exceptions.c`\r\n- [ ] `Objects/fileobject.c`\r\n  - [ ] `PyFile_FromFd` has no coverage\r\n  - [ ] `PyFile_GetLine` over `bytes` input has no coverage \r\n- [x] `Objects/floatobject.c` \r\n  - [x] #94860\r\n- [x] `Objects/frameobject.c` \r\n  - [ ] frame_setlineno has poor coverage in its helper functions [get_arg](http://[droettboom.com/cpython-coverage/llvm-coverage/coverage/home/mdboom/Work/builds/cpython/Objects/frameobject.c.html#L114) and [mark_stacks](http://[droettboom.com/cpython-coverage/llvm-coverage/coverage/home/mdboom/Work/builds/cpython/Objects/frameobject.c.html#L188).\r\n  - [ ] [_PyFrame_GetState](http://droettboom.com/cpython-coverage/llvm-coverage/coverage/home/mdboom/Work/builds/cpython/Objects/frameobject.c.html#L456) has a switch statement where only the default case is covered.\r\n  - [ ] In [_PyFrame_FastToLocalsWithError](http://[droettboom.com/cpython-coverage/llvm-coverage/coverage/home/mdboom/Work/builds/cpython/Objects/frameobject.c.html#L980) there is no test that exercises the COPY_FREE_VARS case.\r\n- [x] `Objects/funcobject.c`\r\n  - [ ] A bunch of API is untested: `PyFunction_GetCode`, `PyFunction_GetGlobals`, `PyFunction_GetModule`, `PyFunction_GetDefaults`, `PyFunction_SetDefaults`, `PyFunction_GetKwDefaults`, `PyFunction_SetKwDefaults`, `PyFunction_GetClosure`, `PyFunction_SetClosure`, `PyFunction_GetAnnotations`, `PyFunction_SetAnnotations` \\\r\n  - [x] #98449\r\n  - [x] #98317 \r\n- [x] `Objects/genericaliasobject.c`\r\n- [x] `Objects/genobject.c`\r\n  - [ ] `gen_new_with_qualname` and API `PyGen_NewWithQualName` and `PyGen_New` have no coverage.\r\n  - [ ] `PyCoro_New` has no coverage\r\n  - [ ] `PyAsyncGen_New` has no coverage\r\n  - [ ] `async_gen_athrow_send` has poor coverage\r\n- [x] `Objects/interpreteridobject.c`\r\n- [x] `Objects/iterobject.c`\r\n  - [x] #95923\r\n- [x] `Objects/listobject.c`\r\n- [x] `Objects/longobject.c`\r\n  - [ ] `_PyLong_Sing_t_Converter` has no coverage\r\n  - [ ] `long_format_binary` doesn't test outputting to UCS2 or UCS4\r\n  - [ ] `int_bit_length_impl` and `int_bit_count_impl` doesn't cover the case where [expression overflows](http://droettboom.com/cpython-coverage/llvm-coverage/coverage/home/mdboom/Work/builds/cpython/Objects/longobject.c.html#L5690)\r\n- [x] `Objects/memoryobject.c`\r\n  - [ ] `init_slice` is [not well-covered](http://droettboom.com/cpython-coverage/llvm-coverage/coverage/home/mdboom/Work/builds/cpython/Objects/memoryobject.c.html#L2408)\r\n- [x] `Objects/methodobject.c`\r\n- [x] `Objects/moduleobject.c`\r\n  - [ ] `PyModule_GetFilename` has no coverage \r\n- [x] `Objects/namespaceobject.c`\r\n- [x] `Objects/object.c`\r\n  - [ ] `PyObject_Print` has no coverage \r\n  - [x] `PyObject_Bytes` does not test the case where there is a `__bytes__`\r\n  - [x] #96627\r\n  - [ ] `PyObject_SetAttrString` doesn't test when object has a `tp_setattr`\r\n  - [ ] `PyObject_GetAttrString` doesn't test when object has a `tp_getattr`\r\n  - [ ] `_PyObject_LookupAttr` doesn't test when object has a `tp_getattr`\r\n- [ ] `Objects/obmalloc.c`\r\n- [x] `Objects/odictobject.c`\r\n- [x] `Objects/picklebufobject.c`\r\n  - [ ] `PyPickleBuffer_FromObject`, `PyPickleBuffer_Release` has no coverage\r\n- [x] `Objects/rangeobject.c`\r\n- [x] `Objects/setobject.c`\r\n- [x] `Objects/sliceobject.c`\r\n  - [ ] `PySlice_GetIndices`/`PySlice_GetIndicesEx` has no coverage\r\n- [ ] `Objects/stringlib/codecs.h`\r\n- [ ] `Objects/stringlib/count.h`\r\n- [ ] `Objects/stringlib/ctype.h`\r\n- [ ] `Objects/stringlib/eq.h`\r\n- [ ] `Objects/stringlib/fastsearch.h`\r\n  - [x] #96760 \r\n- [ ] `Objects/stringlib/find.h`\r\n- [ ] `Objects/stringlib/find_max_char.h`\r\n- [ ] `Objects/stringlib/join.h`\r\n- [ ] `Objects/stringlib/localeutil.h`\r\n- [ ] `Objects/stringlib/partition.h`\r\n- [ ] `Objects/stringlib/replace.h`\r\n- [ ] `Objects/stringlib/split.h`\r\n- [ ] `Objects/stringlib/transmogrify.h`\r\n- [ ] `Objects/stringlib/undef.h`\r\n- [ ] `Objects/stringlib/unicode_format.h`\r\n- [x] `Objects/structseq.c`\r\n- [x] `Objects/tupleobject.c`\r\n- [x] `Objects/typeobject.c`\r\n  - [ ] `wrap_sq_setitem` has no coverage\r\n- [x] `Objects/unicodectype.c`\r\n- [x] `Objects/unicodeobject.c`\r\n  - [ ] `xmlcharrefreplace` doesn't test for codepoints \u003c 100 (This seems almost impossible to occur).\r\n  - [ ] `resize_inplace` has no coverage\r\n  - [ ] `unicode_kind_name` when `!PyUnicode_IS_COMPACT` isn't covered -- *low priority* used by consistency check only\r\n  - [ ] `unicode_write_cstr` doesn't test writing into UCS2 or UCS4\r\n  - [x] #96677\r\n  - [ ] `PyUnicode_AsDecodedObject`, `PyUnicode_AsDecodedUnicode`, `PyUnicode_AsEncodedObject`, `PyUnicode_AsEncodedUnicode` has no coverage\r\n  - [ ] `_Py_DecodeUTF8Ex` and `_Py_EncodeUTF8Ex` has no coverage for `error == surrogateescape`\r\n  - [ ] `PyUnicode_BuildEncodingMap` doesn't handle the `need_dict` case\r\n  - [ ] `ucs1lib_find_slice` and `ucs1lib_rfind_slice` aren't covered.\r\n  - [x] `PyUnicode_Count` has no coverage\r\n  - [x] #98228\r\n  - [x] `PyUnicode_CompareWithASCIIString` has no coverage for comparing with UCS2 or UCS4\r\n  - [ ] `_PyUnicode_EqualToASCIIId` has no coverage\r\n- [x] `Objects/unicodetype_db.h`\r\n- [x] `Objects/unionobject.c`\r\n- [x] `Objects/weakrefobject.c`\r\n- [x] `Parser/action_helpers.c`\r\n  - [ ] `_PyPegen_set_expr_context` doesn't cover \"starred kind\"\r\n  - [ ] `_PyPegen_get_expr_name` switch statement coverage is non-exhaustive\r\n- [x] `Parser/myreadline.c` (N/A Windows-only)\r\n- [x] `Parser/parser.c`\r\n- [x] `Parser/peg_api.c`\r\n- [x] `Parser/pegen.c`\r\n- [x] `Parser/pegen.h`\r\n- [x] `Parser/pegen_errors.c`\r\n  - [x] #94926\r\n- [x] `Parser/string_parser.c`\r\n  - [x] #95925\r\n- [ ] `Parser/tokenizer.c` \r\n  - [x] #94823 \r\n  - [ ] tokenizer.c seems to have no coverage for a few functions related to interactive usage, e.g. [tok_underflow_interactive](http://droettboom.com/cpython-coverage/llvm-coverage/coverage/home/mdboom/Work/builds/cpython/Parser/tokenizer.c.html#L841) and [tok_concatenate_interactive_newline](http://droettboom.com/cpython-coverage/llvm-coverage/coverage/home/mdboom/Work/builds/cpython/Parser/tokenizer.c.html#L301).\r\n- [x] ~`Python/Python-ast.c`~ Generated code\r\n- [x] `Python/Python-tokenize.c`\r\n- [x] `Python/_warnings.c`\r\n  - [ ] `show_warning` doesn't cover the case where there is a `sourceline`.\r\n  - [ ] `PyErr_WarnExplicit` has no coverage\r\n- [ ] `Python/asdl.c`\r\n- [x] `Python/ast.c`\r\n  - [ ] `ensure_literal_*` functions aren't covered\r\n  - [ ] `validate_pattern_match_value` doesn't cover all elements of switch \r\n- [x] `Python/ast_opt.c`\r\n  - [ ] `check_complexity` doesn't cover the `frozenset` case\r\n  - [ ] `ast_foldbody` isn't covered\r\n- [ ] `Python/ast_unparse.c`\r\n- [ ] `Python/bltinmodule.c`\r\n- [ ] `Python/bootstrap_hash.c`\r\n- [x] `Python/ceval.c`\r\n  - [ ] `PyEval_AquireLock` and `PyEval_ReleaseLock` are uncovered\r\n  - [x] #95932\r\n  - [ ] `STORE_ATTR_WITH_HINT` doesn't cover the [case where the dictionary doesn't have Unicode keys](http://droettboom.com/cpython-coverage/llvm-coverage/coverage/home/mdboom/Work/builds/cpython/Python/ceval.c.html#L3797)\r\n  - [x] `CALL_FUNCTION_EX` doesn't cover the case where kwargs is [not an exact dict](http://droettboom.com/cpython-coverage/llvm-coverage/coverage/home/mdboom/Work/builds/cpython/Python/ceval.c.html#L5393)\r\n  - [ ] `PyEval_EvalCodeEx` doesn't cover the case where [kwargs are passed in](http://droettboom.com/cpython-coverage/llvm-coverage/coverage/home/mdboom/Work/builds/cpython/Python/ceval.c.html#L6497)\r\n  - [ ] `PyEval_GetFrame` has no coverage\r\n  - [x] #98300\r\n- [ ] `Python/ceval_gil.h`\r\n- [ ] `Python/codecs.c`\r\n- [x] `Python/compile.c` \r\n  - [ ] [write_instr](http://droettboom.com/cpython-coverage/llvm-coverage/coverage/home/mdboom/Work/builds/cpython/Python/compile.c.html#L201) is not handling the case where ilen \u003e 2. It might be that those are never seen in practice...? If so, feel free to close this bug.\r\n  - [ ] [check_ann_subscr](http://droettboom.com/cpython-coverage/llvm-coverage/coverage/home/mdboom/Work/builds/cpython/Python/compile.c.html#L5989) doesn't have any coverage for slice or tuple kinds. \r\n  - [ ] [optimize_basic_block](http://droettboom.com/cpython-coverage/llvm-coverage/coverage/home/mdboom/Work/builds/cpython/Python/compile.c.html#L8986) has some opcodes that aren't covered in the JUMP_IF_FALSE_OR_POP and the JUMP_IF_TRUE_OR_POP cases.\r\n- [ ] `Python/condvar.h`\r\n- [ ] `Python/context.c`\r\n  - [ ] `PyContext_Copy`, `PyContext_Enter`, `PyContext_Exit` have no coverage\r\n- [ ] `Python/deepfreeze/deepfreeze.c`\r\n- [ ] `Python/dtoa.c`\r\n- [ ] `Python/dup2.c`\r\n- [x] `Python/dynamic_annotations.c`\r\n- [ ] `Python/errors.c`\r\n- [x] `Python/fileutils.c`\r\n  - [ ] `is_valid_wide_char` doesn't test error branches\r\n  - [ ] `encode_ascii`/`decode_ascii` has no coverage (probably very low priority -- comment says only for platforms with a broken mbstowcs (FreeBSD, OpenIndiana)\r\n  - [ ] `_Py_stat` has no coverage\r\n- [ ] `Python/formatter_unicode.c`\r\n- [x] `Python/frame.c`\r\n- [ ] `Python/frozenmain.c`\r\n- [ ] `Python/future.c`\r\n- [ ] `Python/getargs.c`\r\n- [ ] `Python/getopt.c`\r\n- [ ] `Python/hamt.c`\r\n- [ ] `Python/hashtable.c`\r\n- [ ] `Python/import.c`\r\n- [ ] `Python/importdl.c`\r\n- [ ] `Python/initconfig.c`\r\n- [ ] `Python/marshal.c`\r\n- [ ] `Python/modsupport.c`\r\n- [ ] `Python/mysnprintf.c`\r\n- [ ] `Python/mystrtoul.c`\r\n- [ ] `Python/pathconfig.c`\r\n- [ ] `Python/preconfig.c`\r\n- [ ] `Python/pyarena.c`\r\n- [x] `Python/pyfpe.c`\r\n- [ ] `Python/pyhash.c`\r\n- [ ] `Python/pylifecycle.c`\r\n- [ ] `Python/pystate.c`\r\n- [ ] `Python/pystrcmp.c`\r\n- [ ] `Python/pystrhex.c`\r\n- [ ] `Python/pystrtod.c`\r\n- [ ] `Python/pythonrun.c`\r\n- [ ] `Python/pytime.c`\r\n- [x] `Python/specialize.c`\r\n- [ ] `Python/structmember.c`\r\n- [ ] `Python/suggestions.c`\r\n- [x] `Python/symtable.c`\r\n- [ ] `Python/sysmodule.c`\r\n- [ ] `Python/thread.c`\r\n- [x] `Python/traceback.c`\r\n  - [x] tracebacks with angle-bracketed filenames https://github.com/python/cpython/issues/95259 \r\n  - [ ] tb_printinternal with `depth \u003e limit`\r\n  - [ ] `_PyTraceBack_Print_Indented` with overflowing `tracebacklimit`\r\n  - [ ] No coverage for `_Py_DumpDecimal`, `_Py_DumpHexadecimal`, `_Py_DumpASCII`, `dump_frame`, `dump_traceback`, `_Py_DumpTraceback`, `write_thread_id`, `_Py_DumpTracebackThreads` -- possibly they have tests which are disabled under some circumstances. \r\n\r\n\r\n\u003c!-- gh-pr-number: gh-98749 --\u003e\r\n* PR: gh-98749\r\n\u003c!-- /gh-pr-number --\u003e\r\n\r\n\r\n\u003c!-- gh-pr-number: gh-96767 --\u003e\r\n* PR: gh-96767\r\n\u003c!-- /gh-pr-number --\u003e\r\n\r\n\r\n\u003c!-- gh-pr-number: gh-99091 --\u003e\r\n* PR: gh-99091\r\n\u003c!-- /gh-pr-number --\u003e\r\n\r\n\r\n\u003c!-- gh-pr-number: gh-99123 --\u003e\r\n* PR: gh-99123\r\n\u003c!-- /gh-pr-number --\u003e\r\n\r\n\r\n\u003c!-- gh-pr-number: gh-98300 --\u003e\r\n* PR: gh-98300\r\n\u003c!-- /gh-pr-number --\u003e\r\n\r\n\r\n\u003c!-- gh-pr-number: gh-98809 --\u003e\r\n* PR: gh-98809\r\n\u003c!-- /gh-pr-number --\u003e\r\n\r\n\r\n\u003c!-- gh-pr-number: gh-99126 --\u003e\r\n* PR: gh-99126\r\n\u003c!-- /gh-pr-number --\u003e\r\n\r\n\r\n\u003c!-- gh-pr-number: gh-99133 --\u003e\r\n* PR: gh-99133\r\n\u003c!-- /gh-pr-number --\u003e\r\n\r\n\r\n\u003c!-- gh-pr-number: gh-99196 --\u003e\r\n* PR: gh-99196\r\n\u003c!-- /gh-pr-number --\u003e\r\n\r\n\r\n\u003c!-- gh-pr-number: gh-99319 --\u003e\r\n* PR: gh-99319\r\n\u003c!-- /gh-pr-number --\u003e\r\n\r\n\r\n\u003c!-- gh-pr-number: gh-99429 --\u003e\r\n* PR: gh-99429\r\n\u003c!-- /gh-pr-number --\u003e\r\n\r\n\r\n\u003c!-- gh-pr-number: gh-99472 --\u003e\r\n* PR: gh-99472\r\n\u003c!-- /gh-pr-number --\u003e\r\n\r\n\r\n\u003c!-- gh-pr-number: gh-97672 --\u003e\r\n* PR: gh-97672\r\n\u003c!-- /gh-pr-number --\u003e\r\n\r\n\r\n\u003c!-- gh-pr-number: gh-100483 --\u003e\r\n* PR: gh-100483\r\n\u003c!-- /gh-pr-number --\u003e\r\n\r\n\r\n\u003c!-- gh-pr-number: gh-100484 --\u003e\r\n* PR: gh-100484\r\n\u003c!-- /gh-pr-number --\u003e\r\n\r\n\r\n\u003c!-- gh-pr-number: gh-100619 --\u003e\r\n* PR: gh-100619\r\n\u003c!-- /gh-pr-number --\u003e\r\n\r\n\r\n\u003c!-- gh-pr-number: gh-102469 --\u003e\r\n* PR: gh-102469\r\n\u003c!-- /gh-pr-number --\u003e\r\n\r\n\r\n\u003c!-- gh-pr-number: gh-111445 --\u003e\r\n* PR: gh-111445\r\n\u003c!-- /gh-pr-number --\u003e\r\n\r\n\r\n\u003c!-- gh-pr-number: gh-117421 --\u003e\r\n* PR: gh-117421\r\n\u003c!-- /gh-pr-number --\u003e\r\n\r\n\r\n\u003c!-- gh-pr-number: gh-119222 --\u003e\r\n* PR: gh-119222\r\n\u003c!-- /gh-pr-number --\u003e\r\n\r\n\r\n\u003c!-- gh-pr-number: gh-119227 --\u003e\r\n* PR: gh-119227\r\n\u003c!-- /gh-pr-number --\u003e\r\n\n\n\u003c!-- gh-pr-number: gh-119263 --\u003e\n* PR: gh-119263\n\u003c!-- /gh-pr-number --\u003e\n\n\n\u003c!-- gh-pr-number: gh-119264 --\u003e\n* PR: gh-119264\n\u003c!-- /gh-pr-number --\u003e\n\n\n\u003c!-- gh-pr-number: gh-138866 --\u003e\n* PR: gh-138866\n\u003c!-- /gh-pr-number --\u003e\n","author":{"url":"https://github.com/mdboom","@type":"Person","name":"mdboom"},"datePublished":"2022-07-13T19:13:00.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":28},"url":"https://github.com/94808/cpython/issues/94808"}