René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"JIT shim build changes in #148872 can cause build steps to be unnecessarily repeated and with possible build breakage","articleBody":"# Bug report\n\n### Bug description:\n\nPR #148872 for gh-126910 inadvertently introduced an unnecessary and potentially error-inducing build rule execution repetition in the main Python Makefile compared with the last alpha release, 3.15.0a8.  The behavior is seen when both `--enable-experimental-jit` and `--enable-optimizations` are specified to `configure`. If the build then includes both a `make` and a `make install` step (a common occurence in build scripts), the second make causes several of the final build targets to be re-executed. In the simplest case, the effect is likely to just be wasted time. But in some cases, it can actually cause incorrect builds and which may not be obvious until later testing.\n\nGiven the simplified example:\n\n```\n./configure --enable-experimental-jit=yes-off --enable-optimizations\nmake\nmake install\n```\n\nprior to https://github.com/python/cpython/commit/9633c5239daae3a180f8ce263ce77e4e522e6aa4, the build output looked something like:\n\n```\n$ make\n[...]\n./python -E ./Tools/build/generate-build-details.py `cat pybuilddir.txt`/build-details.json\nThe necessary bits to build these optional modules were not found:\n_decimal              _zstd\nTo find the necessary bits, look in configure.ac and config.log.\n\nChecked 115 modules (37 built-in, 75 shared, 1 n/a on linux-x86_64, 0 disabled, 2 missing, 0 failed on import)\nmake[1]: Leaving directory '/home/sysadmin/cpython'\n$ make install\nCreating directory /tmp/prefix/bin\nCreating directory /tmp/prefix/lib\nif test \"no-framework\" = \"no-framework\" ; then \\\n\t/usr/bin/install -c python /tmp/prefix/bin/python3.15; \\\nelse \\\n\t/usr/bin/install -c -s Mac/pythonw /tmp/prefix/bin/python3.15; \\\nfi\nif test \"3.15\" != \"3.15\"; then \\\n[...]\n```\n\nBut now, this same sequence results in:\n```\n$ make\n[...]\n./python -E ./Tools/build/generate-build-details.py `cat pybuilddir.txt`/build-details.json\nThe necessary bits to build these optional modules were not found:\n_decimal              _zstd\nTo find the necessary bits, look in configure.ac and config.log.\n\nChecked 115 modules (37 built-in, 75 shared, 1 n/a on linux-x86_64, 0 disabled, 2 missing, 0 failed on import)\nmake[1]: Leaving directory '/home/sysadmin/cpython'\n$ make install\ngcc -c -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer  -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O3 -Wall    -fno-semantic-interposition -std=c11 -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden -D_Py_TIER2=3 -D_Py_JIT  -I./Include/internal -I./Include/internal/mimalloc  -I. -I./Include    -DPy_BUILD_CORE -o Python/jit.o Python/jit.c\ngcc -c -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer  -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O3 -Wall    -fno-semantic-interposition -std=c11 -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden -D_Py_TIER2=3 -D_Py_JIT  -I./Include/internal -I./Include/internal/mimalloc  -I. -I./Include    -DPy_BUILD_CORE \\\n      -DGITVERSION=\"\\\"`LC_ALL=C git --git-dir ./.git rev-parse --short HEAD`\\\"\" \\\n      -DGITTAG=\"\\\"`LC_ALL=C git --git-dir ./.git describe --all --always --dirty`\\\"\" \\\n      -DGITBRANCH=\"\\\"`LC_ALL=C git --git-dir ./.git name-rev --name-only HEAD`\\\"\" \\\n      -o Modules/getbuildinfo.o ./Modules/getbuildinfo.c\ngcc   -fno-semantic-interposition  -o Programs/_freeze_module Programs/_freeze_module.o Modules/getpath_noop.o Modules/getbuildinfo.o Parser/token.o  Parser/pegen.o Parser/pegen_errors.o Parser/action_helpers.o Parser/parser.o Parser/string_parser.o Parser/peg_api.o Parser/lexer/buffer.o Parser/lexer/lexer.o Parser/lexer/state.o Parser/tokenizer/file_tokenizer.o Parser/tokenizer/readline_tokenizer.o Parser/tokenizer/string_tokenizer.o Parser/tokenizer/utf8_tokenizer.o Parser/tokenizer/helpers.o Parser/myreadline.o Objects/abstract.o Objects/boolobject.o Objects/bytes_methods.o Objects/bytearrayobject.o Objects/bytesobject.o Objects/call.o Objects/capsule.o Objects/cellobject.o Objects/classobject.o Objects/codeobject.o Objects/complexobject.o Objects/descrobject.o Objects/enumobject.o Objects/exceptions.o Objects/genericaliasobject.o Objects/genobject.o Objects/fileobject.o Objects/floatobject.o Objects/frameobject.o Objects/funcobject.o Objects/interpolationobject.o Objects/iterobject.o Objects/lazyimportobject.o Objects/listobject.o Objects/longobject.o Objects/dictobject.o Objects/odictobject.o Objects/memoryobject.o Objects/methodobject.o Objects/moduleobject.o Objects/namespaceobject.o Objects/object.o Objects/obmalloc.o Objects/picklebufobject.o Objects/rangeobject.o Objects/sentinelobject.o Objects/setobject.o Objects/sliceobject.o Objects/structseq.o Objects/templateobject.o Objects/tupleobject.o Objects/typeobject.o Objects/typevarobject.o Objects/unicode_format.o Objects/unicode_formatter.o Objects/unicode_writer.o Objects/unicodectype.o Objects/unicodeobject.o Objects/unionobject.o Objects/weakrefobject.o Python/asm_trampoline.o Python/_contextvars.o Python/_warnings.o Python/Python-ast.o Python/Python-tokenize.o Python/asdl.o Python/assemble.o Python/ast.o Python/ast_preprocess.o Python/ast_unparse.o Python/bltinmodule.o Python/brc.o Python/ceval.o Python/codecs.o Python/codegen.o Python/compile.o Python/context.o Python/critical_section.o Python/crossinterp.o Python/dynamic_annotations.o Python/errors.o Python/flowgraph.o Python/frame.o Python/frozenmain.o Python/future.o Python/gc.o Python/gc_free_threading.o Python/gc_gil.o Python/getargs.o Python/getcompiler.o Python/getcopyright.o Python/getplatform.o Python/getversion.o Python/ceval_gil.o Python/hamt.o Python/hashtable.o Python/import.o Python/importdl.o Python/index_pool.o Python/initconfig.o Python/interpconfig.o Python/instrumentation.o Python/instruction_sequence.o Python/intrinsics.o Python/jit.o jit_shim-x86_64-unknown-linux-gnu.o Python/legacy_tracing.o Python/lock.o Python/marshal.o Python/modsupport.o Python/mysnprintf.o Python/mystrtoul.o Python/object_stack.o Python/optimizer.o Python/optimizer_analysis.o Python/optimizer_symbols.o Python/parking_lot.o Python/pathconfig.o Python/preconfig.o Python/pyarena.o Python/pyctype.o Python/pyfpe.o Python/pyhash.o Python/pylifecycle.o Python/pymath.o Python/pystate.o Python/pystats.o Python/pythonrun.o Python/pytime.o Python/qsbr.o Python/bootstrap_hash.o Python/specialize.o Python/stackrefs.o Python/structmember.o Python/symtable.o Python/sysmodule.o Python/thread.o Python/traceback.o Python/tracemalloc.o Python/uniqueid.o Python/getopt.o Python/pystrcmp.o Python/pystrtod.o Python/pystrhex.o Python/dtoa.o Python/fileutils.o Python/suggestions.o Python/perf_trampoline.o Python/perf_jit_trampoline.o Python/jit_unwind.o Python/remote_debugging.o Python/dynload_shlib.o     Modules/config.o Modules/main.o Modules/gcmodule.o Modules/_asynciomodule.o  Modules/atexitmodule.o  Modules/faulthandler.o  Modules/posixmodule.o  Modules/signalmodule.o  Modules/_tracemalloc.o  Modules/_suggestions.o  Modules/_datetimemodule.o  Modules/_codecsmodule.o  Modules/_collectionsmodule.o  Modules/errnomodule.o  Modules/_io/_iomodule.o Modules/_io/iobase.o Modules/_io/fileio.o Modules/_io/bytesio.o Modules/_io/bufferedio.o Modules/_io/textio.o Modules/_io/stringio.o  Modules/itertoolsmodule.o  Modules/_sre/sre.o  Modules/_sysconfig.o  Modules/_threadmodule.o  Modules/timemodule.o  Modules/_typesmodule.o  Modules/_typingmodule.o  Modules/_weakref.o  Modules/_abc.o  Modules/_functoolsmodule.o  Modules/_localemodule.o  Modules/_opcode.o  Modules/_operator.o  Modules/_stat.o  Modules/symtablemodule.o  Modules/pwdmodule.o -ldl         -lm                      -lm\n./Programs/_freeze_module getpath ./Modules/getpath.py Python/frozen_modules/getpath.h\ngcc -c -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer  -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O3 -Wall    -fno-semantic-interposition -std=c11 -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden -D_Py_TIER2=3 -D_Py_JIT  -I./Include/internal -I./Include/internal/mimalloc  -I. -I./Include    -DPy_BUILD_CORE -DPYTHONPATH='\"\"' \\\n\t-DPREFIX='\"/tmp/prefix\"' \\\n\t-DEXEC_PREFIX='\"/tmp/prefix\"' \\\n\t-DVERSION='\"3.15\"' \\\n\t-DVPATH='\"\"' \\\n\t-DPLATLIBDIR='\"lib\"' \\\n\t-DPYTHONFRAMEWORK='\"\"' \\\n\t-o Modules/getpath.o ./Modules/getpath.c\n[...]\n```\nand eventually getting to the expected `install` rules as before.\n\nWhile from a quick glance it appears that in most cases the only negative affect is the needless repetion of some build rules, this change in behavior was noticed because it caused macOS installer builds to produce broken Python 3.15 installations due to an previously existing and unreported bug now documented and fixed in gh-149351. While this was a fairly esoteric case that flushed out another bug, it is possible that other build pipelines might also be affected by this change in behavior.\n\nI was able to reproduce this change in behavior on both Linux (Ubuntu) and macOS systems with different build versions of build tools including different versions of Make. I did spend a fair amount of time tracking this down and distilling to a simple test case. From the output, it seems like the rebuilds are being triggered by the build rule for `jit.o`:\n\n```\nPython/jit.o: $(srcdir)/Python/jit.c @JIT_STENCILS_H@\n\t$(CC) -c $(PY_CORE_CFLAGS) -o $@ $\u003c\n```\n\nbut I'm not seeing exactly why that rule is being triggered again. I hope someone else does!  I'm marking this as a deferred-blocker pending RM review.\n\ncc: @diegorusso @savannahostrowski \n\n### CPython versions tested on:\n\nCPython main branch\n\n### Operating systems tested on:\n\nLinux, macOS\n\n\u003c!-- gh-linked-prs -->\n### Linked PRs\n* gh-149387\n\u003c!-- /gh-linked-prs -->\n","author":{"url":"https://github.com/ned-deily","@type":"Person","name":"ned-deily"},"datePublished":"2026-05-04T11:49:58Z","url":"https://github.com/python/cpython/issues/149353"}