Title: Make vectorcall and the native call ABI borrow their arguments · Issue #9850 · RustPython/RustPython · GitHub
Open Graph Title: Make vectorcall and the native call ABI borrow their arguments · Issue #9850 · RustPython/RustPython
X Title: Make vectorcall and the native call ABI borrow their arguments · Issue #9850 · RustPython/RustPython
Description: Summary RustPython's vectorcall copies the PEP 590 calling shape (nargs, kwnames) but not its main property: calling without copying the argument vector. Every generic call moves its arguments into an owned Vec, converts it into FuncArgs...
Open Graph Description: Summary RustPython's vectorcall copies the PEP 590 calling shape (nargs, kwnames) but not its main property: calling without copying the argument vector. Every generic call moves its arguments into...
X Description: Summary RustPython's vectorcall copies the PEP 590 calling shape (nargs, kwnames) but not its main property: calling without copying the argument vector. Every generic call moves its arguments ...
Opengraph URL: https://github.com/RustPython/RustPython/issues/9850
X: @github
Domain: github.com
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Make vectorcall and the native call ABI borrow their arguments","articleBody":"## Summary\nRustPython's vectorcall copies the PEP 590 calling shape (`nargs`, `kwnames`) but not its main property: calling without copying the argument vector. Every generic call moves its arguments into an owned `Vec`, converts it into `FuncArgs` (copying keyword names into a map), and then binds positional arguments by popping the front of that `Vec`. CPython does none of this. This issue compares the two designs and proposes a plan to make the native call path borrow its arguments.\n\nFound while reviewing #9758, where the method descriptor now does `args.remove(0)` to take the receiver out of band (`crates/vm/src/builtins/descriptor.rs`, `vectorcall_method_descriptor`). That shift is not new: before #9758 the same `remove(0)` ran inside `FuncArgs::take_positional`. It is a symptom of the owned-`Vec` design, not of the receiver change.\n\n## Comparison\n\n| | CPython | RustPython today |\n|---|---|---|\n| Vectorcall arguments | `PyObject *const *args`, borrowed from the caller's stack (`vectorcallfunc`, `Include/object.h`) | `Vec\u003cPyObjectRef>`, owned (`VectorCallFunc`, `crates/vm/src/types/slot.rs`) |\n| Skip the first argument | `args+1, nargs-1`, pointer arithmetic (`method_vectorcall_*`, `Objects/descrobject.c`) | `args.remove(0)`, memmove (`vectorcall_method_descriptor`) |\n| Prepend `self` | Write into the reserved `args[-1]` slot when `PY_VECTORCALL_ARGUMENTS_OFFSET` (`Include/abstract.h`) is set (`method_vectorcall`, `Objects/classobject.c`) | `args.insert(0, self)`, memmove (`vectorcall_bound_method`, `crates/vm/src/builtins/function.rs`; `frame.rs` call-with-self paths) |\n| Native function ABI | `METH_FASTCALL\\|METH_KEYWORDS`: `meth(self, args, nargs, kwnames)`, all borrowed (`cfunction_vectorcall_FASTCALL_KEYWORDS`, `Objects/methodobject.c`) | `PyNativeFn(vm, receiver, FuncArgs, Callee)`, where `FuncArgs` owns a `Vec` and a keyword map |\n| Keyword arguments | `kwnames` tuple passed through unchanged | `FuncArgs::from_vectorcall_owned` builds a map and copies every keyword name (`as_wtf8().to_owned()`) |\n| Positional binding | Index into `args` (Argument Clinic generated code) | `FuncArgs::take_positional` = `args.remove(0)` per argument, so O(n²) memmove over n arguments |\n\n## What already avoids the cost\n- The `frame.rs` specializations (`CallMethodDescriptor{Noargs,O,Fast,FastWithKeywords}`, builtin call specializations) take arguments directly from the value stack, and the receiver without shifting (`execute_call_method_descriptor`, `take_native_method_args`).\n- `#[from_args(borrowed)]` / `BorrowedArgs` binds against the caller's storage without per-argument ownership transfer.\n\nSo the hot specialized paths are mostly fine. The remaining cost is on every generic call: unspecialized sites, calls through `map`/`sorted(key=)`/`functools`, `PyObject_Vectorcall` from native code, bound Python methods, and C-API calls.\n\n## Plan\nEach step should land separately, keep every error message unchanged, and come with before/after benchmarks.\n\n1. **Borrowed vectorcall arguments.** Change `VectorCallFunc` to take a borrowed argument slice (`&[PyObjectRef]` or an equivalent borrowed view), `nargsf` with an `ARGUMENTS_OFFSET` bit, and `kwnames`. Callers that build arguments on the value stack pass a stack slice with no `Vec` allocation. Callers that want to prepend `self` reserve slot `-1` and set the offset bit, which removes the `insert(0, ...)` in `vectorcall_bound_method` and the call-with-self paths in `frame.rs`. Callees that need ownership (for example a Python frame's `localsplus`) clone from the slice, which costs the same refcount increments as today's move plus no allocation.\n2. **Native ABI without `FuncArgs` ownership.** Give native functions a borrowed argument view (`args`, `nargs`, `kwnames`), as `METH_FASTCALL|METH_KEYWORDS` does. `FuncArgs` stays as the owned form for `tp_call`-style and `*args/**kwargs` paths. The method descriptor then passes `&args[1..]` and the receiver with no shift, and builtin functions pass the slice unchanged.\n3. **Index-based binding.** Make `FromArgs` binders consume positionals by index (a cursor over the borrowed view) instead of `take_positional`'s `remove(0)`, and look keywords up in `kwnames` instead of a prebuilt map. This removes both the per-argument memmove and the keyword-name copies. Generated binders should match Clinic's parsing order and messages.\n4. **C-API.** Pass the borrowed view straight to `METH_FASTCALL` C functions instead of materializing a temporary array.\n\n## Validation\n- Workspace tests, C-API tests, Clippy, `-m test test_descr test_inspect test_builtin test_call test_capi test_functools`.\n- Micro-benchmarks before and after each step (release build, several interleaved samples, medians):\n - unspecialized builtin call with keywords (e.g. `sorted(l, key=f)` setup, `int(x, base=10)` through `map`)\n - unbound descriptor call through a generic path (`map(list.append, ...)`-style)\n - bound Python method call (`obj.method(x)` via a stored bound method)\n - `PyObject_Vectorcall` from a C extension\n\n— *commented by Claude*\n","author":{"url":"https://github.com/youknowone","@type":"Person","name":"youknowone"},"datePublished":"2026-11-15T07:49:47Z","url":"https://github.com/RustPython/RustPython/issues/9850"}
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format) |
| route-controller | voltron_issues_fragments |
| route-action | issue_layout |
| fetch-nonce | v2:13506b93-2086-0f80-51e6-65dc3d5f46b1 |
| current-catalog-service-hash | 81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114 |
| request-id | EAB6:4279A:2A06D42:38914BD:6B01D40D |
| html-safe-nonce | e5a587fb400c4a59d48ee3830c2e61bc4524db3d91d7b0a3461adf6281224450 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJFQUI2OjQyNzlBOjJBMDZENDI6Mzg5MTRCRDo2QjAxRDQwRCIsInZpc2l0b3JfaWQiOiI3NTgyODcyMTQ5OTUwNTgzODIxIiwicmVnaW9uX2VkZ2UiOiJpYWQiLCJyZWdpb25fcmVuZGVyIjoiaWFkIn0= |
| visitor-hmac | 1531b67fc1d70447e12656b3a26cc21c7788616d7381317bec9f8bc9eab5a61c |
| hovercard-subject-tag | issue:6401503709 |
| github-keyboard-shortcuts | repository,issues,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/RustPython/RustPython/9850/issue_layout |
| twitter:image | https://opengraph.githubassets.com/65e95153414104e35a80caa158400841249a5ad7c56bdf0e8fc396e158910f70/RustPython/RustPython/issues/9850 |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/65e95153414104e35a80caa158400841249a5ad7c56bdf0e8fc396e158910f70/RustPython/RustPython/issues/9850 |
| og:image:alt | Summary RustPython's vectorcall copies the PEP 590 calling shape (nargs, kwnames) but not its main property: calling without copying the argument vector. Every generic call moves its arguments into... |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| og:author:username | youknowone |
| hostname | github.com |
| expected-hostname | github.com |
| None | cc5dfdd32f62ebd3223cab3af3aa2b268d3605218237326a48a99d1efd89495c |
| turbo-cache-control | no-preview |
| go-import | github.com/RustPython/RustPython git https://github.com/RustPython/RustPython.git |
| octolytics-dimension-user_id | 39710557 |
| octolytics-dimension-user_login | RustPython |
| octolytics-dimension-repository_id | 135201145 |
| octolytics-dimension-repository_nwo | RustPython/RustPython |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 135201145 |
| octolytics-dimension-repository_network_root_nwo | RustPython/RustPython |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 8c77c9e8439031ad54c75a250694842892f693e1 |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
| react-app-name | issues |
Links:
Viewport: width=device-width