Commit Graph

194 Commits

Author SHA1 Message Date
Sam Gross
5aa91c56bf gh-124296: Remove private dictionary version tag (PEP 699) (#124472) 2024-10-01 12:39:56 -04:00
Savannah Ostrowski
65f1237098 GH-123516: Improve JIT memory consumption by invalidating cold executors (GH-124443)
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
2024-09-27 00:35:42 +00:00
Ken Jin
198756b0f6 gh-117376: Fix off-by-ones in conversion functions (GH-124301)
Fix off-by-ones in conversion function
2024-09-26 02:41:07 +08:00
Sam Gross
f4997bb3ac gh-123923: Defer refcounting for f_funcobj in _PyInterpreterFrame (#124026)
Use a `_PyStackRef` and defer the reference to `f_funcobj` when
possible. This avoids some reference count contention in the common case
of executing the same code object from multiple threads concurrently in
the free-threaded build.
2024-09-24 20:08:18 +00:00
Ken Jin
8810e286fa gh-121459: Deferred LOAD_GLOBAL (GH-123128)
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: Sam Gross <655866+colesbury@users.noreply.github.com>
2024-09-14 00:23:51 +08:00
Sam Gross
b2afe2aae4 gh-123923: Defer refcounting for f_executable in _PyInterpreterFrame (#123924)
Use a `_PyStackRef` and defer the reference to `f_executable` when
possible. This avoids some reference count contention in the common case
of executing the same code object from multiple threads concurrently in
the free-threaded build.
2024-09-12 12:37:06 -04:00
Mark Shannon
4ed7d1d6ac GH-123996: Explicitly mark 'self_or_null' as an array of size 1 to ensure that it is kept in memory for calls (GH-124003) 2024-09-12 15:32:45 +01:00
Savannah Ostrowski
1fbc118c5d GH-123545: Remove duplicate Py_DECREF when handling _PyOptimizer_Optimize errors (GH-123546) 2024-09-05 10:56:07 -07:00
Victor Stinner
f1a0d96f41 gh-123091: Use _Py_IsImmortalLoose() (#123511)
Use _Py_IsImmortalLoose() in bytesobject.c, typeobject.c
and ceval.c.
2024-09-02 14:25:19 +02:00
Mark Shannon
54a05a4600 GH-123232: Factor BINARY_SLICE and STORE_SLICE to handle stats properly for tier 2. (GH-123381) 2024-08-27 10:49:39 +01:00
Kirill Podoprigora
67f2c84bff gh-123205: Python/bytecodes.c: Fix compiler warning (#123206)
Fix MSVC warning "conversion from '__int64' to 'int'"
2024-08-23 15:35:25 -04:00
Mark Shannon
0b0f7befad GH-123232: Fix "not specialized" stats (GH-123236) 2024-08-23 10:46:03 +01:00
Donghee Na
297f2e093e gh-123083: Fix a potential use-after-free in `STORE_ATTR_WITH_HINT` (gh-123092) 2024-08-22 23:49:09 +09:00
Mark Shannon
a4fd7aa4a6 GH-115776: Allow any fixed sized object to have inline values (GH-123192) 2024-08-21 15:52:04 +01:00
Mark Shannon
1eba8bae92 GH-123185: Check for NULL after calling _PyEvalFramePushAndInit (GH-123194) 2024-08-21 12:44:56 +01:00
Mark Shannon
bb1d30336e GH-118093: Make CALL_ALLOC_AND_ENTER_INIT suitable for tier 2. (GH-123140)
* Convert CALL_ALLOC_AND_ENTER_INIT to micro-ops such that tier 2 supports it

* Allow inexact arguments for CALL_ALLOC_AND_ENTER_INIT.
2024-08-20 16:52:58 +01:00
Mark Shannon
c13e7d98fb GH-118093: Specialize CALL_KW (GH-123006) 2024-08-16 17:11:24 +01:00
Brandt Bucher
f84754b705 GH-118093: Turn some DEOPT_IFs into EXIT_IFs (GH-122998) 2024-08-14 07:54:42 -07:00
Mark Shannon
eec7bdaf01 GH-120024: Remove CHECK_EVAL_BREAKER macro. (GH-122968)
* Factor some instructions into micro-ops to isolate CHECK_EVAL_BREAKER for escape analysis

* Eliminate CHECK_EVAL_BREAKER macro
2024-08-14 12:04:05 +01:00
Brandt Bucher
9621a7d017 GH-118093: Handle some polymorphism before requiring progress in tier two (GH-122843) 2024-08-12 12:39:31 -07:00
Sam Gross
ab094d1b2b gh-117139: Replace _PyList_FromArraySteal with stack ref variant (#122830)
This replaces `_PyList_FromArraySteal` with `_PyList_FromStackRefSteal`.
It's functionally equivalent, but takes a `_PyStackRef` array instead of
an array of `PyObject` pointers.

Co-authored-by: Ken Jin <kenjin@python.org>
2024-08-12 14:49:49 -04:00
Sam Gross
736fe4d23e gh-117139: Fix a few _PyStackRef related bugs (#122831)
`BUILD_SET` should use a borrow instead of a steal. The cleanup in `_DO_CALL`
`CONVERSION_FAILED` was incorrect.

Co-authored-by: Ken Jin <kenjin@python.org>
2024-08-12 14:49:33 -04:00
Sam Gross
3e753c689a gh-118926: Spill deferred references to stack in cases generator (#122748)
This automatically spills the results from `_PyStackRef_FromPyObjectNew`
to the in-memory stack so that the deferred references are visible to
the GC before we make any possibly escaping call.

Co-authored-by: Ken Jin <kenjin@python.org>
2024-08-07 13:23:53 -04:00
Sam Gross
674a50ef2f gh-117139: Fix an incorrect borrow in bytecodes.c (#122318)
`_PyDict_SetItem_Take2` steals both the key (i.e., `sub`) and the value.
2024-08-07 19:06:19 +05:30
Mark Shannon
4c31791848 GH-120024: Move three more escaping calls out of conditional statements (GH-122734) 2024-08-06 14:14:52 +01:00
Mark Shannon
a8be8fc6c4 GH-120024: Refactor code a bit so that escaping calls can be wrapped in spill code in code generator (GH-122693) 2024-08-06 08:40:39 +01:00
Mark Shannon
5bd72912a1 GH-122616: Simplify LOAD_ATTR_WITH_HINT and STORE_ATTR_WITH_HINT (GH-122620) 2024-08-05 16:27:48 +01:00
Mark Shannon
7aca84e557 GH-117224: Move the body of a few large-ish micro-ops into helper functions (GH-122601) 2024-08-02 16:31:17 +01:00
Mark Shannon
df13a1821a GH-118095: Add tier two support for BINARY_SUBSCR_GETITEM (GH-120793) 2024-08-01 16:19:05 -07:00
Mark Shannon
a9d56e38a0 GH-122155: Track local variables between pops and pushes in cases generator (GH-122286) 2024-08-01 09:27:26 +01:00
Brandt Bucher
15d4cd0967 GH-116090: Fire RAISE events from _FOR_ITER_TIER_TWO (GH-122413) 2024-07-29 12:17:47 -07:00
Brandt Bucher
64857d849f GH-122294: Burn in the addresses of side exits (GH-122295) 2024-07-26 09:40:15 -07:00
Mark Shannon
95a73917cd GH-122029: Break INSTRUMENTED_CALL into micro-ops, so that its behavior is consistent with CALL (GH-122177) 2024-07-26 14:35:57 +01:00
Mark Shannon
afb0aa6ed2 GH-121131: Clean up and fix some instrumented instructions. (GH-121132)
* Add support for 'prev_instr' to code generator and refactor some INSTRUMENTED instructions
2024-07-26 12:24:12 +01:00
Brandt Bucher
d9efa45d74 GH-118093: Add tier two support for BINARY_OP_INPLACE_ADD_UNICODE (GH-122253) 2024-07-25 14:45:07 -07:00
Brandt Bucher
5f6001130f GH-118093: Add tier two support for LOAD_ATTR_PROPERTY (GH-122283) 2024-07-25 10:45:28 -07:00
Mark Shannon
5e686ff57d GH-122034: Add StackRef variants of type checks to reduce the number of PyStackRef_AsPyObjectBorrow calls (GH-122037) 2024-07-25 18:32:43 +01:00
Mark Shannon
2e14a52cce GH-122160: Remove BUILD_CONST_KEY_MAP opcode. (GH-122164) 2024-07-25 16:24:29 +01:00
Brandt Bucher
794546fd53 GH-118093: Remove invalidated executors from side exits (GH-121885) 2024-07-24 09:16:30 -07:00
Brandt Bucher
7b36b67b1e GH-118093: Add tier two support to several instructions (GH-121884) 2024-07-18 14:24:58 -07:00
Mark Shannon
3eacfc1a4d GH-121784: Generate an error during code gen if a variable is marked unused, but is used and thus cached in a prior uop. (#121788)
* Reject uop definitions that declare values as 'unused' that are already cached by prior uops

* Track which variables are defined and only load from memory when needed

* Support explicit `flush` in macro definitions. 

* Make sure stack is flushed in where needed.
2024-07-18 12:49:24 +01:00
Mark Shannon
8ad6067bd4 GH-121012: Set index to -1 when list iterators become exhausted in tier 2 (GH-121483) 2024-07-08 14:20:13 +01:00
Sam Gross
8e8d202f55 gh-117139: Add _PyTuple_FromStackRefSteal and use it (#121244)
Avoids the extra conversion from stack refs to PyObjects.
2024-07-02 12:30:14 -04:00
Brandt Bucher
33903c53db GH-116017: Get rid of _COLD_EXITs (GH-120960) 2024-07-01 13:17:40 -07:00
Ken Jin
e6543daf12 gh-117139: Fix a few wrong steals in bytecodes.c (GH-121127)
Fix a few wrong steals in bytecodes.c
2024-06-29 02:14:48 +08:00
Ken Jin
22b0de2755 gh-117139: Convert the evaluation stack to stack refs (#118450)
This PR sets up tagged pointers for CPython.

The general idea is to create a separate struct _PyStackRef for everything on the evaluation stack to store the bits. This forces the C compiler to warn us if we try to cast things or pull things out of the struct directly.

Only for free threading: We tag the low bit if something is deferred - that means we skip incref and decref operations on it. This behavior may change in the future if Mark's plans to defer all objects in the interpreter loop pans out.

This implies a strict stack reference discipline is required. ALL incref and decref operations on stackrefs must use the stackref variants. It is unsafe to untag something then do normal incref/decref ops on it.

The new incref and decref variants are called dup and close. They mimic a "handle" API operating on these stackrefs.

Please read Include/internal/pycore_stackref.h for more information!

---------

Co-authored-by: Mark Shannon <9448417+markshannon@users.noreply.github.com>
2024-06-27 03:10:43 +08:00
Mark Shannon
8f5a01707f GH-120982: Add stack check assertions to generated interpreter code (GH-120992) 2024-06-25 16:42:29 +01:00
Brandt Bucher
a47abdb45d GH-117062: Make _JUMP_TO_TOP a general absolute jump (GH-120854) 2024-06-24 08:35:10 -07:00
Irit Katriel
65a12c559c gh-120834: fix type of *_iframe field in _PyGenObject_HEAD declaration (#120835) 2024-06-24 10:23:38 +01:00
Mark Shannon
9cefcc0ee7 GH-120507: Lower the BEFORE_WITH and BEFORE_ASYNC_WITH instructions. (#120640)
* Remove BEFORE_WITH and BEFORE_ASYNC_WITH instructions.

* Add LOAD_SPECIAL instruction

* Reimplement `with` and `async with` statements using LOAD_SPECIAL
2024-06-18 12:17:46 +01:00