Commit Graph

213 Commits

Author SHA1 Message Date
Guido van Rossum
142502ea8d Tier 2 cleanups and tweaks (#115534)
* Rename `_testinternalcapi.get_{uop,counter}_optimizer` to `new_*_optimizer`
* Use `_PyUOpName()` instead of` _PyOpcode_uop_name[]`
* Add `target` to executor iterator items -- `list(ex)` now returns `(opcode, oparg, target, operand)` quadruples
* Add executor methods `get_opcode()` and `get_oparg()` to get `vmdata.opcode`, `vmdata.oparg`
* Define a helper for printing uops, and unify various places where they are printed
* Add a hack to summarize_stats.py to fix legacy uop names (e.g. `POP_TOP` -> `_POP_TOP`)
* Define helpers in `test_opt.py` for accessing the set or list of opnames of an executor
2024-02-20 20:24:35 +00:00
Ken Jin
7cce857622 gh-114058: Foundations of the Tier2 redundancy eliminator (GH-115085)
---------

Co-authored-by: Mark Shannon <9448417+markshannon@users.noreply.github.com>
Co-authored-by: Jules <57632293+JuliaPoo@users.noreply.github.com>
Co-authored-by: Guido van Rossum <gvanrossum@users.noreply.github.com>
2024-02-13 21:24:48 +08:00
Mark Shannon
8144661017 GH-113710: Fix updating of dict version tag and add watched dict stats (GH-115221) 2024-02-12 16:07:38 +00:00
Mark Shannon
e66d0399cc GH-114806. Don't specialize calls to classes with metaclasses. (GH-114870) 2024-02-01 19:39:32 +00:00
Michael Droettboom
ea3cd0498c gh-114312: Collect stats for unlikely events (GH-114493) 2024-01-25 11:10:51 +00:00
Peter Lazorchak
f653caa5a8 gh-89811: Check for valid tp_version_tag in specializer (GH-113558) 2024-01-11 13:33:05 +08:00
Mark Shannon
0ae60b66de GH-113486: Do not emit spurious PY_UNWIND events for optimized calls to classes. (GH-113680) 2024-01-05 09:45:22 +00:00
Mark Shannon
e96f26083b GH-111485: Generate instruction and uop metadata (GH-113287) 2023-12-20 14:27:25 +00:00
Guido van Rossum
7316dfb0eb gh-112320: Implement on-trace confidence tracking for branches (#112321)
We track the confidence as a scaled int.
2023-12-12 21:43:08 +00:00
Mark Shannon
a7b0f63cdb GH-111772: Specialize slot loads and stores for _Py_T_OBJECT (GH-111773) 2023-11-06 13:55:04 +00:00
Michael Droettboom
84b4533e84 gh-109329: Count tier2 opcode misses (#110561)
This keeps a separate 'miss' counter for each micro-opcode, incremented whenever a guard uop takes a deoptimization side exit.
2023-10-30 17:02:45 -07:00
Sam Gross
6dfb8fe023 gh-110481: Implement biased reference counting (gh-110764) 2023-10-30 16:06:09 +00:00
Irit Katriel
a0c414c35d gh-111354: define names for RESUME oparg values (#111365) 2023-10-26 16:30:18 +01:00
Irit Katriel
67a91f78e4 gh-109094: replace frame->prev_instr by frame->instr_ptr (#109095) 2023-10-26 13:43:10 +00:00
Mark Shannon
b0699aa544 GH-111213: Fix a few broken stats (GH-111216) 2023-10-26 11:33:12 +01:00
Michael Droettboom
9eb2489266 gh-109329: Add stat for "trace too short" (GH-110402) 2023-10-05 16:12:06 +01:00
Michael Droettboom
e561e98058 GH-109329: Add tier 2 stats (GH-109913) 2023-10-04 14:52:28 -07:00
Brandt Bucher
22e65eecaa GH-105848: Replace KW_NAMES + CALL with LOAD_CONST + CALL_KW (GH-109300) 2023-09-13 10:25:45 -07:00
Michael Droettboom
5dcbbd8861 GH-109330: Dump and compare stats using opcode names, not numbers (GH-109335) 2023-09-12 14:12:57 -07:00
Guido van Rossum
bcce5e2718 gh-109039: Branch prediction for Tier 2 interpreter (#109038)
This adds a 16-bit inline cache entry to the conditional branch instructions POP_JUMP_IF_{FALSE,TRUE,NONE,NOT_NONE} and their instrumented variants, which is used to keep track of the branch direction.

Each time we encounter these instructions we shift the cache entry left by one and set the bottom bit to whether we jumped.

Then when it's time to translate such a branch to Tier 2 uops, we use the bit count from the cache entry to decided whether to continue translating the "didn't jump" branch or the "jumped" branch.

The counter is initialized to a pattern of alternating ones and zeros to avoid bias.

The .pyc file magic number is updated. There's a new test, some fixes for existing tests, and a few miscellaneous cleanups.
2023-09-11 18:20:24 +00:00
Victor Stinner
fd5989bda1 gh-108753: _Py_PrintSpecializationStats() uses Py_hexdigits (#109040) 2023-09-07 04:47:57 +02:00
Victor Stinner
a0773b89df gh-108753: Enhance pystats (#108754)
Statistics gathering is now off by default. Use the "-X pystats"
command line option or set the new PYTHONSTATS environment variable
to 1 to turn statistics gathering on at Python startup.

Statistics are no longer dumped at exit if statistics gathering was
off or statistics have been cleared.

Changes:

* Add PYTHONSTATS environment variable.
* sys._stats_dump() now returns False if statistics are not dumped
  because they are all equal to zero.
* Add PyConfig._pystats member.
* Add tests on sys functions and on setting PyConfig._pystats to 1.
* Add Include/cpython/pystats.h and Include/internal/pycore_pystats.h
  header files.
* Rename '_py_stats' variable to '_Py_stats'.
* Exclude Include/cpython/pystats.h from the Py_LIMITED_API.
* Move pystats.h include from object.h to Python.h.
* Add _Py_StatsOn() and _Py_StatsOff() functions. Remove
  '_py_stats_struct' variable from the API: make it static in
  specialize.c.
* Document API in Include/pystats.h and Include/cpython/pystats.h.
* Complete pystats documentation in Doc/using/configure.rst.
* Don't write "all zeros" stats: if _stats_off() and _stats_clear()
  or _stats_dump() were called.
* _PyEval_Fini() now always call _Py_PrintSpecializationStats() which
  does nothing if stats are all zeros.

Co-authored-by: Michael Droettboom <mdboom@gmail.com>
2023-09-06 15:54:59 +00:00
Victor Stinner
b298b395e8 gh-108765: Cleanup #include in Python/*.c files (#108977)
Mention one symbol imported by each #include.
2023-09-06 15:56:08 +02:00
Guido van Rossum
59e46932c8 gh-108488: Initialize JUMP_BACKWARD cache to 0, not 17 (#108591)
This mis-initialization caused the executor optimization to kick in sooner than intended. It also set the lower 4 bits of the counter to `1` -- those bits are supposed to be reserved (the actual counter is in the upper 12 bits).
2023-08-29 18:14:56 +00:00
Dong-hee Na
66b4d9c9f0 gh-107265: Revert "Ensure _PyCode_Quicken does not handle ENTER_EXECUTOR" (#108485)
This reverts commit d6ac5c7b10.

Reason: the assert we just added could be triggered (see issue).
2023-08-25 15:23:39 +00:00
Dong-hee Na
d6ac5c7b10 gh-107265: Ensure _PyCode_Quicken does not handle ENTER_EXECUTOR (gh-108460) 2023-08-24 23:03:26 +00:00
Irit Katriel
72119d16a5 gh-105481: remove regen-opcode. Generated _PyOpcode_Caches in regen-cases. (#108367) 2023-08-23 18:39:00 +01:00
Brandt Bucher
326f0ba1c5 GH-106485: Dematerialize instance dictionaries when possible (GH-106539) 2023-08-09 19:14:50 +00:00
Brandt Bucher
a9caf9cf90 GH-105848: Simplify the arrangement of CALL's stack (GH-107788) 2023-08-09 18:19:39 +00:00
Brandt Bucher
ea72c6fe3b GH-107596: Specialize str[int] (GH-107597) 2023-08-08 13:42:43 -07:00
Mark Shannon
2ba7c7f7b1 Add some GC stats to Py_STATS (GH-107581) 2023-08-04 10:34:23 +01:00
Victor Stinner
1a3faba9f1 gh-106869: Use new PyMemberDef constant names (#106871)
* Remove '#include "structmember.h"'.
* If needed, add <stddef.h> to get offsetof() function.
* Update Parser/asdl_c.py to regenerate Python/Python-ast.c.
* Replace:

  * T_SHORT => Py_T_SHORT
  * T_INT => Py_T_INT
  * T_LONG => Py_T_LONG
  * T_FLOAT => Py_T_FLOAT
  * T_DOUBLE => Py_T_DOUBLE
  * T_STRING => Py_T_STRING
  * T_OBJECT => _Py_T_OBJECT
  * T_CHAR => Py_T_CHAR
  * T_BYTE => Py_T_BYTE
  * T_UBYTE => Py_T_UBYTE
  * T_USHORT => Py_T_USHORT
  * T_UINT => Py_T_UINT
  * T_ULONG => Py_T_ULONG
  * T_STRING_INPLACE => Py_T_STRING_INPLACE
  * T_BOOL => Py_T_BOOL
  * T_OBJECT_EX => Py_T_OBJECT_EX
  * T_LONGLONG => Py_T_LONGLONG
  * T_ULONGLONG => Py_T_ULONGLONG
  * T_PYSSIZET => Py_T_PYSSIZET
  * T_NONE => _Py_T_NONE
  * READONLY => Py_READONLY
  * PY_AUDIT_READ => Py_AUDIT_READ
  * READ_RESTRICTED => Py_AUDIT_READ
  * PY_WRITE_RESTRICTED => _Py_WRITE_RESTRICTED
  * RESTRICTED => (READ_RESTRICTED | _Py_WRITE_RESTRICTED)
2023-07-25 15:28:30 +02:00
Victor Stinner
ee15844db8 gh-106320: Move _PyMethodWrapper_Type to internal C API (#107064) 2023-07-22 20:57:59 +00:00
Mark Shannon
0c90e75610 GH-100288: Specialize LOAD_ATTR for simple class attributes. (#105990)
* Add two more specializations of LOAD_ATTR.
2023-07-10 11:40:35 +01:00
Guido van Rossum
17af98227f gh-106320: Fix specialize.c compilation by including pycore_pylifecycle.h (#106434)
Compilation of Python/specialize.c was broken on macOS for me by gh-106400.
2023-07-04 21:42:12 +00:00
Brandt Bucher
7b2d94d875 GH-106008: Make implicit boolean conversions explicit (GH-106003) 2023-06-29 13:49:54 -07:00
Guido van Rossum
51fc725117 gh-104584: Baby steps towards generating and executing traces (#105924)
Added a new, experimental, tracing optimizer and interpreter (a.k.a. "tier 2"). This currently pessimizes, so don't use yet -- this is infrastructure so we can experiment with optimizing passes. To enable it, pass ``-Xuops`` or set ``PYTHONUOPS=1``. To get debug output, set ``PYTHONUOPSDEBUG=N`` where ``N`` is a debug level (0-4, where 0 is no debug output and 4 is excessively verbose).

All of this code is likely to change dramatically before the 3.13 feature freeze. But this is a first step.
2023-06-26 19:02:57 -07:00
Mark Shannon
04492cbc9a GH-91095: Specialize calls to normal Python classes. (GH-99331) 2023-06-22 09:48:19 +01:00
Brandt Bucher
2beab5bdef GH-105840: Fix assertion failures when specializing calls with too many __defaults__ (GH-105847) 2023-06-16 11:01:15 -07:00
Mark Shannon
e830289c52 GH-105229: Remove remaining two-codeunit superinstructions (GH-105326)
* Remove LOAD_CONST__LOAD_FAST and LOAD_FAST__LOAD_CONST superinstructions.
2023-06-08 12:35:34 +01:00
Mark Shannon
0689340366 GH-105229: Replace some superinstructions with single instruction equivalent. (GH-105230) 2023-06-05 11:07:04 +01:00
Brandt Bucher
1eb950ca55 GH-104405: Add missing PEP 523 checks (GH-104406) 2023-05-12 22:23:13 +00:00
Carl Meyer
77262458fe gh-87729: improve hit rate of LOAD_SUPER_ATTR specialization (#104270) 2023-05-11 08:08:13 -06:00
penguin_wwy
373bca0cc5 GH-102181: Improve specialization stats for SEND (GH-102182) 2023-05-10 22:40:59 +00:00
Carl Meyer
afe7703744 gh-104184: fix building --with-pydebug --enable-pystats (#104217) 2023-05-09 08:53:19 -06:00
Mark Shannon
738c226786 GH-103082: Code cleanup in instrumentation code (#103474) 2023-04-29 04:51:55 +00:00
Carl Meyer
ebf97c50f2 gh-103978: avoid using 'class' as an identifier (#103979) 2023-04-28 19:20:50 +00:00
Jelle Zijlstra
6c4124d11a gh-103879: Fix refleak in super specialization (#103882) 2023-04-26 08:50:19 -07:00
Carl Meyer
ef25febcf2 gh-87729: specialize LOAD_SUPER_ATTR_METHOD (#103809) 2023-04-25 17:45:51 +00:00
Mark Shannon
411b169281 GH-103082: Implementation of PEP 669: Low Impact Monitoring for CPython (GH-103083)
* The majority of the monitoring code is in instrumentation.c

* The new instrumentation bytecodes are in bytecodes.c

* legacy_tracing.c adapts the new API to the old sys.setrace and sys.setprofile APIs
2023-04-12 12:04:55 +01:00