For several builtin functions, we now fall back to __main__.__dict__ for the globals
when there is no current frame and _PyInterpreterState_IsRunningMain() returns
true. This allows those functions to be run with Interpreter.call().
The affected builtins:
* exec()
* eval()
* globals()
* locals()
* vars()
* dir()
We take a similar approach with "stateless" functions, which don't use any
global variables.
Adds `_PyObject_GetMethodStackRef` which uses stackrefs and takes advantage of deferred reference counting in free-threading while calling method objects in vectorcall.
After gh-130704, the interpreter replaces some uses of `LOAD_FAST` with
`LOAD_FAST_BORROW` which avoid incref/decrefs by "borrowing" references
on the interpreter stack when the bytecode compiler can determine that
it's safe.
This change broke some checks in C API extensions that relied on
`Py_REFCNT()` of `1` to determine if it's safe to modify an object
in-place. Objects may have a reference count of one, but still be
referenced further up the interpreter stack due to borrowing of
references.
This provides a replacement function for those checks.
`PyUnstable_Object_IsUniqueReferencedTemporary` is more conservative:
it checks that the object has a reference count of one and that it exists as a
unique strong reference in the interpreter's stack of temporary
variables in the top most frame.
See also:
* https://github.com/numpy/numpy/issues/28681
Co-authored-by: Pieter Eendebak <pieter.eendebak@gmail.com>
Co-authored-by: T. Wouters <thomas@python.org>
Co-authored-by: mpage <mpage@cs.stanford.edu>
Co-authored-by: Mark Shannon <mark@hotpy.org>
Co-authored-by: Victor Stinner <vstinner@python.org>
In the free threaded build, the `_PyObject_LookupSpecial()` call can lead to
reference count contention on the returned function object becuase it
doesn't use stackrefs. Refactor some of the callers to use
`_PyObject_MaybeCallSpecialNoArgs`, which uses stackrefs internally.
This fixes the scaling bottleneck in the "lookup_special" microbenchmark
in `ftscalingbench.py`. However, the are still some uses of
`_PyObject_LookupSpecial()` that need to be addressed in future PRs.
* Add location information when accessing already closed stackref
* Add #def option to track closed stackrefs to provide precise information for use after free and double frees.
This fixes a fairly subtle bug involving finalizers and resurrection in
debug free threaded builds: if `_PyObject_ResurrectEnd` returns `1`
(i.e., the object was resurrected by a finalizer), it's not safe to
access the object because it might still be deallocated. For example:
* The finalizer may have exposed the object to another thread. That
thread may hold the last reference and concurrently deallocate it any
time after `_PyObject_ResurrectEnd()` returns `1`.
* `_PyObject_ResurrectEnd()` may call `_Py_brc_queue_object()`, which
may internally deallocate the object immediately if the owning thread
is dead.
Therefore, it's important not to access the object after it's
resurrected. We only violate this in two cases, and only in debug
builds:
* We assert that the object is tracked appropriately. This is now moved
up betewen the finalizer and the `_PyObject_ResurrectEnd()` call.
* The `--with-trace-refs` builds may need to remember the object if
it's resurrected. This is now handled by `_PyObject_ResurrectStart()`
and `_PyObject_ResurrectEnd()`.
Note that `--with-trace-refs` is currently disabled in `--disable-gil`
builds because the refchain hash table isn't thread-safe, but this
refactoring avoids an additional thread-safety issue.
* Implement C recursion protection with limit pointers for Linux, MacOS and Windows
* Remove calls to PyOS_CheckStack
* Add stack protection to parser
* Make tests more robust to low stacks
* Improve error messages for stack overflow
Revert "GH-91079: Implement C stack limits using addresses, not counters. (GH-130007)" for now
Unfortunatlely, the change broke some buildbots.
This reverts commit 2498c22fa0.
* Implement C recursion protection with limit pointers
* Remove calls to PyOS_CheckStack
* Add stack protection to parser
* Make tests more robust to low stacks
* Improve error messages for stack overflow
Fix a few thread-safety bugs to enable test_opcache when run with TSAN:
* Use relaxed atomics when clearing `ht->_spec_cache.getitem`
(gh-115999)
* Add temporary suppression for type slot modifications (gh-127266)
* Use atomic load when reading `*dictptr`
This exposes `_Py_TryIncref` as `PyUnstable_TryIncref()` and the helper
function `_PyObject_SetMaybeWeakref` as `PyUnstable_EnableTryIncRef`.
These are helpers for dealing with unowned references in a safe way,
particularly in the free threading build.
Co-authored-by: Petr Viktorin <encukou@gmail.com>
