118 Commits

Author SHA1 Message Date
Serhiy Storchaka
d8e6bdc0d0 gh-135801: Add the module parameter to compile() etc (GH-139652)
Many functions related to compiling or parsing Python code, such as
compile(), ast.parse(), symtable.symtable(),
and importlib.abc.InspectLoader.source_to_code() now allow to pass
the module name used when filtering syntax warnings.
2025-11-13 13:21:32 +02:00
Krzysztof Magusiak
b723c8be07 gh-124503: Optimize ast.literal_eval() for small input (GH-137010)
The implementation does not create anymore local functions which reduces
the overhead for small inputs. Some other calls are inlined into a
single `_convert_literal` function.
We have a gain of 10-20% for small inputs and only 1-2% for bigger
inputs.
2025-07-31 12:55:00 +03:00
Serhiy Storchaka
4d31d19a1d gh-134718: Omit optional Load() values in ast.dump() (GH-134934) 2025-05-31 10:32:53 +03:00
Serhiy Storchaka
cc344e8dd0 gh-134718: Fix ast.dump() for empty non-default values (GH-134926) 2025-05-30 17:25:07 +03:00
Hugo van Kemenade
4ac916ae33 gh-130645: Add color to stdlib argparse CLIs (gh-133380) 2025-05-05 19:46:46 +02:00
Semyon Moroz
2b4e2b7830 gh-133367: Add missing options to ast CLI (#133369)
Co-authored-by: Stan Ulbrych <89152624+StanFromIreland@users.noreply.github.com>
Co-authored-by: sobolevn <mail@sobolevn.me>
2025-05-05 17:17:43 +00:00
Semyon Moroz
6ce60f1574 gh-131178: Add tests for ast command-line interface (#133329)
Co-authored-by: sobolevn <mail@sobolevn.me>
2025-05-04 21:11:13 +00:00
Hunter Hogan
6d53b75283 gh-133046: Reformat the `ast` module docstring (GH-133050)
* Fix formatting in module docstring for `ast` https://github.com/python/cpython/issues/133046
* #133046 ast docstring: remove header, dedent, 80-char width.
* Keep existing wrapping

---------

Co-authored-by: Adam Turner <9087854+aa-turner@users.noreply.github.com>
2025-04-27 14:35:56 -07:00
Jelle Zijlstra
06822bfbf6 gh-118761: Fix star-import of ast (#132025)
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
2025-04-02 18:15:05 +00:00
Adam Turner
f20f02e6b5 gh-118761: Optimise import time for ast (#131953) 2025-04-02 17:22:15 +01:00
Tomasz Pytel
a8cb5e4a43 gh-129598: ast: allow multi stmts for ast single with ';' (#129620) 2025-03-19 15:29:40 -07:00
Shantanu
8df5193d37 gh-127975: Avoid reusing quote types in ast.unparse if not needed (#127980) 2025-01-31 08:49:06 +00:00
Serhiy Storchaka
7d2c39752f gh-91818: Use default program name in the CLI of many modules (GH-124867)
As argparse now detects by default when the code was run as a module.

This leads to using the actual executable name instead of simply "python"
to display in the usage message ("usage: python -m ...").
2024-10-10 00:20:53 +03:00
Bénédikt Tran
15232a0819 gh-121210: handle nodes with missing attributes/fields in ast.compare (#121211) 2024-07-02 16:23:17 +05:30
Jelle Zijlstra
ae11d68ab9 gh-117865: Defer import of re in ast (#119546)
This is used only by ast.get_source_segment(), so it seems sensible to avoid importing it.

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-05-28 11:04:08 -07:00
Alex Waygood
0220663e26 gh-119562: Remove unused private string constants from ast.py (#119576) 2024-05-26 13:31:02 +00:00
Alex Waygood
008bc04dcb gh-119562: Remove AST nodes deprecated since Python 3.8 (#119563) 2024-05-26 12:34:48 +00:00
Batuhan Taskaya
d065edfb66 gh-60191: Implement ast.compare (#19211)
* bpo-15987: Implement ast.compare

Add a compare() function that compares two ASTs for structural equality. There are two set of attributes on AST node objects, fields and attributes. The fields are always compared, since they represent the actual structure of the code. The attributes can be optionally be included in the comparison. Attributes capture things like line numbers of column offsets, so comparing them involves test whether the layout of the program text is the same. Since whitespace seems inessential for comparing ASTs, the default is to compare fields but not attributes.

ASTs are just Python objects that can be modified in arbitrary ways. The API for ASTs is under-specified in the presence of user modifications to objects. The comparison respects modifications to fields and attributes, and to _fields and _attributes attributes. A user could create obviously malformed objects, and the code will probably fail with an AttributeError when that happens. (For example, adding "spam" to _fields but not adding a "spam" attribute to the object.) 

Co-authored-by: Jeremy Hylton <jeremy@alum.mit.edu>
2024-05-22 01:39:26 +00:00
Jelle Zijlstra
ca269e58c2 gh-116126: Implement PEP 696 (#116129)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
2024-05-03 06:17:32 -07:00
Nikita Sobolev
692e902c74 gh-116023: Add show_empty=False to ast.dump (#116037)
Co-authored-by: Carl Meyer <carl@oddbird.net>
2024-04-24 11:02:38 +03:00
Frank Hoffmann
69ab93082d gh-112364: Correct unparsing of backslashes and quotes in ast.unparse (#115696) 2024-02-21 10:24:08 +00:00
Shantanu
3d5df54cdc Clean up backslash avoiding code in ast, fix typo (#113605)
As of #108553, the `_avoid_backslashes` code path is dead

`scape_newlines` was introduced in #110271. Happy to drop the typo fix
if we don't want it
2024-01-16 11:24:25 -08:00
Serhiy Storchaka
b3d2427f22 gh-58032: Do not use argparse.FileType in module CLIs and scripts (GH-113649)
Open and close files manually. It prevents from leaking files,
preliminary creation of output files, and accidental closing of stdin
and stdout.
2024-01-10 15:07:19 +02:00
Pablo Galindo Salgado
cc389ef627 gh-110259: Fix f-strings with multiline expressions and format specs (#110271)
Signed-off-by: Pablo Galindo <pablogsal@gmail.com>
2023-10-05 14:26:44 +01:00
Shantanu
23f9f6f464 gh-108843: fix ast.unparse for f-string with many quotes (#108981) 2023-09-18 14:56:19 +01:00
Anthony Shaw
2c4c26c4ce gh-108469: Update ast.unparse for unescaped quote support from PEP701 [3.12] (#108553)
Co-authored-by: sunmy2019 <59365878+sunmy2019@users.noreply.github.com>
2023-09-05 21:01:23 +01:00
Irit Katriel
10a91d7e98 gh-108113: Make it possible to create an optimized AST (#108154) 2023-08-21 16:31:30 +00:00
Jelle Zijlstra
957a974d4f gh-104799: PEP 695 backward compatibility for ast.unparse (#105846) 2023-06-16 09:31:23 -07:00
Jelle Zijlstra
a5f244d627 gh-104656: Rename typeparams AST node to type_params (#104657) 2023-05-21 21:25:09 -07:00
Jelle Zijlstra
24d8b88420 gh-103763: Implement PEP 695 (#103764)
This implements PEP 695, Type Parameter Syntax. It adds support for:

- Generic functions (def func[T](): ...)
- Generic classes (class X[T](): ...)
- Type aliases (type X = ...)
- New scoping when the new syntax is used within a class body
- Compiler and interpreter changes to support the new syntax and scoping rules 

Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com>
Co-authored-by: Eric Traut <eric@traut.com>
Co-authored-by: Larry Hastings <larry@hastings.org>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-15 20:36:23 -07:00
Alex Waygood
376137f6ec gh-90953: Emit deprecation warnings for ast features deprecated in Python 3.8 (#104199)
`ast.Num`, `ast.Str`, `ast.Bytes`, `ast.Ellipsis` and `ast.NameConstant` now all emit deprecation warnings on import, access, instantation or `isinstance()` checks.

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2023-05-06 16:48:07 +00:00
Tian Gao
36860134a9 gh-103285: Rewrite _splitlines_no_ff to improve performance (#103307) 2023-04-23 23:03:49 -06:00
Batuhan Taskaya
1acdfec359 gh-99341: Cover type ignore nodes when incrementing line numbers (GH-99422) 2022-11-22 02:41:14 -08:00
Gregory P. Smith
8baef8ae36 gh-95588: Drop the safety claim from ast.literal_eval docs. (#95919)
It was never really safe and this claim conflicts directly with the big warning in the docs about it being able to crash the interpreter.
2022-10-01 17:55:40 -07:00
Shantanu
200c9a8da0 gh-92986: Fix ast.unparse when ImportFrom.level is None (#92992)
This doesn't happen naturally, but is allowed by the ASDL and compiler.
We don't want to change ASDL for backward compatibility reasons
(#57645, #92987)
2022-09-05 20:14:50 +03:00
Anthony Sottile
9c2b9348e2 ast.parse: check feature_version common case first (GH-94640) 2022-08-29 17:05:24 +03:00
Shantanu
a5dde0fe4f gh-95066: ast: Replace assert with ValueError (GH-95072)
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2022-07-26 11:43:09 +02:00
Batuhan Taskaya
f6fd8aac13 gh-92671: Don't omit parentheses when unparsing empty tuples (GH-92673) 2022-05-16 13:38:06 +01:00
Matthew Rahtz
e8e737bcf6 bpo-43224: Implement PEP 646 grammar changes (GH-31018)
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2022-03-26 09:55:35 -07:00
Irit Katriel
d60457a667 bpo-45292: [PEP-654] add except* (GH-29581) 2021-12-14 16:48:15 +00:00
Binbin
17b16e13bb Fix typos in multiple files (GH-26689)
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
2021-06-12 22:47:44 -04:00
Batuhan Taskaya
51cef8be8c bpo-44142: drop redundant parantheses when unparsing tuples as assignment targets (GH-26156) 2021-05-16 16:33:22 +03:00
Batuhan Taskaya
e4e931a67e bpo-44081: improve ast.unparse() for lambdas with no parameters (GH-26000) 2021-05-15 15:55:53 +03:00
Batuhan Taskaya
3d98ececda bpo-43417: Better buffer handling for ast.unparse (GH-24772) 2021-05-09 02:32:04 +03:00
Ikko Ashimine
9ee8448243 Fix typo in ast.py (GH-25740)
parantheses -> parentheses
2021-05-04 06:18:58 -07:00
Nick Coghlan
1e7b858575 bpo-43892: Make match patterns explicit in the AST (GH-25585)
Co-authored-by: Brandt Bucher <brandtbucher@gmail.com>
2021-04-28 22:58:44 -07:00
Ethan Furman
a02cb474f9 bpo-38659: [Enum] add _simple_enum decorator (GH-25497)
add:

* `_simple_enum` decorator to transform a normal class into an enum
* `_test_simple_enum` function to compare
* `_old_convert_` to enable checking `_convert_` generated enums

`_simple_enum` takes a normal class and converts it into an enum:

    @simple_enum(Enum)
    class Color:
        RED = 1
        GREEN = 2
        BLUE = 3

`_old_convert_` works much like` _convert_` does, using the original logic:

    # in a test file
    import socket, enum
    CheckedAddressFamily = enum._old_convert_(
            enum.IntEnum, 'AddressFamily', 'socket',
            lambda C: C.isupper() and C.startswith('AF_'),
            source=_socket,
            )

`_test_simple_enum` takes a traditional enum and a simple enum and
compares the two:

    # in the REPL or the same module as Color
    class CheckedColor(Enum):
        RED = 1
        GREEN = 2
        BLUE = 3

    _test_simple_enum(CheckedColor, Color)

    _test_simple_enum(CheckedAddressFamily, socket.AddressFamily)

Any important differences will raise a TypeError
2021-04-21 10:20:44 -07:00
Ethan Furman
503cdc7c12 Revert "bpo-38659: [Enum] add _simple_enum decorator (GH-25285)" (GH-25476)
This reverts commit dbac8f40e8.
2021-04-19 19:12:24 -07:00
Ethan Furman
dbac8f40e8 bpo-38659: [Enum] add _simple_enum decorator (GH-25285)
add:

_simple_enum decorator to transform a normal class into an enum
_test_simple_enum function to compare
_old_convert_ to enable checking _convert_ generated enums
_simple_enum takes a normal class and converts it into an enum:

@simple_enum(Enum)
class Color:
    RED = 1
    GREEN = 2
    BLUE = 3

_old_convert_ works much like _convert_ does, using the original logic:

# in a test file
import socket, enum
CheckedAddressFamily = enum._old_convert_(
        enum.IntEnum, 'AddressFamily', 'socket',
        lambda C: C.isupper() and C.startswith('AF_'),
        source=_socket,
        )

test_simple_enum takes a traditional enum and a simple enum and
compares the two:

# in the REPL or the same module as Color
class CheckedColor(Enum):
    RED = 1
    GREEN = 2
    BLUE = 3

_test_simple_enum(CheckedColor, Color)

_test_simple_enum(CheckedAddressFamily, socket.AddressFamily)

Any important differences will raise a TypeError
2021-04-19 18:04:53 -07:00
Kodi Arfer
08ff4369af bpo-43521: Allow ast.unparse with empty sets and NaN (GH-24897)
Automerge-Triggered-By: GH:pablogsal
2021-03-18 10:36:06 -07:00