Python 3.13.0b1: exec() does not populate locals()

Bug report

Bug description:

x.py

xxx = 118888

readx.py

def f():
    with open("x.py", encoding="utf-8") as f:
        exec(compile(f.read(), "x.py", "exec"))
    return locals()["xxx"]

print(f())

shell

$ python3.12 readx.py
118888

$ python3.13 readx.py
Traceback (most recent call last):
  File ".../readx.py", line 6, in <module>
    print(f())
          ~^^
  File ".../readx.py", line 4, in f
    return locals()["xxx"]
           ~~~~~~~~^^^^^^^
KeyError: 'xxx'

This breaks e.g. pillow 10.3.0 which has:

def get_version():
    version_file = "src/PIL/_version.py"
    with open(version_file, encoding="utf-8") as f:
        exec(compile(f.read(), version_file, "exec"))
    return locals()["__version__"]

In https://github.com/python-pillow/Pillow/blob/10.3.0/setup.py#L23

CPython versions tested on:

3.13

Operating systems tested on:

Linux

Linked PRs

@gaogaotiantian

Bisected to b034f14
cc @gaogaotiantian

This is an expected and intentional behavior change due to PEP 667. locals() now has a clear semantic when called inside a function - a snapshot of the local variables, and xxx (or __version__) is not one of them.

I won't even consider this is "breaking" as the docs clearly states:

modifications to the default locals dictionary should not be attempted. Pass an explicit locals dictionary if you need to see effects of the code on locals after function exec() returns.

So this is an illegal usage that happens to work in a favored way to begin with.

If you want the result of the local changes, pass in an explicit dictionary:

def get_version():
    version_file = "src/PIL/_version.py"
    d = {}
    with open(version_file, encoding="utf-8") as f:
        exec(compile(f.read(), version_file, "exec"), globals(), d)
    return d["__version__"]

I'm aware that this might be a bit inconvenience to the library maintainers, but this is the right way to go and we are making efforts to make locals() more consistent and predictable.

It it true that the 3.12 docs say that readx.py should not be expected to work. But it did then and previously, even though not now. What's New 3.13 only says

PEP 667: FrameType.f_locals when used in a function now returns a write-through proxy to the frame’s locals, rather than a dict. See the PEP for corresponding C API changes and deprecations.

From this, I would not expect changes in how locals() behaves, in particular in the effect of exec bindings. I think this should be mention.

Even the Python subsection of the PEP's Back Compatibility section says nothing. It only mentions a couple of things that do not change.

We are aware that the docs are not fully ready for beta 1, but this behavior is described in detail in locals. https://docs.python.org/3.13/library/functions.html#locals . We can add some notes to exec or eval or whatsnews, but the key change is actually on the locals() function, which the first version of docs is written for.

Can this be closed now the docs were updated in #119201?

@ncoghlan

@ncoghlan Does #119201 make this obsolete?

Yeah, the function level snapshot behaviour is now covered in the What's New porting guide: https://docs.python.org/3.13/whatsnew/3.13.html#changes-in-the-python-api

It is also mentioned in a versionchanged note on exec itself: https://docs.python.org/3.13/library/functions.html#exec

The general write-up of PEP 667 also mentions locals() first before covering FrameType.f_locals: https://docs.python.org/3.13/whatsnew/3.13.html#whatsnew313-locals-semantics

The most minimal change to fix this kind of exec invocation is to pass explicit target namespaces as suggested in #118888 (comment) (this exec call is already implicitly being called with separate globals and locals namespaces, so explicitly calling it that way won't change the behaviour of the executed code)

Alternatively, for the examples given, https://docs.python.org/3/library/runpy.html#runpy.run_path is a better tool when the task is "run the Python file at this location and return its top level namespace" (it will respect Python source encoding declarations properly, while explicitly opening the files as utf-8 ignores them).

Considering this further, I'm thinking it may be worth tweaking the text in "What's New" a bit, as somebody reading even the updated What's New entry might not make the leap from "the mutation semantics of locals() have changed in optimised scopes" to "the semantics of exec(), eval(), and other code execution APIs that default to targeting locals() have changed in optimised scopes".

[3.13] gh-118888: Further PEP 667 docs updates (gh-119894)

pythongh-118888: Further PEP 667 docs updates (pythongh-119893)

setup.py: Fix version retrieval on Python 3.13+

Update `setup.py` to accommodate `Python3.13.0`

py-Pillow: add py313 subport

Fix get_version for Python 3.13

Fixes __version__ not found error with Python 3.13 (see python/cpytho…

fix locals() writable, does not seem to work on py313

Fixes __version__ not found error with Python 3.13 (see python/cpytho…

Navigation Menu

Repository navigation

Python 3.13.0b1: exec() does not populate locals() #118888

Bug report

Bug description:

x.py

readx.py

shell

CPython versions tested on:

Operating systems tested on:

Linked PRs

5 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Python 3.13.0b1: exec() does not populate locals() #118888

Description

Bug report

Bug description:

x.py

readx.py

shell

CPython versions tested on:

Operating systems tested on:

Linked PRs

Activity

Eclips4 commented on May 10, 2024

gaogaotiantian commented on May 10, 2024

terryjreedy commented on May 10, 2024

gaogaotiantian commented on May 11, 2024

hugovk commented on May 25, 2024

terryjreedy commented on May 25, 2024

ncoghlan commented on May 30, 2024

ncoghlan commented on May 30, 2024

5 remaining items

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions