Skip to content

gh-158539: Fix exception mode missing handlers in generators/coroutines - #158581

Open
hetaozdh wants to merge 1 commit into
python:mainfrom
hetaozdh:gh-158539
Open

hetaozdh wants to merge 1 commit into
python:mainfrom
hetaozdh:gh-158539

Conversation

@hetaozdh

@hetaozdh hetaozdh commented Oct 1, 2026 •

Copy link
Copy Markdown

Fixes #158539.

Problem

Exception mode (--mode=exception) decides whether a thread is handling an
exception from THREAD_STATUS_HAS_EXCEPTION, which was computed by reading the
exc_state embedded in PyThreadState. Generators, coroutines and async
generators repoint tstate->exc_info at their own _PyErr_StackItem while
they run (gen_send_ex2()), so an except block running in one of them — or
in a function they call — stores the exception in that item and was never
sampled.

The practical effect is that exception mode reports almost nothing for asyncio
programs: every except runs in a coroutine, or in something called from one.
The run still reports a 0.00 error rate, so the missing samples are silent.

Fix

Compute the flag the way the interpreter does, by following tstate->exc_info
and its previous_item chain, like _PyErr_GetTopmostException() (the helper
behind sys.exception()):

  • export thread_state.exc_info and err_stackitem.previous_item in
    _Py_DebugOffsets (and register them with the offsets validation in
    _remote_debugging);
  • read exc_info from the thread state buffer. When it points at the embedded
    exc_state — the common case, e.g. a thread-level handler — keep the
    existing zero-extra-read fast path. Otherwise treat it as a remote
    _PyErr_StackItem and walk the chain until a non-NULL exc_value is found.

A failed remote read is best-effort: it clears the error and reports "no
exception", rather than turning one unreadable stack item into a failed
get_stack_trace() call.

Tests

Lib/test/test_external_inspection.py:

  • four new scenarios in TestExceptionDetectionScenarios covering a handler in
    a generator, in a function called from a generator expression, in a
    coroutine, and in a function called from a coroutine;
  • a new TestExceptionDetectionInProcess class that inspects the current
    process, so it does not need subprocess debugging permissions (macOS) and can
    run on the buildbots. Besides the positive cases it covers the exception-mode
    thread filter and two negative cases (a generator with no exception, and a
    generator's finally after a handled exception).

All four positive cases fail without the fix and pass with it; test_profiling
and test_external_inspection pass with it.

Notes

AI tooling was used while preparing this change; I have reviewed it in detail
and can explain all of it.

…routines

The sampling profiler's exception mode decided whether a thread was
handling an exception by reading the embedded PyThreadState.exc_state.
Generators, coroutines and async generators repoint tstate->exc_info
at their own _PyErr_StackItem while they run, so an except block
running in one of them (or in a function they call) stored the
exception in that item instead, and was never sampled.

Follow tstate->exc_info and its previous_item chain, mirroring
_PyErr_GetTopmostException(), and export the two debug offsets needed
to walk the chain from remote memory.  The common case where exc_info
points at the embedded exc_state keeps the existing zero-extra-read
fast path.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

profiling.sampling: exception mode misses except blocks running in generators and coroutines

1 participant