Skip to content

Move canonicalize_pypi_name and the PEP 508 name scanner out of crawlers and vendor into one PyPI name module #883

Description

[agent] Filed by the scheduled architecture audit routine (ecosystems and formats). Register: discussion #560 register.

Kind: refactor (mechanical move). Source: review §2.1 and Part 6.4 ("layering inversion"), register E39 (PyPI-name half).

Problem

On 9c43dfc, PEP 503 name canonicalization lives in a crawler: crawlers/python_crawler.rs#L43-L65 defines pub(crate) fn canonicalize_pypi_name. 29 files outside crawlers/ import it from there:

  • 12 in vendor/, including lock_inventory ×3;
  • 7 in patch/redirect/ and redirect/upstream/;
  • 6 in utils/, including utils::purl;
  • 3 in vex/discover/;
  • 1 in vendor/test_support.

Every one of those is an edge into crawlers, which should sit at the bottom of the dependency order beside them, not under them. utils::purl → crawlers is the clearest case.

The leading PEP 508 project-name scan ("take [A-Za-z0-9._-]* from the start") is written four times, and two rules have drifted on whether the name must start alphanumeric:

Site Requires an alphanumeric first char
vendor/common.rs#L592-L601 pep508_name (the shared one, used ~25 times) no
vendor/pypi_lock.rs#L420-L428 value_identity (inline) no
vendor/pypi_requirements.rs#L1058-L1067 (inline) yes
vex/discover/pypi_other.rs#L308-L316 pep508_direct_reference (inline) yes

vex/discover has no way to reach vendor::common::pep508_name without another upward edge, which is why it carries its own copy.

Symptoms and impact

I know of no user-visible bug. The cost is layering: crawlers can't be moved behind a locator interface (E36 / #855) while 29 modules reach into it for a string function. A future change to name rules, such as the PEP 685 extras normalization, also has four places to land.

Proposed change

  1. Add core/src/formats/pypi_name.rs (or utils/pypi_name.rs; formats/ has no PyPI model yet) holding canonicalize_pypi_name and pep508_name. Move both bodies unchanged.
  2. Rewrite the 29 use crate::crawlers::python_crawler::canonicalize_pypi_name; lines and the vendor::common::pep508_name callers to the new path. Keep no re-export in crawlers.
  3. Replace the three inline scans (value_identity, pypi_requirements, pep508_direct_reference) with pep508_name. Keep the "must start alphanumeric" check as an explicit caller-side guard where it exists today, so this PR changes no behavior.

Size and scope

Acceptance criteria

  • grep -rn "crawlers::python_crawler::canonicalize_pypi_name" crates returns nothing outside crawlers/.
  • One definition each of canonicalize_pypi_name and pep508_name in core, and no inline [-_.] name scan left in the three files above.
  • Unit tests for both helpers move with them, including runs of separators, uppercase, leading whitespace and an empty name.
  • cargo test -p socket-patch-core --lib and cargo clippy --workspace --all-features -- -D warnings stay green.

Dependencies

None. It unblocks the Ecosystem move (E39) and the locator split (#855, E36).

No activity

Activity on this issue will appear here.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent:triagedarch-auditFiled by a scheduled architecture audit routine (see the architecture review discussion)pm:pippip / requirements.txtpriority:p1refactorStructural change: duplicated code or logic, missing abstraction, layering, dead code

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions