[agent] Filed by the scheduled architecture audit routine (ecosystems and formats). Register: discussion #560 register.
Kind: refactor (mechanical move). Source: review §2.1 and Part 6.4 ("layering inversion"), register E39 (PyPI-name half).
Problem
On 9c43dfc, PEP 503 name canonicalization lives in a crawler: crawlers/python_crawler.rs#L43-L65 defines pub(crate) fn canonicalize_pypi_name. 29 files outside crawlers/ import it from there:
- 12 in
vendor/, including lock_inventory ×3;
- 7 in
patch/redirect/ and redirect/upstream/;
- 6 in
utils/, including utils::purl;
- 3 in
vex/discover/;
- 1 in
vendor/test_support.
Every one of those is an edge into crawlers, which should sit at the bottom of the dependency order beside them, not under them. utils::purl → crawlers is the clearest case.
The leading PEP 508 project-name scan ("take [A-Za-z0-9._-]* from the start") is written four times, and two rules have drifted on whether the name must start alphanumeric:
| Site |
Requires an alphanumeric first char |
vendor/common.rs#L592-L601 pep508_name (the shared one, used ~25 times) |
no |
vendor/pypi_lock.rs#L420-L428 value_identity (inline) |
no |
vendor/pypi_requirements.rs#L1058-L1067 (inline) |
yes |
vex/discover/pypi_other.rs#L308-L316 pep508_direct_reference (inline) |
yes |
vex/discover has no way to reach vendor::common::pep508_name without another upward edge, which is why it carries its own copy.
Symptoms and impact
I know of no user-visible bug. The cost is layering: crawlers can't be moved behind a locator interface (E36 / #855) while 29 modules reach into it for a string function. A future change to name rules, such as the PEP 685 extras normalization, also has four places to land.
Proposed change
- Add
core/src/formats/pypi_name.rs (or utils/pypi_name.rs; formats/ has no PyPI model yet) holding canonicalize_pypi_name and pep508_name. Move both bodies unchanged.
- Rewrite the 29
use crate::crawlers::python_crawler::canonicalize_pypi_name; lines and the vendor::common::pep508_name callers to the new path. Keep no re-export in crawlers.
- Replace the three inline scans (
value_identity, pypi_requirements, pep508_direct_reference) with pep508_name. Keep the "must start alphanumeric" check as an explicit caller-side guard where it exists today, so this PR changes no behavior.
Size and scope
- About 35 changed import lines, about 30 moved lines and about 20 deleted inline lines. Production diff is under 120 lines, with no behavior change.
- Out of scope:
Acceptance criteria
Dependencies
None. It unblocks the Ecosystem move (E39) and the locator split (#855, E36).
[agent] Filed by the scheduled architecture audit routine (ecosystems and formats). Register: discussion #560 register.
Kind: refactor (mechanical move). Source: review §2.1 and Part 6.4 ("layering inversion"), register E39 (PyPI-name half).
Problem
On
9c43dfc, PEP 503 name canonicalization lives in a crawler:crawlers/python_crawler.rs#L43-L65definespub(crate) fn canonicalize_pypi_name. 29 files outsidecrawlers/import it from there:vendor/, includinglock_inventory×3;patch/redirect/andredirect/upstream/;utils/, includingutils::purl;vex/discover/;vendor/test_support.Every one of those is an edge into
crawlers, which should sit at the bottom of the dependency order beside them, not under them.utils::purl→crawlersis the clearest case.The leading PEP 508 project-name scan ("take
[A-Za-z0-9._-]*from the start") is written four times, and two rules have drifted on whether the name must start alphanumeric:vendor/common.rs#L592-L601pep508_name(the shared one, used ~25 times)vendor/pypi_lock.rs#L420-L428value_identity(inline)vendor/pypi_requirements.rs#L1058-L1067(inline)vex/discover/pypi_other.rs#L308-L316pep508_direct_reference(inline)vex/discoverhas no way to reachvendor::common::pep508_namewithout another upward edge, which is why it carries its own copy.Symptoms and impact
I know of no user-visible bug. The cost is layering:
crawlerscan't be moved behind a locator interface (E36 / #855) while 29 modules reach into it for a string function. A future change to name rules, such as the PEP 685 extras normalization, also has four places to land.Proposed change
core/src/formats/pypi_name.rs(orutils/pypi_name.rs;formats/has no PyPI model yet) holdingcanonicalize_pypi_nameandpep508_name. Move both bodies unchanged.use crate::crawlers::python_crawler::canonicalize_pypi_name;lines and thevendor::common::pep508_namecallers to the new path. Keep no re-export incrawlers.value_identity,pypi_requirements,pep508_direct_reference) withpep508_name. Keep the "must start alphanumeric" check as an explicit caller-side guard where it exists today, so this PR changes no behavior.Size and scope
Ecosystemout ofcrawlers/types.rs(the other half of E39; it overlaps --ecosystems rejectsNPMandnpm, pypi, which socket.yml patches.ecosystems accepts: the ecosystem name parser is written three times #773 and Tracking: build and classify purls through one validated utils::purl API #748);Acceptance criteria
grep -rn "crawlers::python_crawler::canonicalize_pypi_name" cratesreturns nothing outsidecrawlers/.canonicalize_pypi_nameandpep508_namein core, and no inline[-_.]name scan left in the three files above.cargo test -p socket-patch-core --libandcargo clippy --workspace --all-features -- -D warningsstay green.Dependencies
None. It unblocks the
Ecosystemmove (E39) and the locator split (#855, E36).