Skip to content

readline: add unicodeLineSeparators option to interface - #66478

Open
FrancescoCastaldi wants to merge 20 commits into
nodejs:mainfrom
FrancescoCastaldi:readline-unicode-line-separators-opt-out
Open

FrancescoCastaldi wants to merge 20 commits into
nodejs:mainfrom
FrancescoCastaldi:readline-unicode-line-separators-opt-out

Conversation

@FrancescoCastaldi

Copy link
Copy Markdown

This PR adds an opt-out option unicodeLineSeparators (defaulting to true) to readline.createInterface() and readlinePromises.createInterface().

Rationale

In Node.js 24.0.0 (via PR #57591 / issue #22448), Unicode line separator (\u2028) and paragraph separator (\u2029) were added as line terminators in readline. While this adheres to the Unicode standard for general text processing, it causes regressions when parsing JSON Lines (JSONL) data containing \u2028 or \u2029 inside valid JSON string literals. Since JSON allows raw \u2028 and \u2029 in strings, splitting on these characters breaks valid JSON records across multiple lines, causing JSON.parse() to fail with syntax errors (as reported in #60606).

Providing unicodeLineSeparators: false allows callers (such as JSONL parsers, data pipelines, and streaming serializers) to restrict line splitting exclusively to standard ASCII newlines (\r\n, \n, \r) while preserving full backward compatibility by keeping true as the default.

Key Changes

  1. Added unicodeLineSeparators option to readline.createInterface() and readlinePromises.createInterface().
  2. Validated option type with validateBoolean (ERR_INVALID_ARG_TYPE).
  3. Gated the \u2028/\u2029 search in onStreamData and _ttyWrite behind this.unicodeLineSeparators.
  4. Maintained pre-compiled module regexes for both modes (lineEndingWithUnicode and lineEndingWithoutUnicode) to preserve performance.
  5. Added unit tests covering JSONL parsing, CRLF splitting without Unicode separation, default preservation, argument validation, promises interface, async iterators, and TTY writes.
  6. Added benchmark script in benchmark/readline/readline-stream-split.js.
  7. Updated documentation in doc/api/readline.md.

Fixes: #60606

Checklist:

  • test passes
  • tests and/or benchmarks are included
  • documentation is changed or added
  • commit message follows commit guidelines

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/performance

@github-actions

github-actions Bot commented Oct 3, 2026

Copy link
Copy Markdown
Contributor

Welcome to Node.js, and thank you for your first contribution!

Before review, please take a moment to read:

Please make sure every commit is signed off. For a first pull request, GitHub Actions require collaborator approval and Jenkins CI must be started by a collaborator or triager, so an initial wait is normal.

@nodejs-github-bot nodejs-github-bot added needs-ci PRs that need a full CI run. readline Issues and PRs related to the built-in readline module. labels Oct 3, 2026
Signed-off-by: Francesco Castaldi <info@francescocastaldi.it>
Signed-off-by: Francesco Castaldi <info@francescocastaldi.it>
…isabled

Signed-off-by: Francesco Castaldi <info@francescocastaldi.it>
Signed-off-by: Francesco Castaldi <info@francescocastaldi.it>
Signed-off-by: Francesco Castaldi <info@francescocastaldi.it>
Signed-off-by: Francesco Castaldi <info@francescocastaldi.it>
Signed-off-by: Francesco Castaldi <info@francescocastaldi.it>
Signed-off-by: Francesco Castaldi <info@francescocastaldi.it>
Signed-off-by: Francesco Castaldi <info@francescocastaldi.it>
Signed-off-by: Francesco Castaldi <info@francescocastaldi.it>
Signed-off-by: Francesco Castaldi <info@francescocastaldi.it>
Signed-off-by: Francesco Castaldi <info@francescocastaldi.it>
Signed-off-by: Francesco Castaldi <info@francescocastaldi.it>
Signed-off-by: Francesco Castaldi <info@francescocastaldi.it>
Signed-off-by: Francesco Castaldi <info@francescocastaldi.it>
Signed-off-by: Francesco Castaldi <info@francescocastaldi.it>
Signed-off-by: Francesco Castaldi <info@francescocastaldi.it>
Signed-off-by: Francesco Castaldi <info@francescocastaldi.it>
Signed-off-by: Francesco Castaldi <info@francescocastaldi.it>
Fixes: nodejs#60606
Signed-off-by: Francesco Castaldi <info@francescocastaldi.it>
@FrancescoCastaldi
FrancescoCastaldi force-pushed the readline-unicode-line-separators-opt-out branch from 662a77a to d5483ab Compare October 3, 2026 10:55
@codecov

codecov Bot commented Oct 3, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.44%. Comparing base (617e082) to head (d5483ab).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #66478      +/-   ##
==========================================
+ Coverage   90.42%   90.44%   +0.01%     
==========================================
  Files         790      790              
  Lines      275435   275450      +15     
  Branches    52826    52830       +4     
==========================================
+ Hits       249070   249117      +47     
+ Misses      16765    16727      -38     
- Partials     9600     9606       +6     
Files with missing lines Coverage Δ
lib/internal/readline/interface.js 93.31% <100.00%> (+0.06%) ⬆️

... and 28 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-ci PRs that need a full CI run. readline Issues and PRs related to the built-in readline module.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Regression Node 24.0.0+: parsing JSONL with U+2028 (line separator)/ U+2029 (paragraph separator)

2 participants