Skip to content

fix: preserve Markdown hard line breaks - #2559

Open
Maciej Zmitrukiewicz (MaciejZet) wants to merge 1 commit into
microsoft:mainfrom
MaciejZet:fix/preserve-markdown-hard-breaks
Open

Maciej Zmitrukiewicz (MaciejZet) wants to merge 1 commit into
microsoft:mainfrom
MaciejZet:fix/preserve-markdown-hard-breaks

Conversation

@MaciejZet

Copy link
Copy Markdown

Summary

  • MarkItDown._convert was stripping every trailing space, so a CommonMark hard break (two or more spaces before a newline) became a soft break.
  • Lines that end with at least two spaces are now kept. A single trailing space, a trailing tab, and a whitespace-only line are still removed. Runs of blank lines are still collapsed, so this does not overlap the fenced-code blank-line work in fix: keep blank lines inside fenced code blocks #2446 or the PPTX soft-break work in fix(pptx): convert soft line breaks instead of emitting a vertical tab #2513.
  • HTML <br> and DOCX <w:br/> are emitted by markdownify as those two spaces. Existing expectations for that output now keep the hard break.

Test plan

  • tests/test_markdown_whitespace.py: hard breaks, CRLF, backslash breaks, tabs, blank-line collapsing, fenced code, real .md input, and HTML <br>
  • Updated HTML underline, DOCX break, and strikethrough expectations
  • pytest tests in packages/markitdown on Python 3.12 with the all extra: 1024 passed, 14 skipped

Normalization stripped every trailing space, so a CommonMark hard break became a soft break. HTML and DOCX line breaks are emitted as those two spaces.
@MaciejZet

Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant