Skip to content

fix: break over-long words in one pass instead of re-queuing the tail - #26

Open
BrianWillows wants to merge 1 commit into
tecfu:masterfrom
BrianWillows:fix/quadratic-long-word
Open

BrianWillows wants to merge 1 commit into
tecfu:masterfrom
BrianWillows:fix/quadratic-long-word

Conversation

@BrianWillows

Copy link
Copy Markdown

Summary

When a word is longer than the wrap width, the loop splits off one line-sized piece
and pushes the remainder back onto the queue:

case lineLength < wordLength: {
  const splitIndex = breakword(word, lineLength)
  const splitWord = [...word]
  words.unshift(splitWord.slice(0, splitIndex + 1).join(''))
  words.splice(1, 0, splitWord.slice(splitIndex + 1).join(''))
  break
}

Every pass then spreads the whole remainder ([...word]), measures it
(wcwidth(word) at the top of the loop), and slices/joins it again — so emitting
~width characters costs O(remaining). For a word of n characters that is
O(n²).

A single long token in otherwise ordinary text is enough to trigger it — a URL, a
hash, a base64 blob:

chars size before after
4,000 4 KB 16 ms 2 ms
8,000 8 KB 43 ms 2 ms
32,000 31 KB 1,267 ms 5 ms
64,000 63 KB 5,958 ms 20 ms
128,000 125 KB 24,637 ms 14 ms

Narrower widths mean more iterations, so an ordinary 80-column terminal is in the
bad part of the curve: a 60 KB URL embedded in normal prose took ~1.4 s at
width: 80.

Fix

Break the over-long word into all of its chunks in a single pass, rather than
re-queuing the tail and re-scanning it each time. The chunk boundaries are computed
the same way (accumulate wcwidth until the next character would exceed
lineLength), so the output is unchanged.

words = chunks.concat(words) rather than unshift(...chunks) — a long word can
produce more chunks than the argument limit allows to be spread.

This removes the last use of breakword, so the now-unused import is dropped. Happy
to leave the dependency in package.json or remove it there too, whichever you
prefer.

Verification

  • Existing test suite passes: 25/25.
  • Output is byte-for-byte identical to the current implementation across 2,405
    generated inputs — CJK wide characters, tabs, mixed alphabets, widths 1/5/20/80,
    and explicit long-word cases ('a'.repeat(500) at width 20, '日'.repeat(60) at
    width 8, single characters wider than the line, etc.).
  • The single-character-too-wide path still falls through to the existing
    config.errorChar handling.
  • npm run lint output is unchanged apart from being one line shorter (the
    pre-existing no-fallthrough error at the deliberate fall-through remains).

Notes

Found and fixed with AI assistance (Claude). Happy to add a regression test that
asserts a long word wraps in bounded time if you'd like one.

When a word is longer than the wrap width, the loop split off one
line-sized piece and pushed the remainder back onto the queue. Each pass
then spread the whole remainder into an array ([...word]), measured it
with wcwidth(word), and sliced/joined it again, so a word of n characters
cost O(n) work per ~width characters emitted - quadratic overall.

A single long token (a URL, hash or base64 blob) in otherwise ordinary
text is enough to trigger it:

  chars    size    before      after
   4,000    4KB      16ms       2ms
  32,000   31KB   1,267ms       5ms
 128,000  125KB  24,637ms      14ms

Narrower widths are worse, so an 80-column terminal sits in the bad part
of the curve: a 60KB URL at width 80 took ~1.4s.

Break the word into all its chunks in one pass instead. Output is
unchanged: verified byte-for-byte identical against 2,405 inputs covering
CJK wide characters, tabs, width-1 through width-80, and long words, plus
the existing 25 tests.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@BrianWillows

Copy link
Copy Markdown
Author

Nudge on this one - still merges cleanly against master as of today, CI green.

It replaces the re-queue-the-tail loop with a single pass, so wrapping one long token stops being quadratic. Existing tests pass unchanged.

Small enough to read in a minute; glad to adjust the style to match the rest of the file if it's off.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants