diff: do final refinement at byte-level for non-word bytes
This results in significantly more readable diffs on commits like
659393bec2 in this repo.
Before:
test bench_diff_10k_lines_reversed ... bench: 38,122,998 ns/iter (+/- 557,688)
test bench_diff_10k_modified_lines ... bench: 32,556,563 ns/iter (+/- 548,114)
test bench_diff_10k_unchanged_lines ... bench: 4,231 ns/iter (+/- 15)
test bench_diff_1k_lines_reversed ... bench: 958,296 ns/iter (+/- 46,963)
test bench_diff_1k_modified_lines ... bench: 3,014,723 ns/iter (+/- 15,830)
test bench_diff_1k_unchanged_lines ... bench: 249 ns/iter (+/- 2)
test bench_diff_git_git_read_tree_c ... bench: 78,599 ns/iter (+/- 1,079)
After:
test bench_diff_10k_lines_reversed ... bench: 38,289,493 ns/iter (+/- 413,712)
test bench_diff_10k_modified_lines ... bench: 37,352,516 ns/iter (+/- 1,293,950)
test bench_diff_10k_unchanged_lines ... bench: 4,238 ns/iter (+/- 13)
test bench_diff_1k_lines_reversed ... bench: 967,253 ns/iter (+/- 8,506)
test bench_diff_1k_modified_lines ... bench: 3,358,028 ns/iter (+/- 37,154)
test bench_diff_1k_unchanged_lines ... bench: 233 ns/iter (+/- 1)
test bench_diff_git_git_read_tree_c ... bench: 95,787 ns/iter (+/- 740)
So the biggest slowdown is when there are modified lines.
0dd000d236