All guides

TSV and CSV guides

CRLF vs LF — line endings in data files

Line endings are the one difference between two files that are otherwise byte-identical, and the one nobody looks at until a lookup fails on the last column only.

Every row in a text file ends with something, and there are three candidates for what that something is. They are invisible, they are not interchangeable, and mixing them in one file is both common and easy to do by accident.

The usual way you find out is oddly specific: everything works except the last column, where every value refuses to match.

The three endings

  • LF — one byte, 0x0A. Unix, Linux, and macOS since OS X. Written \n.
  • CRLF — two bytes, 0x0D 0x0A. Windows, and also the wire format for HTTP headers and email. Written \r\n. RFC 4180, the CSV specification, says CRLF — which surprises people who assume LF is the neutral choice.
  • CR — one byte, 0x0D, alone. Classic Mac OS, up to version 9. You will not produce one today, but old exports and some legacy systems still emit them, and a parser that only splits on \n sees such a file as a single enormous line.

Nothing in the file announces which is in use. It is inferred by looking, which is why tools disagree about it.

Why only the last column breaks

This is the symptom worth recognising, because it identifies the problem on its own.

A file is written with CRLF. Something reads it and splits rows on \n alone. Every row is separated correctly — but the \r that came before each \n was not part of the separator, so it stays attached to the end of the row. The row’s final field now ends with a carriage return that nothing draws.

So "active" in the last column is really "active\r". It fails every comparison, prints identically in every log, and survives being copied. It is an invisible character problem with a structural cause, and looking for a stray paste will not find it.

The tell is that it affects the last column and only the last column, in every row, uniformly. Whitespace pasted by a human is never that tidy.

How a file ends up mixed

A single file with more than one style is the situation that confuses tooling most, and it arrives in ordinary ways:

  • Rows appended by a different tool. The original export wrote CRLF, a script added forty rows with LF, and now the file has both.
  • An edit in the wrong editor. Some editors keep the file’s existing endings and some apply their own to lines you touched.
  • A merge of two exports from two systems.
  • A cell containing a line break. A quoted field with a newline inside it is legitimate, and a naive detector counts it as evidence of another style. It is not a mixture at all, but tools will report it as one.

Mixed endings are worth reporting and rarely worth panicking about. The repair is to choose one and rewrite consistently — but choose deliberately, because whatever consumes the file next may have an opinion.

Git, and the diff that changed everything

The most common way to lose an afternoon here has nothing to do with parsing.

Git on Windows often ships with core.autocrlf=true, which converts LF to CRLF when files land in your working directory and back to LF when you commit. For source code this is invisible and helpful. For a data file it means the bytes on disk are not the bytes in the repository.

The symptom is a diff claiming every line of a 50,000-row file changed when you edited one cell. What changed is the line endings, across the whole file, because something rewrote it with the other style.

The fix is a .gitattributes that says what the file is:

*.tsv text eol=lf
*.csv text eol=lf

Or, when the file must keep exactly the bytes it has and Git should not interpret it at all:

*.tsv -text

Set this before the file is committed rather than after. Changing it later rewrites the file again, which produces one more enormous diff.

Choosing one, and the trailing newline

For a file that stays inside a Unix or cross-platform toolchain, LF. For a file going to Windows software or to a colleague who will open it in Excel, CRLF is the safer guess and is what the CSV specification asks for.

There is a fourth decision hiding next to this one: whether the last row ends with a line ending at all. POSIX says a text file’s final line is terminated, and tools assume it — wc -l undercounts a file without one, and concatenating two such files joins the last row of the first to the header of the second.

The rule that matters more than either choice: a file you are only editing should come back with the endings it arrived with. A repair that also silently converts CRLF to LF has changed 50,000 lines to fix one cell, and has broken the file for whatever reads it next.

Doing it here

The editor on this site detects the line ending on open, counts each style separately, and reports the result in the status bar rather than assuming one. If more than one style is present it says so, and it distinguishes a genuine mixture from a line break inside a quoted cell, which is not the same thing.

For TSV and CSV, the export dialog offers LF or CRLF between records, or the imported setting (including CR), plus a final-line-break setting. A uniform input ending and the presence or absence of a final break are retained by default. Mixed record endings become the most frequent style; ties prefer CRLF, then LF, then CR. Cell line breaks remain unchanged. For example, a,b\r\nc,d\ne,f exports with CRLF at both record boundaries and no final break. Quotes and encoding may still change, so this is not a promise that editing one cell changes only one line.

Nothing is uploaded: the file is read and written by your browser.