Convert NDJSON to JSON

A JSON document is one value. An NDJSON file is a sequence of them with newlines in between, and no amount of goodwill makes that a document — a parser reads the first object, arrives at the opening brace of the second, and stops with an unexpected-token error. That is not a bug in the parser, and it is the reason this page exists.

The consequence turns up in ordinary places. A JSON Schema validator refuses the file. `response.json()` in a browser throws. An `import data from "./records.json"` fails at build time. A configuration loader reports a syntax error on line 2 of a file whose line 2 is perfectly good JSON. In every case the fix is the same one line of structure: brackets around the whole thing and commas between the records.

Every line becomes one element of a top-level array, in file order, and that is true even when the source has a single line — a one-record file produces a one-element array rather than a bare object. The output is indented two spaces per level and ends with a newline.

Being able to predict the shape matters more than it sounds. It means a script consuming the result can index into it without checking what came back, and it means the length of the array is the number of non-blank lines in the source. If your source is a single-object file and you wanted the object rather than a list of one, that unwrapping is a separate step this conversion will not do for you.

Most APIs do not take a bare array. They take an object with the records under a name — `{"records": […]}`, `{"data": […]}`, `{"items": […]}` — and often with a count or a batch identifier beside it. The conversion produces the array and stops there, because inventing a key name would be guessing at a contract it cannot see.

Adding it afterwards is trivial and worth doing deliberately rather than by hand in an editor, where a missing brace at the end of a large file is easy to make and tedious to find. `jq '{records: .}' out.json` wraps it; `jq '{count: length, records: .}' out.json` wraps it with the count the endpoint probably also wants. Check the field name against the API documentation before you send anything, since a rejected batch usually reports only that the body was invalid.

This is the safe direction out of NDJSON, and it is worth saying plainly because the neighbouring targets are not. Sending the same file to CSV, TSV, XLSX or Parquet flattens every nested object into dotted columns and reconciles the records against a single column set. Here nothing of the sort happens: a record with a `user` object three levels deep arrives with that object three levels deep.

Types survive for the same reason. A number stays a number, `true` stays a boolean, `null` stays null, and a numeric string stays quoted. Nothing is inferred, because nothing needs to be — the source was already JSON and the target is JSON. Key order within each record is preserved as well, which makes the output diffable against another export of the same data.

If any line fails to parse, nothing is written. The message identifies the line: "This file could not be read as NDJSON — line 3 is not valid JSON." No partial document is produced and no record is quietly skipped.

For this target that behaviour is worth more than it is for a spreadsheet. A truncated final line — the usual cause, from a process killed mid-flush or a rotated file cut at a boundary — would otherwise produce a document that parses, validates, and is missing an unknown number of records. Deleting or repairing the named line takes one command, and you know exactly what you removed. Trailing blank lines are not a problem: they are skipped, including the final newline nearly every writer leaves behind.

The output is pretty-printed with two-space indentation, so a record that occupied one dense line in the source occupies a line per leaf value plus its braces. A file of flat records with a dozen fields each therefore grows by roughly a factor of a dozen lines, and the byte count grows by the indentation on every one of them.

None of that is data, and it compresses away to almost nothing over the wire. If the destination is a human or a diff, the indentation is the point. If the destination is an upload field with a size limit, run the result through `jq -c .` to get the compact form, or reconsider whether the intake really needed one document — many that appear to will happily take the NDJSON if asked.

Both halves of this conversion are whole-value operations. The lines are parsed into an array before anything is written, and the array is serialised in one pass, so the peak cost is the source plus the parsed structure plus the output string. Nothing streams, and nothing can — an array is not finished until its last element is.

The free ceiling is 100 MB per file and the practical ceiling is the tab. Tens of megabytes is uneventful; several hundred megabytes of records is where a browser starts to labour, and that is exactly the size at which turning a stream into a single document stops being a good idea anyway. NDJSON splits safely at any line boundary, so `split -l 200000` gives you files that each convert cleanly.

Converting the result back the other way returns what you started with. A top-level JSON array is written one element per line, so the records, their keys, their key order and their types all come back unchanged. There is no lossy step in either direction to accumulate.

That makes the pair useful as a working move rather than a migration. Convert to JSON, run the validator or the schema check, fix what it complains about in a proper JSON editor with bracket matching, convert back, and feed the corrected stream to the loader that wanted lines in the first place. The NDJSON stays the transport format and the JSON is the shape you work in.

If the consumer accepts newline-delimited input, give it that. BigQuery, ClickHouse, DuckDB, Elasticsearch bulk loads and most log shippers name NDJSON as an input format precisely because a single array forces the reader to hold everything at once. Converting to satisfy a preference rather than a requirement makes the file bigger and the load slower.

Keep the original in any case. The array has no advantage the lines lack except being one value, and the lines have several the array lacks: they append without rewriting, they filter with `jq` without being loaded whole, and a corrupted byte damages one record rather than the whole document.

The conversion is plain JavaScript in the page. No engine is downloaded, no request carries the file, there is no account and no daily quota, and the network tab during a conversion is the check rather than this sentence.

That matters here because the files that need this conversion are rarely public. An event export, an audit log, a customer extract on its way into a schema validator — all of them are exactly the material an organisation forbids pasting into a web service. There is nothing to forbid when nothing is sent.