Convert NDJSON to YAML

Structured logging puts whole documents inside single fields. A stack trace, a rendered SQL statement, a request body, an assertion diff — each is text with newlines in it, and JSON has exactly one way to hold a newline, which is to write the two characters backslash and n. A forty-frame Python traceback therefore arrives as one line several thousand characters long with `\n` between every frame, and reading it means either widening the terminal or piping it through something.

YAML has another option and this conversion uses it. Any string containing a newline is written as a block scalar: the key, then `|-`, then the real lines indented beneath it. The trace reads as a trace. Nothing about the value changes — the same characters go in and come out — but the file stops fighting you, and for an export you are going to read rather than load, that is the entire argument.

A YAML author looking at a line-delimited source would reasonably expect a multi-document stream: each record its own document, `---` between them. That is not what is produced. The whole file becomes one document containing a sequence, with each record introduced by a dash and its fields indented under it.

The choice has a consequence worth knowing about before you build on it. This site refuses to read a multi-document YAML stream at all — that conversion stops rather than picking a document — so the single-document form is the one that round-trips. It also means the output is a single value, which is what a YAML loader in a script expects: `yaml.safe_load(open(f))` gives you a list, not a generator you have to remember to iterate.

A bare YAML scalar is resolved by the parser, so a writer has to quote anything whose bare form would come back as a different type. This one quotes the cases that matter for exported data: a string of digits, a value beginning with a hash or an asterisk, a value containing a colon and space, and anything with leading or trailing whitespace. An order reference held as the string "00412" arrives quoted and keeps its leading zeros.

That is the writer being careful, not the format being safe, and the distinction matters when somebody edits the file afterwards. Delete the quotation marks around "00412" and it becomes the number 412 the next time anything loads it. If the YAML is going to be hand-edited and then read back by a program, the quoting is load-bearing rather than decorative.

The file is written to YAML 1.2, where the only booleans are `true` and `false`. Under that rule `NO`, `yes`, `on` and `off` are ordinary strings and are correctly written unquoted. Load the same file with PyYAML, Ruby’s Psych or one of the older Go loaders and they are booleans, because those implement YAML 1.1.

For a configuration file you would notice, because you read it. For an export you would not: a country column of ISO codes has a `NO` in it somewhere among thirty thousand rows, and it becomes `False` on load with nothing to indicate it. If the YAML is destined for a 1.1 parser, either quote that column before loading or keep the JSON, which has no such ambiguity anywhere in it.

Every other popular destination for a line-delimited export flattens. CSV, TSV, XLSX, SQL and Parquet all turn a nested `request` object into columns named `request.method` and `request.path`, and reconcile the records against one column set. Nothing of that kind happens here.

A record with a `context` block four levels deep arrives four levels deep, indented. Arrays become dashed lists. Records that carry different fields keep carrying different fields — there is no union of keys, no empty cells and no sparse table. For reading an export where the interesting part is a subtree, that is precisely what you want, and it is why this target is resolved lossless while the table targets are not.

A string with no newlines in it but plenty of length — a user agent, a URL with a long query string, a one-line error message — is wrapped at around eighty columns onto continuation lines indented under the key. YAML treats a folded plain scalar’s line breaks as spaces, so the value is unchanged when it is read back.

It does mean a URL can appear split across two lines in the file, which is disconcerting the first time and is not damage. If you need to copy such a value out, take it from the loaded structure rather than from the text, or convert to JSON instead, where every value stays on the line it started on.

None. A YAML file that a person maintains earns its format from the `#` lines explaining why a value is what it is, and a converted export has no such lines because the source had nowhere to keep them. What you get is well-formed YAML with no annotation at all.

That is the right outcome for a file you are reading once and deleting. It is the wrong outcome if the YAML is about to be committed, because the next reader will find records with no provenance — no note of which system produced them, when the export was taken or what filter was applied. Adding three comment lines at the top before committing costs nothing and is the whole difference between a data file and a mystery.

Empty lines are skipped, including the trailing newline nearly every writer leaves at the end, so the number of list entries is the number of non-blank lines in the source. A line that is not valid JSON stops the conversion with its number in the message rather than being dropped.

For an export of failures that behaviour earns its keep. The last line of a log written by a process that crashed is very often half-written, and a converter that skipped it silently would hand you a file that reads as complete while missing the record you were looking for. With the line number you can decide whether the tail is worth salvaging before you draw any conclusions from the rest.

The free ceiling is 100 MB and the whole file is parsed into memory before anything is written, so the practical limit is the browser tab rather than a quota. Tens of megabytes converts without complaint.

The sensible limit is lower than the technical one, though, because the point of this target is reading. Nobody reads thirty thousand records. Filter first — `jq -c 'select(.level=="error")' events.ndjson > errors.ndjson` takes a second and leaves you with the forty records you actually meant, which is a YAML file you can scroll through and reason about. Converting the whole export produces a file that is readable in principle and unread in practice.

In this browser tab, by plain JavaScript. Nothing is uploaded, there is no engine to download first and no account, and the network tab during a conversion is the way to confirm it rather than this paragraph.

It matters more for this pair than for most. The files people convert here are error exports and incident dumps, and those carry the things that were in flight when something broke: request paths, session identifiers, occasionally a token in a query string that should not have been logged. That is the exact category nobody should paste into a web service, and here there is nothing to paste.