Data Tools

Thirty-seven tools for data and the formats it arrives in

The survey these applications are chosen from calls this the group Rust is strongest at, and it is right: a parser and a codec are exactly what the language is for. Every tool here takes one document in and gives one out, runs as WebAssembly inside this browser tab, and makes no request while it works — which matters more here than almost anywhere, because the input to one of these is usually a database export, a network capture with an authorisation header in it, or a file of customer records somebody wants anonymised before it goes anywhere.

Documents first: a comparison that knows two documents with the same values in a different key order are the same document, a path expression, a flattener, the array-to-lines conversion, a key sorter for making two versions comparable, an escaper that counts how many rounds a string had been through, and a schema both inferred and checked against. Then tables — columns picked and renamed, duplicates found after the normalising you choose, tables merged by column *name* rather than by position, delimiters and decimal points changed together, Markdown and HTML in both directions, fixed-width files, a row-by-row comparison on a key, and an anonymiser that gives the same value the same replacement every time so that the table still joins to itself.

Then the flat configuration formats, each converted to the others with its comments carried where the target has any. Then data turned into code: TypeScript, Rust and Go types worked out from examples, a `CREATE TABLE` worked out from values, insert statements escaped the way each database wants, and a diagram drawn from a schema. Then bytes — a hex dump, a binary comparison, a Base64 inspector that says what the bytes turned out to be, gzip and deflate both ways, MessagePack and CBOR with a listing of what every byte was for, and a protocol-buffer message read without its schema. Last, the two files a machine wrote: a log split into fields and counted, and a network capture with the credentials in it pointed out.

One document

JSON diff
Compares two documents by what they mean, not by how they are written.

JSONPath evaluator
Runs a path expression over a document and shows what it found.

Flatten JSON
A nested document as one path a line, and back again.

JSON Lines converter
An array of documents as one document a line, and back.

Sort JSON keys
Puts the keys in order so that two versions can be compared.

JSON string escape
A text as a JSON string, and back — however many times it was escaped.

JSON Schema generator
Works out a schema from example documents — the more, the better it fits.

JSON Schema validator
Checks a document against a schema and names every fault with its path.

Rows and columns

CSV columns
Picks, renames, reorders and filters the columns of a table.

Remove duplicate rows
Finds rows that are the same after normalising, and keeps the one you choose.

Merge and split tables
Joins tables by column name, or cuts one into parts that each keep the header.

Change CSV delimiter
Rewrites a file with another delimiter, quotation marks and decimal point.

Table to Markdown
A table as Markdown, HTML or lined-up text, with the columns aligned.

Markdown table editor
Adds, drops, moves or sorts a column and gives the table back with the pipes lined up.

HTML table to CSV
Pulls every table out of a page, merged cells filled in.

Fixed-width parser
Splits lines whose columns are positions rather than delimiters.

Table diff
Compares two tables on a key and shows what changed, cell by cell.

Anonymise a table
Replaces names, addresses and numbers, the same value the same way every time.

Configuration files

Dotenv converter
An environment file as JSON, YAML, TOML or the other flat formats.

Java properties converter
A properties file with its continued lines and Unicode escapes read properly.

INI converter
An INI file as JSON, YAML or TOML, sections kept as levels.

Data as code

JSON to TypeScript
Interfaces worked out from example documents, optional fields marked.

JSON to Rust
Structures with serialisation attributes and the renames a Rust name needs.

JSON to Go
Structures with serialisation tags and exported field names.

Data to SQL schema
A CREATE TABLE worked out from the values, with the key candidates named.

SQL insert generator
Statements from a table, values escaped the way each dialect wants.

SQL IN clause builder
A column of values as a list, quoted where it has to be and cut into batches.

Diagram from a schema
Reads CREATE TABLE statements and draws the relationships between them.

Bytes

Hex viewer
Bytes with their offsets and a text column, and what the header says.

Binary file diff
Byte for byte, with the offset and length of every differing run.

Base64 inspector
Decodes a string and says what kind of file it turned out to be.

Compress and encode
Text through gzip, zlib or raw deflate and into Base64 — and back.

MessagePack and CBOR
Binary documents read into a tree, with a listing of what every byte was for.

Protobuf decoder
A message split into field numbers and values, without its schema.

A database file

SQLite viewer
The header, the schema and the rows of a SQLite database, read here rather than uploaded. Not a query engine.

What a machine wrote

Log file parser
Splits lines into fields, filters them, and counts what happened most.

Network log analyser
Requests by size, time and status, with the credentials in it pointed out.

Questions

Which tools from the group are missing?: Two. A SQLite viewer, because the input here is a text area and a database file never becomes text — and because browsing one usefully means running queries, which means a query engine, which is a bigger thing than this whole application. And a Markdown table *editor*, because editing in a grid is not what this framework does; the conversion and formatting half of that job is here under the table tools.
Why is a comparison structural rather than textual?: Because a textual comparison of two JSON documents reports every line as changed when somebody's formatter reordered the keys, and reports a change when `1` was rewritten as `1.0`. Neither is a change to anybody reading the file. What is listed here is only what a reader would call a difference — which is the entire reason to open the tool.
Can it read every YAML file?: No, and it says which part it reads: block mappings and sequences, flow collections, quoted and plain scalars, comments and the block scalars. Anchors, aliases and tags it **refuses by name** rather than dropping — a converter that silently lost an anchor would produce a file that looks right and means something else, which is worse than refusing. The same rule holds for the path expressions and for JSON Schema.
Why does the compressor say it is simple?: Because it is: it writes stored and fixed Huffman blocks with a plain longest-match search, which every reader accepts and which a library would beat by a few per cent. Decompression is the whole of the standard, including the dynamic Huffman blocks this one never writes, because a decompressor that only read its own output would be useless on a real file. Brotli is named and refused: its static dictionary alone is larger than most of the applications in this workspace.

This tool failed to start