Three months of ddbsh: from a C++11 shell to a library, a WASM build, and a web UI

ddbsh is a SQL-like CLI for Amazon DynamoDB: you type statements terminated by ;, it decides the cheapest access pattern (GetItem vs Query vs Scan), calls the API, and prints the result. The original was a single C++11 program built on flex/bison, the AWS SDK for C++, and GNU readline. Over the last three months it grew into something larger without losing a byte of its observable behavior. This post is the map; the rest of the series drills into each change (over the next several days).

The arc

The work moved along a version trail from the v0.9.x series through v0.10.x to v0.11.18. Three forces drove it:

  1. Fidelity first. The original’s golden test suite — the .exp files that pin exact stdout for each command, including the serialized EXPLAINJSON — is the regression contract. Every refactor had to keep the golden suite green (currently 710/710 families against real DynamoDB), so the shell behaves identically to the original for every standard command.
  2. Library first. The engine (lexer → parser → command model → expression engine → SQL→API planner → execution → output/EXPLAIN formatting) became a consumable C++23 library with a clean, SDK-free, terminal-free public API. The ddbsh command is now a thin wrapper around it.
  3. Cross-platform and reach. The same engine compiles to WebAssembly and runs in a browser, fronted by a small signing proxy, with a web UI on top.

What shipped

  • Composite (multi-attribute) GSI keys — a GSI key role can name up to four partition and four sort attributes, with the Query-vs-Scan planner extended to reason about partial composite keys.
  • Vector search — vector indexes, SELECT ... NEAREST <n> TO [...] mapping to the SearchVectors API, and vector values written as numeric lists.
  • The library refactor — the singleton CDDBSh became an explicit Session; the error model became std::expected<T, Error>; the AWS SDK was isolated behind an injected client interface.
  • An embedding API — an ODBC / Sybase DB-Library-style surface (open → prepare → bind → execute → fetch-row → read-column), plus an in-memory backend for offline tests.
  • The WebAssembly port — the engine compiled with Emscripten, a ddbsh-wire JSON codec, and a ddbsh-web-proxy that SigV4-signs and forwards.
  • The web UI — a browser front end with history, file upload, and a result pane, served from a single webroot.

The tenets

The project runs on eight tenets. The main ones: fidelity is the default and divergences are deliberate and documented; the engine is a library with no terminal dependency; errors and output are contracts reproduced exactly; tests are the spec; and extensions are opt-in and never change an existing command.

Everything that follows is additive. If you only ever typed select * from t where id = "x"; into the original shell, it still does exactly what it always did — now it just happens to be a thin client over a reusable engine.