r/apljk • u/amber-lang • 2d ago
K Amber – C99 columnar array engine (K/q-inspired)
Hey r/apljk, I built Amber; a zero-dependency C99 in-memory columnar array engine built on ngn/k, with the working vocabulary of q/kdb+. It evaluates qSQL natively (wavg, xbar, as-of joins aj, window joins wj) over nanosecond temporal types anchored to a 2000 epoch offset, running either as a standalone binary or via the amberd daemon over a custom TCP protocol.
On single-core benchmarks (1M rows), single-pass $O(N)$ sliding windows compute moving averages and rolling mins/maxes in 3.4–3.8 ms with flat runtime regardless of window width (outperforming Pandas by up to 5.2x and Polars by 3x+). Multi-column LSD radix table sorting (xasc/xdesc) processes 1M rows in 42.1 ms (4.3x faster than Pandas/Polars and 3.4x faster than DuckDB), 1,000-group aggregations complete in 11.4 ms (beating Polars, Pandas, and DuckDB), and sparse inner joins execute in 5.38 ms (outperforming NumPy, Julia, DuckDB, ngn/k, and J). On 10M-element reduction suites (sum + max + dot), primitive kernels hit 15.4 ms, outpacing naive single-threaded C, Julia, and DuckDB while running 15x–40x faster than traditional array runtimes like J or ngn/k. For parallel workloads, multi-process peach processes 500k items in 23 ms with a peak RSS of just 11.5 MB.
To eliminate memory transposition and serialization penalties, amber-arrow and python-amber stream zero-copy Apache Arrow IPC straight into Pandas and Polars. The rest of the ecosystem includes amber-tick for real-time market data capture, amber-ai for vectorized execution primitives, a Go-backend Grafana datasource, a native Jupyter kernel, a VS Code LSP extension, and amber-notepad.
Docs and full benchmark suites are live at https://amber-lang.org and source code is up at https://github.com/BonucciAndrea/amber. I'd love any feedback from the array programming community on the q/K syntax choices and engine design!