DuckDB – Data power tools for your laptop, now in Clojure (2023)

Duckdb CLI is a powerhouse, it can load files as diverse as gzipped json lines, so you can stuff compressed logs straight into a directory yet still easily query them with SQL when you need to.

I’m a big fan of tmducken. We use it heavily in our prod systems. That said, we’ve recently started exploring ducktape [1] in our new projects and have been really impressed with the performance. It also support more complex types on insertions and queries which has been helpful for us. Not affiliated with the project, but just wanted to show it some love since it’s a bit newer. It was created by an active contributor to tmducken.

[1] https://github.com/dynamic-alpha/ducktape

At Cronitor we use ClickHouse, but we're leaving it behind for our next product and building directly on Parquet and DuckDB.

We think the future of observability in the AI age is self-hosted directly on NVMe backed by cheap and limitless object storage. I don't want to send customer conversations and agent thoughts to a giant multi-tenant borg SaaS database like Sentry or BetterStack.