An embeddable SQL database like SQLite, but supports Postgres features - DuckDB / github / HN

The creator of duckdb argues that people using pandas are missing out of the 50 years of progress in database research - HN - DuckDB Internals (CMU Advanced Databases / Spring 2023) - duck DB use a vectorized push-base model (implemented in C++) - support larger than memory execution - DuckDB – The SQLite for Analytics (Mark Raasveldt, CWI)

• HN is a lightweight and super fast library/CLI for working with Parquet. It’s SQLite for column formats
• The DuckDB Local UI / HN - the UI is not open source.
• Should You Ditch Spark for DuckDb or Polars? / HN
• How does DuckDB compares to SQLite (chatGPT)

see also

• A sharded DuckDB on 63 nodes runs 1T row aggregation challenge in 5 sec
• DuckLake - advanced data lake features without traditional lakehouse complexity by using Parquet files and your SQL database. It’s an open, standalone format from the DuckDB team