#parquet
How fast DuckDB can query Parquet files?
Querying daily Parquet files of web logs with DuckDB on an M1 Mac: a top-pages aggregation in 0.07 s and a full ZSTD Parquet rewrite in 4.9 s.
Compressing data with Parquet
Converting a 3 GB SQLite dataset of GitHub code snippets to Parquet: 427 MB with Snappy and 265 MB with Gzip, under 9% of the original size.
Compressing AWS S3 logs after getting HackerNewsed
Merging thousands of small S3 access logs into Parquet with an F# tool after a Hacker News spike, then charting referers and top posts with Plotly Dash.