I write about building data platforms, AI systems, and cloud infrastructure. No hype. No theory far away from practice. Just what works in production and what doesn't.
My perspective is drawn from the trenches. I've worked on the infrastructure for the world's largest online retailer, delivered the data warehouse for League of Legends serving 100M+ users, and migrated a 5PB data warehouse to the cloud at Amadeus. I teach Databases & Data Management at Frankfurt School of Finance & Management. Read more at vectorian.be.
Writing
Processing CloudFront Logs with Rust
Turning gzipped AWS CloudFront access logs into Parquet with Rust and Polars: finding files with a glob, parsing the log lines and building a DataFrame.
Running .NET 8 web applications in Docker
Building a .NET 8 web API on Debian and running it in Docker: Debian and Alpine images compared (221 MB vs 111 MB), plus a quick load test.
Running Bun as AWS Lambda function
Running Bun on AWS Lambda: building and publishing the Bun Lambda layer, creating the execution role and the function, and testing the deployed handler.
How fast DuckDB can query Parquet files?
Querying daily Parquet files of web logs with DuckDB on an M1 Mac: a top-pages aggregation in 0.07 s and a full ZSTD Parquet rewrite in 4.9 s.
Using LLaMA with M1 Mac
Running Meta's LLaMA 7B on an M1 Mac using only the CPU: compiling llama.cpp, converting and quantizing the model with Python 3.11, and generating text.
Using Python 3.11 with AWS Lambda
AWS Lambda's Python runtime stopped at 3.9, so this runs Python 3.11 from a container image: a base image, an app image and deployment with Terraform.
Beyond the Borrow Checker
Why Rust suits ETL jobs: static builds, Cargo dependency management, serde and async, with a side-by-side Python and Rust version of the same job.
Misusing Ninja
Using the Ninja build system beyond compiling code: rules, builds and dependencies explained by example, then used to drive Docker builds and cloud deploys.
New tools
Command line tools worth switching to, most of them written in Rust: Starship, Alacritty, Helix, Ruff, ripgrep, exa, Zola and duf, and what each does well.
Using LDAP in Docker with caching
LDAP logins inside Ubuntu Docker containers with PAM and NSS, services supervised by S6, and unscd caching lookups to spare the directory server.