May 28, 2020
Fullstack Kubernetes

I've often wondered what it would be like if all software applications (databases, APIs, UI clients, pubsub, secret managers, etc) ran on the same infrastructure, so I'd never have to worry about working with different infrastructure patterns e.g. CDNs, Kubernetes, VMs, Functions, the one off PaaS or BaaS everyone forgot about, load balanced, multi-region, auto-scaling groups, etc!

I wanted; a familiar frontend, a lightweight and unopinionated API layer that plays nice with top data science software and can also be made available as a standalone service, a data layer with a natively distributed design, high fidelity pubsub (ideally with some delivery guarantee), and a secrets manager – all to run together on a kubernetes cluster.

So, I took a couple hours and built this,

It's got a react UI (using react-admin), a FastAPI API layer, Apache Pulsar for Pub Sub, Cockroach DB for a SQL familiar data layer, and Vault for secrets management.

Network latencies could cause a layer build to timeout but no worries, just rerun the ./stackup script again to fill in the blanks.

Note that this project is using GKE but it should run on any similarly configured kubernetes cluster. If it doesn't, let me know!

15 years ago, a single developer could not do something like this with such low overhead. These kinds of things keep me looking forward to what the software and cloud computing industry will do in the future.

You could use this stack for plenty of cool things; a starting point for building a distributed API service, web app, messaging service, ML platform, core software platform for distributed hardware product ops, you name it – but this may be limited to certain hardware as it takes a decent amount of memory and cpu to run pulsar. But that's easy enough to remove, as the code tree is topically grouped and we're using a contextmanager to interface with pulsar.

If you found this interesting and useful, give a star and if you'd like to use something like this in production, let me know; a twitter dm works just fine. Cheers 🍻⎈