October 11, 2023
Just Add Coffee

About 10 years ago, I was sitting in a software engineering class, learning how to build ruby-on-rails applications to meet business needs. I learned a ton; from how to connect software applications, how to inspect source code modules, how to serve webpages, and how to use capistrano to deploy a Rails app to DigitalOcean. How freaking cool!

One day, an alumni of my university program visited that class and gave this talk; How GitHub uses GitHub to build GitHub. After that talk, I was hooked.

Why? What hooked me? I think it has to do with the idea that the procedures I write in code and the data that I store is this thing that could give rise to a collective of people and computers that can make tremendous change in the world. It's a living novel, an informed machine, and now, one that can even respond to us verbally.

Since then I've worked in small companies, one large bank undergoing their "software company transformation", taught at my alma mater, and started my own companies. I haven't worked at a FAANG company, haven't had an exit, and I don't have tens of thousands of followers anywhere ...yet 😉

So with that in mind, I want to share some thoughts and perspectives that have gotten me to where I am today, what's worked for me, what hasn't, in hopes that it might help you get to your next big thing.

Culture & People

When I think about culture, I think about the last two months.

How did those two months make me feel? How do others feel? That's culture. It's not something that you put on a webpage and call it a day. It's what happens in DM's, water-cooler chats, phone calls, and the best way to connect with people; being in person enjoying the company of others.

Culture is made up by the people you work with, and finding and hiring the right people is so hard to do.

Start hiring when it hurts, and don't rush it! Start hiring when you can't take it anymore. It's going to be tempting to hire that first candidate that really crushes the interview after you opened the position two days ago, and sometimes that does happen and they are the best person for the role. It usually doesn't work out this way. It usually takes longer, and it's worth it.

You don't need to follow the patterns and culture of your previous employer, what someone else said to do, even when that someone is you. People always change and evolve, we learn and unlearn. Understanding the trends and patterns of your team is one of the most important indicators of culture health.

Technical Tradeoffs

I think less is more and constraints are the real golden handcuffs.

I love this part of the Zen of Python

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.

I think simplicity comes from de-scoping. It's not about extending deadlines or pushing people to the limits. It's about the good ol' fashioned delete button.

Getting to the next step is the name of the game, building something that is simple, explicit, and beautiful is all you can ask for. Remember that good enough is fine, never ship crap, and that weeks of work can be done in a decision – by just saying no.

Every "yes" is expensive. Every yes is a promise to your customer, your employee, your shareholders. Your understanding of your minimum viable segment will help you make your yes-es more lightweight by ensuring reproducibility. Adding on weeks of work for one yes is rarely worth it.

Let's get into some technical particulars. You can skip this section if you like or if you aren't technical.

I try not to use Infrastructure as Code tools anymore. I think they were super streamlined and simple back in 2016-2017, but today I find that they add more overhead for some teams than necessary. For larger teams with specific infra and DevSecOps-esque teams, using terraform and those kinds of tools might make more sense, but for me, it's not a part of my daily workflow. I'd rather write some python, javascript, or go that provisions infrastructure or have the setup be so automated, repeatable, simple that clicking a button actually can get me and my team to that next big step.

I'm mostly writing backend services with FastAPI, so I built snok to help me automate 80% of the boring stuff so I can focus on the tough 20%. Kubernetes is still my go-to orchestrator, it is the de-facto standard for B2B SaaS at the time of writing this post so I think it's important for all devs to know a little about it.

If I have to build a frontend on top of my FastAPI service, or a standalone frontend app, I'll opt for Svelte Kit, SolidJS, or Astro and push that up to Vercel.

If I'm writing something full-stack, I'll either opt for Ruby on Rails and deploy to Render, or go with snok which leverages HTMX for the tough parts of building frontends (hypermedia is awesome!).

Right now, I'm totally geeking out about Modal. You get GPUs, queuing, key-value stores, NFS access, and more, with a fantastic python developer experience, for an incredibly low price. Snok deploys natively to Modal.

In terms of databases, I've run PostgreSQL, Minio, and MySQL on Kubernetes, but I'd recommend a serverless option for most teams today. Planetscale is a no brainer now that they will have embeddings support, and Neon is another great choice for teams that prefer Postgres.

I'm learning all about Zig, Rust, and Nix now. I think these technologies could be huge parts of the future of software. We're already seeing some of that with Rust.

And in terms of cloud providers, there are plenty of options. Consider a few things;

  1. Your team's current workspace stack.
  2. Your team's experience level with tools.
  3. Developer experience.

Most teams don't need to get in the guts of AWS IAM, most teams need to be able to ship a docker container to the cloud, run a frontend, store some data, and run queries on it. You can get so much of that with GCP, and if you're already using Google Workspace, you have in-built IAM. It's really great. AWS and Azure are perfectly fine too, but require a bit more automation and attention. You need to do AWS the right way from the start or it will cause lots of pain later on which isn't great for culture.

If you're using Kubernetes, definitely go with GKE on GCP. Additionally, consider the following tradeoffs with deploying to Kubernetes, helm is great and you're leveraging a DSL (Domain Specific Language) to do so. So there's logic in your config. If your team does not have experience managing helm charts, I would suggest going with skaffold and kustomize instead. It's more declarative, easy to abstract with simple directory structures, and can make B2B deployments really simple when things change up really quickly; much easier to just make a new directory and override with a config than add more logic into a chart that could impact all kinds of deployments at once.

I also can't forget Hetzner and DigitalOcean. I learned the value of baremetal and simple VMs early on and these providers are great choices if you need a full VM or baremetal machine for a great price.

Also, don't write your own observability stack unless you have to. Sentry, Logz, and GCP's monitoring stack do incredible jobs. GCP's mobile app is awesome too! If you have to roll your own, I'd suggest using OpenTelemetry, and buckling your seatbelts.

Oh and who's not using GitHub Actions for CI these days? Argo Workflows is a great DIY alternative, but really, it's hard to find something super close to what GH Actions provides. Actuated might be something to pay attention to.

To wrap up this section, one book I recommend for all technical folk working on and around software is Designing Data-Intensive Applications. It's probably the most important technical book I've read to date, my goal is to read it about once every 18 months or so.

Design Matters

Focusing on design is something I'm doing more and more now. Not just the UI/UX of things, but the whole system, the way I think about doing research and getting feedback.

I've found that sprinting isn't the greatest way to design something, shaping is far better. Shaping consists of setting an appetite for how much time and energy my team and I want to spend on something we're working on. There's a set start and end, with de-scoping as our only adjustment tool.

I'm taking lots of notes from Shape Up. Give yourself and your team the time and direction to build something great, and be human about it, incorporating rest is important.

One of my favorite design perspectives comes from Virgil Abloh;

Designing the room the candle sits in makes more of a difference than designing the candle.

If some of these points really resonated with you, I'd love to hear from you. Please feel free to reach out via any of the links below.