Go is Great, So Why Aren't We Using It?

Anybody who has spent time chatting with me at one of our Zapier retreats or listening to one of my Friday lightning talks can attest that I think Go is a great programming language. In this post I’ll share why I think it's great, some cons of using it, and why it’s not yet part of our stack at Zapier.

Go is a programming language developed at Google to help large teams of developers successfully build and maintain high performance software projects. It was open sourced in 2009 and reached Go 1.0 in 2012. Since then, many organizations, including dev-focused companies like Docker and Hashicorp, have started using the language in their projects to great success.

My First Go at Go

I wrote my first line of code in Go about four years ago. At first, I didn't "get it." The language seemed interesting, programmers were touting all of the great things about Go, but I wasn't able to conceptualize the benefits. That is, until I started porting a real project from Python to Go. With a real world need I was able to appreciate what Go offered.

The project I ported was a very small REST API that interfaced with Redis. It was the basis for a straightforward caching system that moved "cold" keys to the disk, but kept "hot" data in Redis. What I needed was a simple way to distribute this API to be collocated with the Redis servers that did not consume much memory and was very fast. Enter Go, which–spoiler alert–fits into those requirements nicely.

Why is Go Great?

There are a lot of reasons that I think Go is a great programming language. With a real world need, I was able to prototype, test, and deploy a production system in days with minimal CPU and memory overhead all while handling 5x more requests per second.
This is because Go is a simple language with a very small (but complete) standard library. I was able to grasp most of the language concepts, built-in packages, and common idioms in just a few hours.

Other than this, Go has a few other things going for it:

• Statically typed
• Compiled
• Single binary for distribution
• Garbage collected
• Concurrency and Parallelism primitives are built-in
• Can link against popular C libraries
• Great documentation
• Has a vibrant and growing community

Zapier is primarily a Python shop. While some of the above is true for Python, getting that whole list with Go is pretty appealing. And as we work with thousands of API calls per hour, performance is important to us. So, why haven’t we jumped at adopting Go?

Why is Go not so great?

No programming language is perfect; all make very explicit trade-offs. Go optimizes for compilation speed and application performance at the expense of programmer productivity.

The most controversial of these trade offs is that Go provides a statically typed system, but does not offer generics. This means container data structures outside of lists and hash tables cannot be written generically without typecasts to and from anonymous pointers (the interface{} type).

A few other downsides to programming in Go:

• Dependencies can be difficult to manage, opting for vendoring third-party code in your repository.
• The type system, while simple, is very different than a classical object orientation and inheritance model. It is very C-like. Structs + Interfaces. No classes. No inheritance.
• There are no formal exceptions or try/catch. This could can be framed as a positive, but Go often trips up developers with its panic/recover system.
• Some argue that garbage collection isn't suited for a systems programming language like Go. I for one welcome our garbage collection overlords.

Go! What is it Good For?

Go is well suited for systems programming. It’s touted as middle ground between lower-level languages like C and high-level programming languages like Python. These are some of the best types of applications built in Go:

• Command line apps
• Network Servers
• Daemons
• Databases

But, some are even using it for:

• Embedded Development
• Lightweight Microservices
• Infrastructure

Go is flexible enough to solve most problems, albeit some better than others. And the list of companies using Go is impressive, including Docker, Cloudflare, Dropbox, Intel, Hashicorp, and, of course, Google.

So… Why Isn't Zapier Using It Yet?

Zapier is a monolithic application built on Python and Django. There’s a lot of momentum with Python and it is very difficult changing course to another programming language while the service is running. It’s not impossible, though!

We're at the very (very!) early stages of a breaking apart our monolith into a microservices architecture. These services are currently implemented as needed. However, once we hit critical mass on one of these systems where a high-performance rewrite might be necessary, I could see Go being an option. We’d follow a process of:

1. Measure the current system and determine bottlenecks.
2. Spike a solution addressing bottlenecks in Go.
3. Measure the solution as a canary service in Production.
4. Engineer all the things.
5. (Repeal and) Replace

Adopting a programming language is a difficult decision and Go is not the only language we must consider. Some others on the table, with equal pros/cons:

• Elixir (on the Erlang VM)
• Rust
• Python 3 (crazy, I know!)
• PyPy
• Cython

All of this is to say: Go is a great programming language that I’ve used to build highly performant systems. While it makes trade-offs like most programming languages, most are either warranted or easy to work around. Go is easy to learn and most programmers can be highly productive with it in a matter of weeks.

Maybe one day you'll read another post about how we replaced XYZ with a custom ZYX built in Go. If you’d like to be part of the team that gets to think about potential new languages and improves our existing application, check out our open positions.