I saved 87% on compute costs by switching languages

Since 2023 has been deemed the “year of efficiency” for software companies, I wanted to share some extreme cost-efficiencies that came from something seemingly as innocuous as language choice. My team, by actually porting a service over and measuring its resource consumption, found that moving from Ruby to Rust can save 75% to 90% on compute costs, and that these savings generally apply when moving from an interpreted language to a compiled one.

How it started

Our findings were a bit serendipitous, as our original goal hadn’t been to save money. In fact, our original goal was to make a service perform better and alert less often. That service was my team’s “gateway” service, which is a service that coordinates calls to multiple other backend services to provide a simplified API to frontend clients. We were moving it from Ruby on Rails to Rust.

We wanted to do the migration to Rust for performance and safety reasons. For one, we wanted to reliably handle more requests concurrently, and without issues, by using Rust’s fast, safe concurrency model. Since our gateway is mostly IO-bound waiting on responses from its backing services, we figured this change would allow us to handle more requests more quickly, possibly with fewer servers. Further, being in Rust gave us a lot of options, since Rust boasts 5 of the top 10 fastest web frameworks according to techempower.com. We had already used the framework actix-web on other projects, so we knew we could get this service ported over quickly. In so doing, we replaced Rails, which ranks at 131 on the list, with actix-web which ranks number 5.

On top of safe concurrency, we wanted static typing on all of the data going in and out of this gateway. This made the code easier to read—we no longer had to look at the source code of the backing services to figure out which fields were coming in to our gateway!—and it made validations much simpler since the type of each field was declared explicitly.

The unexpected cost benefits

While we knew Rust was going to reduce our CPU and memory footprints, we were blown away by how much they were reduced. Based off of actual metrics (recorded by AWS and reported in Datadog), we saw a 94% decrease in CPU utilization, and a 97% decrease in memory utilization. We compared the data month over month, and made sure that the hardware and the load on the service was the same during the time periods we looked at. It was very clear: Rust was doing the same job our Ruby on Rails service had been doing but with a fraction of the CPU and memory.

This is an incredible finding for cost optimizations because most cloud providers charge by how much CPU and memory you provision. Let’s use AWS as an example. For their ECS (Elastic Container Service) offering, vCPUs cost $0.04048 per hour and memory costs $0.004445 per GB per hour. (This is for the US East 1 region as of July 2023). Also, ECS forces you to configure your CPU and memory in certain ways; i.e. you must always have at least twice as many GBs as you have CPUs provisioned, so if you wanted 2 vCPUs you’d have to allocate at least 4 GBs as well.

Applying both of those learnings, we could hypothetically take our ECS task with 2 vCPUs and 4 GB, and downsize that to a task with 0.25 vCPUs and 0.5 GB (that’s as small as it gets) and still serve the same traffic volume (on average). That takes the cost from $71.09 a month for that task, down to $8.89 per month to run. That’s an 87% decrease in ECS cost.

A couple caveats

Before I promise anything too magical, I want to point out a few caveats.

For one, CPU usage on the service we migrated was already pretty low, so it’s hard to say for sure that using Rust would consistently reduce CPU usage by 94%. I haven’t tested it with a CPU-bound service, so I’m not sure whether the gains would be better or worse in that case.

Second, you probably don’t want to drop your CPU and memory as low as it can go. While most modern services do have some sort of autoscaling in place, it can take a few minutes for the policy to kick in and for a new node to get started. To prevent any dropped requests, you’d still want to over-provision CPU and memory slightly to account for traffic spikes. That means you might not get the full 87% decrease in cost.

Conclusion

Caveats being said, it’s clear that language choice can have a big monetary impact. Even with a conservative view of the numbers I shared, switching from Ruby on Rails (or Python, or other energy-hungry languages) to Rust (or Go, or C++, etc.), could yield large cost savings. Think of what your company could do with all that extra cash if it cut compute costs by 75-90%!

For reference, here’s a figure comparing some languages by their resource consumption taken from the paper linked above.