Benchmarking AWS Lambda runtimes in 2021: cold start
Since AWS Lambda is so popular and supports so many languages and custom runtimes, I was trying to find on Internet the benchmarks for all supported languages with detailed results, but I didn’t find anything relevant. Some articles are old or don’t contain all languages.
Let’s compare the performance of all supported runtimes + 2 custom runtimes (Rust and GraalVM).
Source code is here: https://github.com/Aleksandr-Filichkin/aws-lambda-runtimes-performance. It requires the minimum local setup(almost all is Dockerized)
- Java (11)
- NodeJs (14.x)
- Python (3.9)
- Go(1.x)
- Ruby(2.7)
- .Net(3.1)
- Rust(1.54.0)
- GraalVM(21.2)
Disclaimer:
All benchmarks were performed in September 2021
I’m not an expert in all these languages and I’m happy to see MR in GitHub repo with performance improvements. I’m going to support these repo and run the perfomance test every 3 months. I believe in opensource collaboration :)
Test scenario
We are going to test API-Gateway -> AWS Lambda->DynamoDb flow.
We will test only POST endpoint which will save the book into the DynamoDb table into the known AWS region(us-east-2).
Cold start
Firstly we are going to test the cold start(the first invocation on AWS Lambda instance) delay. To measure this metric we will use X-Ray tracing. From my point of view, it's the most honest result, because it includes all steps(init/invocation/overhead).
Result:
Ruby:
Python:
NodeJS:
Golang:
Java:
Java with 128 doesn’t work, it has OutOfMemoryError :)
GraalVM:
.Net:
Rust:
Summary
- All languages(except Java and .Net) have a pretty small cold start.
- Java even cannot start with 128Mb. It needs more memory. But GraalVM can help in this case. Feel free to read a detailed page about GraalVM and AWS Lambda
- Rust beats all runtimes for all setups, the only exception is 128 MB where Python is the best.
- The huge setup helps only for Java and .Net.
Next test
The warm comparison you can find in the next article.
