Benchmarking AWS Lambda runtimes in 2021: cold start

Since AWS Lambda is so popular and supports so many languages and custom runtimes, I was trying to find on Internet the benchmarks for all supported languages with detailed results, but I didn’t find anything relevant. Some articles are old or don’t contain all languages.

Let’s compare the performance of all supported runtimes + 2 custom runtimes (Rust and GraalVM).

Source code is here: https://github.com/Aleksandr-Filichkin/aws-lambda-runtimes-performance. It requires the minimum local setup(almost all is Dockerized)

  • Java (11)
  • NodeJs (14.x)
  • Python (3.9)
  • Go(1.x)
  • Ruby(2.7)
  • .Net(3.1)
  • Rust(1.54.0)
  • GraalVM(21.2)

Disclaimer:

All benchmarks were performed in September 2021

I’m not an expert in all these languages and I’m happy to see MR in GitHub repo with performance improvements. I’m going to support these repo and run the perfomance test every 3 months. I believe in opensource collaboration :)

Test scenario

We are going to test API-Gateway -> AWS Lambda->DynamoDb flow.

We will test only POST endpoint which will save the book into the DynamoDb table into the known AWS region(us-east-2).

The main flow

Cold start

Firstly we are going to test the cold start(the first invocation on AWS Lambda instance) delay. To measure this metric we will use X-Ray tracing. From my point of view, it's the most honest result, because it includes all steps(init/invocation/overhead).

Result:

Cold start result

Ruby:

256 MB, Ruby cold start

Python:

256 MB, Python cold start

NodeJS:

256 MB, NodeJS cold start

Golang:

256 MB, Golang cold start

Java:

Java with 128 doesn’t work, it has OutOfMemoryError :)

256 MB, Java cold start

GraalVM:

256 MB, GraalVM cold start

.Net:

256 MB, .Net cold start

Rust:

256 MB, Rust cold start

Summary

  • All languages(except Java and .Net) have a pretty small cold start.
  • Java even cannot start with 128Mb. It needs more memory. But GraalVM can help in this case. Feel free to read a detailed page about GraalVM and AWS Lambda
  • Rust beats all runtimes for all setups, the only exception is 128 MB where Python is the best.
  • The huge setup helps only for Java and .Net.

Next test

The warm comparison you can find in the next article.

https://filia-aleks.medium.com/aws-lambda-battle-2021-performance-comparison-for-all-languages-c1b441005fd1

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store