AWS Lambda battle 2021: performance comparison for all languages (cold and warm start)

  • NodeJs (14.x)
  • Python (3.9)
  • Go(1.x)
  • Ruby(2.7)
  • .Net(3.1)
  • Java (11)
  • Rust(1.54.0)
  • GraalVM(21.2)

Disclaimer:

Test scenario

The main flow

Cold start test

  • Removed useless dependencies.
  • Move as much as possible to the initialization phase(for example, in Java move everything to static) to use CPU burst on startup.
  • Specified the Region.
  • Got rid of any DI frameworks

Result:

Cold start result
Cold start result
  • All languages(except Java and .Net) have a pretty small cold start.
  • Java even cannot start with 128Mb. It needs more memory. But GraalVM can help in this case. Feel free to read a detailed page about GraalVM and AWS Lambda
  • Rust beats all runtimes for all setups, the only exception is 128 MB where Python is the best.
  • The huge setup helps only for Java and .Net.

WARM test

Which metrics will we check?

  • The average(per minute) duration for each language (256MB setup,(128MB short result you can find at the end)
  • The maximum(per minute) duration for each language (256MB setup)

NodeJS

NodeJS 256MB average duration
NodeJS 256MB maximum duration

Python

Python 256MB average duration
Python 256MB maximum duration

Ruby

Ruby 256MB average duration
Ruby 256MB maximum duration

.NET

.Net 256MB average duration
.Net 256MB maximum duration

Golang

Golang 256MB average duration
Golang 256MB maximum duration

Java

Java 256MB average duration
Java 256MB maximum duration
Java 256 MB, no C2 optimization.

GraalVM:

GraalVM 256MB average duration
GraalVM 256MB maximum duration

Rust

Rust 256MB average duration
Rust 256MB maximum duration

All together

5K iteration for 3 timeslots(256MB Lambda)
256 MB Lambda

Also, I tested the same flow for 128MB lambda. And here we can see a big difference.

128MB average warm state
128MB average warm state
128MB maximum(per minute) warm state

Conclusion:

Cold start:

  • All languages(except Java and .Net) have a pretty small cold start.
  • Java even cannot start with 128Mb. It needs more memory. But GraalVM can help in this case.
  • Rust beats all runtimes for all setups for cold start, the only exception is 128 MB where Python is the best.

Warm start:

  • Golang and Rust are the winners. They have the same brilliant performance.
  • .Net has almost the same performance as Golang and Rust, but only after 1k iterations(after JIT).
  • GraalVM has a stable great performance almost the same as .Net and a bit worse than Rust and Golang. But it doesn’t perform well for the smallest setup.
  • Java is the next after GraalVM.The same as .Net, Java needs some time(1–3k iterations) for JIT(C1). Unfortunately for this particular use case, I was not able to achieve the expected great performance after JIT C2 compilation. Maybe AWS just disabled it.
  • Python has stable good performance but works too slow for the 128 MB
  • Ruby has almost the same performance as Python, but we see some duration growing after 20 min invocations(after 15k iteration).
  • NodeJs is the slowest runtime, after some time it becomes better(JIT?) but still is not good enough. In addition, we see the NodeJS has the worst maximum duration.

--

--

--

Java, AWS expert

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Cache and Persist in Spark Scala | Dataframe | Dataset

Static type checking in Rails with Sorbet

App Development with Swift Unit2 (1)

Spot Fleet Weighting and Pricing Explained

Basic Tips for Coding for Beginners

Building a Pipeline to load the data from Blob storage to CosmosDB using Databricks

How to Upload and Download Blobs from Azure Blob Storage Using Python

Games built with Rust

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Aleksandr Filichkin

Aleksandr Filichkin

Java, AWS expert

More from Medium

CDK Tag Inheritance with Golang

golang 1.18+ generics: The Good, The Bad, The Ugly.

Using conda to version control aws-cdk

Installing OpenJDK on M1 Macbook Pro