Java 11 vs 8 performance for AWS Lambda

We are running a lot of serverless applications in AWS and I was really happy that AWS finally introduce Java 11 support for AWS Lambda.

I guess many teams already switched to Java 11 and think that it brings some performance improvement. AWS Lambda Java 11 was announced in November 2019. Lambda functions are written in Java 11 run on Amazon Linux 2, the latest generation of Amazon Linux, and Amazon Corretto 11. Sounds like it should be faster than old Java 8 stuff.

Unfortunately, there is no performance comparison between Java 8 and 11 runtimes for AWS Lambda.

Let's compare AWS Lamda's performance for Java 8 and 11 runtimes.

What our Lambda will do?

API Gateway+AWS Lambda+DynamoDB

Test flow

This is the most popular architecture use case for AWS Lambda.

Let's write the code.

All code you can see as usual in my GitHub (https://github.com/Aleksandr-Filichkin/blog-aws-lambda-java-8-vs-11)

So Handler class implement RequestHandler and overrides one public handleRequest.

You can ask why we have this static stuff :)

This is not a mistake, this is the AWS Lambda killer feature. Load classes, warm-up all clients on startup. Why? Because of AWS provides all available CPU(CPU Burst) during Lambda initialization(first lambda instance invocation). It reduces initialization significantly!

I highly recommend you to watch this excellent AWS Lambda Java tuning video

I improved the code as much as possible.

1. Use CPU burst on Lambda startup
2. Get rid of IoC frameworks such as Spring/Guice/Dagger 1
3. Reduce jar file and load as fewer classes as possible. (Jar size is 6Mb, quite small for Java)
4. Use AWS SDK 2.0
5. Explicitly set AWS Region and CredentialProvider for AWS clients

So now we just build two jars for Java 11 and 8.

Deployment

The next step is deployment. I created two independent AWS Lambda functions with Java 11 and Java 8 runtimes and uploaded jars accordingly.

Also, I configurated two AWS API Gateways and pointed them to our Lambda functions.

Then I created a DynamoDB table with on-demand provisioning. In this case, DynamoDB was not a bottleneck.

I enabled X-Ray for detailed Lambda tracing.

Test Scenario

I used JMeter to send POST requests (create new User)to AWS API Gateway. I sent 1 request in a big loop to check the performance of a single AWS Lambda instance.

To measure the performance I used API Gateway Latency in CloudWatch. This is the fairest metric because it shows real user latency and depends on the user's network connection.

Metrics:

1. Max latency (per min)
2. Average latency (per min)
3. 90 Percentile latency (per min)

256 MB

256 MB Lambda

512MB

512 MB Lambda

Results warmed up Lambda

As expect we have a big latency for the first invocation and very small latency for the next invocations.

When Lamba is warmed up we see almost the same result for Java 8 and 11.

The average latency for 256 MB (warm-up)

The average latency for 512 MB (warm-up)

Results for cold start

We have a significant difference between Java 8 and 11 for the cold start.

Java 8 beats Java 11 during cold start. Latency difference is huge for 256 MB — more than 2 s

To prove that is not temporaly issue I have been testing cold start latency for a week. The same result I see every day

Cold start latency daily

AWS X-RAY trace

Let’s look deeper and check what is going with AWS Lambda during cold start for Java 8 and 11. AWS X-Ray will help with it.

Java 8

Java 11 256MB X-RAY trace

Java 8

Java 8 256Mb X-Ray trace

Conclusion

New Doesn’t Mean Better :)