GraalVM + AWS Lambda or solving Java cold start problem

Do you also like Serverless and Java, but hate a Java code start? In this article, we are going to demonstrate how to solve Java cold start issue in AWS Lambda.

Let’s write some book microservice and measure the performance.

As usual all code you can see in my Github: https://github.com/Aleksandr-Filichkin/java-graalvm-aws-lambda

Our test architecture

API-Gateway -> AWS Lambda->DynamoDb

Version 1 (plain Java without improvements)

• Java 11
• AWS SDK-V2 for DynamoDB(extended DynamoDb client)
• No DI (Spring, Dagger, etc)
• No special frameworks

Code is here
https://github.com/Aleksandr-Filichkin/java-graalvm-aws-lambda/tree/main/lambda-v1

Our handler is:

Result:

REPORT RequestId: e89b743e-bf08–4c8a-9783–0772102d4e90 Duration: 10845.21 ms Billed Duration: 10846 ms Memory Size: 256 MB Max Memory Used: 168 MB Init Duration: 2650.86 ms

Cold start for version 1 (256 Mb)

Version 2(plain Java with improvements)

• Java 11
• AWS SDK-V2 for Dynamodb
• No DI (Spring, etc)
• No special frameworks
• Utilize CPU burst on startup (move everything to static, warm-up dynamoDB client)
• Reduce dependencies(exclude Netty)
• Specify AWS Regions
• Specify Credential Provider

Code is here: https://github.com/Aleksandr-Filichkin/java-graalvm-aws-lambda/tree/main/lambda-v2

Handler:

Result:

REPORT RequestId: 236850d2–3b85–4b98–9a8f-15dee608e212 Duration: 4037.08 ms Billed Duration: 4038 ms Memory Size: 256 MB Max Memory Used: 170 MB Init Duration: 3604.04 ms

Billable time reduced 2.5 times!

X-Ray trace for the first call

What is AWS Lambda Custom runtime?

• Just a function.zip with a bootstrap shell script or binary executable file (compiled for Amazon Linux)
• Any programming language
• Not a new feature (since 2018)

How can we compile Java code to a binary file? Rigth, GraalVM Native

GraalVM

GraalVM JIT and AOT

AOT vs JIT: Startup Time

JIT:

• Load JVM executables
• Load classes from the file system
• Verify bytecodes
• Start interpreting
• Run static initializers
• First-tier compilation
• Gather profiling feedback
• Second tire compilation(C2 or GraalVM)
• Finally, run with the best machine code

AOT:

• Load executable with a prepared heap
• Immediately start with the best machine code

AOT vs JIT

Version 3 (AWS Lambda Custom Runtime + GraalVM)

Code is here https://github.com/Aleksandr-Filichkin/java-graalvm-aws-lambda/tree/main/lambda-v3

To build a native binary I’m using Docker:

Result:

REPORT RequestId: 982648d4–2a68–49fa-9a89-bb7c36519be9 Duration: 372.73 ms Billed Duration: 704 ms Memory Size: 256 MB Max Memory Used: 90 MB Init Duration: 330.61 ms

GraalVM X-Ray tracing for 256MB

Graal native drawbacks:

•Manual/explicit mapping for reflections

•Not all libraries can be compiled(closed-world assumption)

•Too slow(CPU intensive) build time

•Big size of binary file (for our service the jar size is 8.8MB, GraalVM binary is 56MB)

• Only Serial GC is supported for GraalVM CE version(Enterprise has G1)

Useful GraalVM tips:

•Use JVM agentlib to tracks all usages of dynamic features of an execution on a regular Java VM:
$JAVA_HOME/bin/java -agentlib:native-image-agent=config-output-dir=path

•Use Dashboard to analyze the binary file (https://www.graalvm.org/docs/tools/dashboard)

•For logger use slf4j-simple

•Use UPX if the binary file is big(AWS Lambda limit is 250 MB)

• Use Quarkus, Micronaut, etc

Summary

Cold start comparison

What about the warmed-up state?

I sent ~10.000 requests to the single instance of Lambda V2 (Java optimized) and Lambda V3(GraalVM). GraalVM has constant great performance ~7ms, for Java, we have a bad performance at the beginning and then it becomes ~ 15 ms(look like the Second-tier JIT optimization still was not applied)

Orange is GraalVM, red is Java

Conclusion

•GraalVM solves Java cold start issue and warm-up performance is great

•GraalVM requires additional explicit configuration

•Not all libraries can be compiled