Aws-sdk-java-v2 performance testing

AWS has just released a new Java SDK (https://aws.amazon.com/blogs/developer/aws-sdk-for-java-2-x-released/) and I would like to review the main feature that was introduced in the new SDK.

Of course, the main feature is nonblocking clients.

We have been waiting for release version for more than a year!!! As you can see on GitHub the first release candidate version was introduced on 7 Jul 2017.

Why do we need the new SDK?

We have been using Spring WebFlux with the old AWS SDK for several months in production environment. It works pretty well but it’s not a true nonblocking application because AWS services didn’t have a nonblocking SDK.

Old SDK version already has asynchronous variants of service clients, but they are a managed thread pool on top of synchronous clients, so each request still requires its own thread.

But the new SDK is built on Netty to support true nonblocking I/O.

In this article, we will compare how old and new SDKs work with a reactive microservice based on Spring WebFlux.

What will we compare?

· Real performance RPS under different load. We will use JMeter for performance test

· JVM thread count. We will use JVisualVM for JVM monitoring

What AWS service will we use for testing?

I would like to start with the most useful, powerful and cheap service SNS https://aws.amazon.com/sns/.

So we are actively using SNS for inter microservice communication (together with SQS).

Java source code

All the code you can see in my GitHub

https://github.com/Aleksandr-Filichkin/java-aws-sdk-comparison

The app is straightforward. We have one REST Controller (Controller.java) that exposes two REST endpoints:

· publish a message to SNS using old AWS SDK

· publish a message to SNS using new AWS SDK

We don’t have any Classloading issues, because AWS says that two versions of SDK can be used together!

So we have one interface:

public interface SNSService {

    CompletableFuture<Boolean> sendMessage(String message);

}

And two implementations: SNSServiceOldSDK and SNSServiceNewSDK. Both services use async SNS client. There are some changes in implementation because the old SDK doesn’t support CompletableFuture and we have to make some trick.

What do you need to start the service?

To start the service you need :

· Java 11

· Maven

· AWS account (feel free to use free account)

If you are not familiar with Spring WebFlux I do recommend to read about this powerful mechanism.

From my point of view, it’s the most interesting Spring 5 feature. But I will focus on Spring WebFlux in my next article.

Swagger

I guess everybody knows Swagger and I also introduced it to this test project. There is one problem with Spring WebFlux and Swagger: there is still no release version of spring-fox 3.0.0 you can track the progress here

So let’s start measure performance…

Old and new SNS async clients have the configuration for the concurrent running request.

For old SDK it’s maxConnections

For new SDK it’s maxConcurrency

By default the maximum number of concurrently running requests is 50. This is rarely enough for a real high load service.

We will measure performance for 200 and 1000 concurrent running request

Let’s start with the old SDK

For sns.max.connection=200

Load scenario: 500 concurrent clients, 60 iterations

Result:

Result for old SDK for concurrent runing request= 200

JVisualVM result:

Result for old SDK for concurrent running request= 200. SNS Async client from the old SDK created 200 threads, CPU is about 30%. Heap size is 710 MB

New SDK

For sns.max.connection=200

Result for new SDK for sns.max.connection= 200

JVisualVM result:

SNS Async client from new SDK created only 20 threads, CPU is about 30%. Heap size is about 360 MB

Let’s increase sns.max.connection to 1000

Performance test scenario

OLD SDK

Result for new SDK for sns.max.connection= 1000

JVisualVM result:

SNS Async client from old SDK created 1000 threads, CPU is about 80%. Max size is about 1522 MB

NEW SDK

Result for new SDK for sns.max.connection= 1000

JVisualVM result:

SNS Async client from new SDK created only 20 threads, CPU is about 80%. Heap size is about 433 MB. All the same as for previous test

Conclusion:

The new AWS SDK demonstrated great performance and low resource usage. It creates only 20 threads comparing with 1000 for old client. Also application with new SDK takes two times less memory than with old SDK.

This is very important for a containerized application that operates with limited amount of memory. In Java(64-bit VM) single thread stack takes 1024k. So 1000 threads is plus 1GB memory. I recommend use new AWS SDK for non-blocking applications.

TestLink