Cloud shrinkflation

July 23, 2024 - 10 mins read

After having witnessed the atrocious buzz of LLMs and the rise of AI-at-every-cost, I’ve turned back to a world where I’m more familiar and that lost its fame, cloud.

This post delves into optimizing Java Spring Boot startup times, a crucial aspect of cloud deployments.

Disclaimer : this post will be a bit more technical than the previous one but you can still read it after having a pint. While I aim for working code snippets, intermediate ones might need adjustments. This article consolidates information from various sources, applying them to a real-world scenario. Also something worth to notice, all this walkthrough article sums up all the Baeldung guides and stuff you can find on the topic, the goal of this post is to share a real-life application and not the Spring PetShop (although I like cats).

The Need for Speed

Because I’m a Java developer with more or less 8 years of experience, I’ve begin my backend journey using Spring to deploy my spaghetti .war on the Apache Tomcat. This meat ball was shipped to the customer who was in charge to deploy it on its servers, and we had absolutely no vision on it, no need to say but the target architecture was a big monolith.

As the industry moved, I’ve moved on Spring Boot and containers, what a wonderful world to be able to spin an application just with few command lines and not having to set up .zipped Tomcat in your Eclipse, which was leading to strange manipulations.

In my final form, I’ve discovered Kubernetes and container orchestration, because automation is flowing in my veins this is the pinnacle of what I could hope.

I just lost my pod in K8S.

All these fabulous experiences (no) have crossed the path of my OCD, and especially on the field of startup times.

Sonic the hedgehog

In most of the articles about serverless performances or cloud architectures, one key concept is the cold start time that allows to spin backend units instantly. I haven’t been so far as I mainly worked on thick backends, but I was very concerned by the start time topic, as Kubernetes Readiness and Liveness probe are a pain to tune.

Generally speaking, I wanted to optimize the way I build my ~~shit~~ engineering prowess, so let’s begin. You will have to entrust me with all the adaptations, I have no bench to prove my work only your love.

Get this container out of my sight

The initial Dockerfile was straightforward:

FROM eclipse-temurin:21
WORKDIR /app
COPY target/*.jar /app/scep.jar
EXPOSE 9000
ENTRYPOINT ["java", "-jar", "scep.jar"]

Here we’re taking advantage from the past mvn clean install performed by the Gitlab Runner.

But this method has some flaws:

You have to compile in local before packaging the container.
We’re using a JDK which might have unecessary dependencies.

# Build stage
FROM maven:3.9.5-eclipse-temurin-21 AS build
COPY . /app
WORKDIR /app
RUN mvn --settings ./.m2/settings.xml clean install -DskipTests

# Run stage
FROM eclipse-temurin:21
WORKDIR /app
COPY --from=build /app/target/*.jar /app/scep.jar
EXPOSE 9000
ENTRYPOINT ["java", "-jar", "scep.jar"]

Industrialization here we go, we have now a two stages Dockerfile.

The first stage is dedicated to the mvn clean install and the second one fetch the result of this compilation so we have total control over our compilation environment instead of relying on shady Gitlab runners, in my company where we have several runners dedicated to different network zones, this is helpful.

Database externalization

Let’s do better, because I work with Liquibase which has become the de-facto standard for schema migrations, maybe we can gain some times if we remove the burdens of a Liquibase migration that we can’t control on the execution time (sometimes it’s painful, sometimes it can be short).

We refactor all the database modelization in a separate maven module, and have a dedicated Dockerfile for this :

# Build stage
FROM maven:3.9.5-eclipse-temurin-21 AS build
COPY .. /app
WORKDIR /app
RUN mvn \
    --settings .m2/settings.xml \
    clean install -pl scep-model -am \
    -DskipTests

# Run stage
FROM liquibase/liquibase:4.28-alpine
COPY --from=build /app/scep-model/target/classes/db/changelog.xml /liquibase/changelog/changelog.xml
COPY --from=build /app/scep-model/target/classes/specific_ressource.tab /liquibase/specific_ressource.tab

That sounds great ! Now the whole modelization has its dedicated container and is properly processed by Liquibase with the main changelog.xml file, and its ressources.

JVM, garlic and herbs

Time to get back to our bread & butter Java process, in the mean time behind the scenes the application was exploded between several modules to reach a microservices architecture.

We’ll focus on the build of one of theses module as the build process is the same for all of them.

One hint we could explore is the custom JRE building, in my previous examples there, we were relying on Eclipse Temurin JDK versions, but there are probably JVM modules and stuff we don’t need.

# Build stage
FROM maven:3.9.5-eclipse-temurin-21 AS build

COPY .. /app
WORKDIR /app

RUN mvn \
    --settings .m2/settings.xml \
    clean install -pl dispatcher -am \
    -DskipTests && \
    # Unjar.
    jar xf /app/dispatcher/target/dispatcher.jar && \
    # Jdeps.
    jdeps --ignore-missing-deps -q \
        --recursive \
        --multi-release 21 \
        --print-module-deps \
        --class-path 'BOOT-INF/lib/*' \
        /app/dispatcher/target/dispatcher.jar > deps.info && \
    # Jlink to build JRE.
    jlink \
        --add-modules $(cat deps.info),jdk.crypto.ec \
        --strip-debug \
        --no-header-files \
        --no-man-pages \
        --output /scep_jre

# Run stage
FROM debian:buster-slim

# JRE configuration
ENV JAVA_HOME "/opt/java/jdk"
ENV PATH "$PATH:$JAVA_HOME/bin"

COPY --from=build /scep_jre $JAVA_HOME

WORKDIR /app
COPY --from=build /app/dispatcher/target/dispatcher.jar /app/dispatcher.jar

EXPOSE 9000
ENTRYPOINT ["java", "-jar", "/app/dispatcher.jar"]

Here we can now use a base image from Debian, we scanned the JVM modules using jdeps and re-used its result in the jlink, please note the addition of the module jdk.crypto.ec which is required for all SSL operations (for example JDBC using SSL), I don’t know so far why jdeps didn’t scanned it.

Ok let’s sum up, now we are relying only on the docker image for our application compilation and packaging, we have tailored a JVM for this application and we vented out the Liquibase part that tooks an unpredictable time for migration.

OCD time

But we wan’t to do better, it’s time to introduce Class Data Sharing (CDS). In a nutshell, we will run a first time the target application and create a snapshot, leveraging Spring Context Events to kill the process right after all the beans initialization.

# Build stage
FROM maven:3.9.5-eclipse-temurin-21 AS build

COPY .. /app
WORKDIR /app

RUN mvn \
        --settings .m2/settings.xml \
        clean install -pl dispatcher -am \
        -DskipTests && \
    # Move the jar to the target folder.
    mv ./dispatcher/target/dispatcher.jar ./dispatcher.jar && \
    # Unjar.
    jar xf /app/dispatcher.jar && \
    # Jdeps.
    jdeps --ignore-missing-deps -q \
        --recursive \
        --multi-release 21 \
        --print-module-deps \
        --class-path 'BOOT-INF/lib/*' \
        /app/dispatcher.jar > deps.info && \
    # Jlink to build JRE.
    jlink \
        --add-modules "$(cat deps.info)",jdk.crypto.ec \
        --strip-debug \
        --no-header-files \
        --no-man-pages \
        --generate-cds-archive \
        --output /scep_jre

# Run stage
FROM debian:bookworm-slim

# JRE configuration
ENV JAVA_HOME "/opt/java/jdk"
ENV PATH "$PATH:$JAVA_HOME/bin"

COPY --from=build /scep_jre $JAVA_HOME

WORKDIR /app
COPY --from=build /app/dispatcher.jar /app/dispatcher.jar

# Generate CDS archive
RUN java -XX:ArchiveClassesAtExit=/app/cds-dispatcher.jsa \
    -Dspring.context.exit=onRefresh \
    -jar /app/dispatcher.jar \
    --spring.liquibase.enabled=false \
    --spring.jpa.hibernate.ddl-auto=none \
    --spring.sql.init.mode=never \
    --spring.jpa.properties.hibernate.boot.allow_jdbc_metadata_access=false \
    --spring.jpa.properties.hibernate.temp.use_jdbc_metadata_defaults=false \
    --spring.datasource.hikari.allow-pool-suspension=true \
    --application.output.folder=/app \
    --cds.mode=true \
    --spring.main.lazy-initialization=false

EXPOSE 9000
ENTRYPOINT ["java", "-XX:SharedArchiveFile=/app/cds-dispatcher.jsa", "-jar", "/app/dispatcher.jar"]

There are many things that should draw your attention there :

spring.context.exit=onRefresh to listen properly the Spring Context Events.
All the configuration properties that are overriden so we can do a dry-run in Docker without the need of a local database or message queuing technology.
Here I have implemented a cds.mode property key so some of my beans skip PostConstruct logic only for the sake of the CDS run. For SAST tools it might be dead code but it will represents only a dozen LoC so I’m fine with it.
spring.main.lazy-initialization=false is one of the strategy to have an optimized startup time, all the beans instantiation logic can be processed at the first request instead of heavy loading everything. As we are performing a dry-run, we need heavy loading.

Cool, this Spring Boot has now a built-in dry-run, relying on its tailored JRE. Maybe we could go further ?

Push it to the limits

We want to go deeper, and deeper, so let’s use what Spring offers us, Spring Ahead Of Time compilation. This will make Spring inspect our application and perform compilation optimization for bean management instead of the runtime discovery.

# Build stage
FROM maven:3.9.5-eclipse-temurin-21 AS build

COPY .. /app
WORKDIR /app

RUN mvn \
        --settings .m2/settings.xml \
        clean install -pl dispatcher -am \
        -DskipTests && \
    # Move the jar to the target folder.
    mv ./dispatcher/target/dispatcher.jar ./dispatcher.jar && \
    # Unjar.
    jar xf /app/dispatcher.jar && \
    # Jdeps.
    jdeps --ignore-missing-deps -q \
        --recursive \
        --multi-release 21 \
        --print-module-deps \
        --class-path 'BOOT-INF/lib/*' \
        /app/dispatcher.jar > deps.info && \
    # Jlink to build JRE.
    jlink \
        --add-modules "$(cat deps.info)",jdk.crypto.ec \
        --strip-debug \
        --no-header-files \
        --no-man-pages \
        --generate-cds-archive \
        --output /scep_jre

# Optimization stage
FROM debian:bookworm-slim AS optimizer
ENV JAVA_HOME "/opt/java/jdk"
ENV PATH "$PATH:$JAVA_HOME/bin"

WORKDIR /app

COPY --from=build /scep_jre $JAVA_HOME
COPY --from=build /app/dispatcher.jar ./
RUN java -Djarmode=layertools -jar dispatcher.jar extract

# Run stage
FROM debian:bookworm-slim

# JRE configuration
ENV JAVA_HOME "/opt/java/jdk"
ENV PATH "$PATH:$JAVA_HOME/bin"

WORKDIR /app

COPY --from=optimizer $JAVA_HOME $JAVA_HOME
COPY --from=optimizer /app/dependencies/ ./
COPY --from=optimizer /app/spring-boot-loader/ ./
COPY --from=optimizer /app/snapshot-dependencies/ ./
COPY --from=optimizer /app/application/ ./

# Generate CDS archive
RUN java -XX:ArchiveClassesAtExit=/app/cds-dispatcher.jsa \
    -Dspring.aot.enabled=true \
    -Dspring.context.exit=onRefresh \
    org.springframework.boot.loader.launch.JarLauncher \
    --spring.liquibase.enabled=false \
    --spring.jpa.hibernate.ddl-auto=none \
    --spring.sql.init.mode=never \
    --spring.jpa.properties.hibernate.boot.allow_jdbc_metadata_access=false \
    --spring.jpa.properties.hibernate.temp.use_jdbc_metadata_defaults=false \
    --spring.datasource.hikari.allow-pool-suspension=true \
    --application.output.folder=/app \
    --cds.mode=true \
    --spring.main.lazy-initialization=false

EXPOSE 9000
ENTRYPOINT ["java", "-Dspring.aot.enabled=true", "-XX:SharedArchiveFile=/app/cds-dispatcher.jsa", "org.springframework.boot.loader.launch.JarLauncher"]

The key part there to notice is the optimizer stage that has been introduced. This stage will explode our fat jar in the directory, for more efficient file access instead of lookup in the jar. It will also fetch the tailored JVM, so we’ll have everything at disposal.

Then we’ll work in the Run stage, copying everything from the optimizer stage, only note the addition of -Dspring.aot.enable=true both at the CDS and entrypoint parts.

Can we go deeper ?

I think so too.

The ultimate step there to go further would be native compilation with tool such as GraalVM, but there are pre-requisites such as removing every profile-related properties or specifics way to nest configuration beans to be able to tackle the reflection which is mainly used by Spring.

However, there are similarities with all the processing we’ve done so far such as dry-run to generate beans hint or use of Ahead-Of-Time compilation.

Another strategy to explore is to limit the transitional dependencies of Spring, so you use only what’s needed, and if they are unecessary remove them from the startup initialization:

...
@EnableAutoConfiguration(excludeName = { 
		"org.springframework.cloud.client.ReactiveCommonsClientAutoConfiguration",
		"org.springframework.cloud.client.discovery.simple.SimpleDiscoveryClientAutoConfiguration",
		"org.springframework.cloud.client.CommonsClientAutoConfiguration",
		"org.springframework.cloud.commons.config.CommonsConfigAutoConfiguration",
		"org.springframework.cloud.client.loadbalancer.LoadBalancerDefaultMappingsProviderAutoConfiguration",
		"org.springframework.cloud.client.discovery.composite.CompositeDiscoveryClientAutoConfiguration",
		"org.springframework.cloud.client.serviceregistry.ServiceRegistryAutoConfiguration",
		"org.springframework.cloud.autoconfigure.LifecycleMvcEndpointAutoConfiguration"
		})
...
@Configuration
public class DispatcherConfiguration {
}

These are the beans I removed in my application configuration bean but the list could be far longer, it’s a real pain to investigate what’s really required.

I hope this post will generate less frustration than the one I felt when I started to optimize everything I could.

The activity of Spring native compilation is interesting to track as it’s the promise for Function as a Service powered by Spring, but for now I’d stick to the JVM, especially for the build time part which might be a pain point in a team with few developers: Blog post about Spring native performances

It’s worth to notice that at the current state, Spring Native is outperformed by language such as Go that are betterly designed for FaaS: HelloWorld comparison case between Spring Native and Go

At last, there’s an interesting point we can mention, but if in your CI there are tools such as Trivy for container scanning, you should be happy with the outcome as the tailored JRE reduce the attack surface to it’s minima.