Introduction

Our production deployment uses Amazon AWS, and in recent months, our overall cloud utilization has surged, driven by an increased demand for CPU and memory resources. Consequently, our cloud costs have gradually risen alongside the expanding workloads. To mitigate these costs, we recently explored a transition to AWS Graviton Processors, capitalizing on their ARM architecture for enhanced performance and cost-effectiveness. As our initial workloads were on AMD-powered EC2 instances, upgrading our build systems became imperative to create container images compatible with ARM processors. This blog post delves into the intricacies of our migration journey, detailing how we successfully transitioned our container workloads from AMD to ARM processors and the tangible benefits we derived from this strategic migration.

AMD vs ARM

Before we jump to the specifics, here’s a little background on AMD vs ARM processors.

AMD processors are type CISC (Complex Instruction Set Computers) processors. The primary goal of CISC architecture is to complete a task in as few lines of assembly as possible. This is achieved by building processor hardware that is capable of understanding and executing a series of operations. A CISC design will have instructions for every logical and mathematical function. For example, there will be an instruction for multiplying two numbers, square root of integer etc.

ARM, on the other hand, is a type of RISC (Reduced Instruction Set Computer) processor. RISC designs have fewer available instructions to program with. It relies on primitive instructions for doing complex or high-order functions. These reduced instructions require less transistors or hardware space than complex instructions, leaving more room for general-purpose registers.

The choice between RISC and CISC depends on specific application requirements, with RISC excelling in certain scenarios due to its streamlined design, while CISC offers advantages in handling intricate tasks within a single instruction.

Building container images for ARM processors

We utilize Docker build to create the container images for our services, encompassing a blend of Java and Go applications. While contemplating the migration of our production system to ARM processors, our non-production systems will persist on AMD processors. To ensure seamless compatibility across both environments, we’ve embarked on the task of building multi-arch images. The following sections elaborate on the steps we’ve taken to enhance our Java, Go, and shared external services, enabling the creation of multi-arch images that cater to both production and non-production systems.

Creating multi-arch images for Java services

Docker provides the simplicity of creating multi-arch images using docker buildx. We used this to build the multi-arch images for our Java services. We updated our build step to perform docker buildx build –platform linux/amd64,linux/arm64 … to create multi-arch images.

Before creating the multi-arch image, we also had to take care of the following:

1. Create multi-arch base image: Our Java services use a base image of openjdk-17 which is already a multi-arch image. We extend this image to include tomcat and use that as the base image for all our services. We modified the build step to create a multi-arch base tomcat-openjdk-17 image using docker buildx.
2. Templatize external package url: Some of our services bundle external packages in the image. These references were previously of the form ADD https://external-package/package-<version>-linux-amd64.tar.gz /tmp/. We injected the ARGs TARGETOS and TARGETARCH in the Dockerfile and parameterized these references to ADD https://external-package/package-<version>-${TARGETOS}-${TARGETARCH}.tar.gz /tmp. This resulted in pulling the appropriate package for linux/arm64 and linux/amd64 build stages.

Creating multi-arch images for Go services

We used ko – a simple, fast container image builder for Go applications – to build multi-arch container images for our golang services. We used the following directive in .ko.yaml to build images for linux/arm64 and linux/amd64

defaultPlatforms:
  - linux/arm64
  - linux/amd64

Creating multi-arch images for external shared services

We use Kafka, ZooKeeper, and Mongo in our stack. The public images for these services are already multi-arch. We retagged these images using the docker buildx image tools and pushed these images to our repository. It is important to use image tools as simple retagging results in the creation of images for the platform from which the command was run.

docker buildx imagetools create -t ghcr.io/nirmata/mongo:tag mongo:tag

Observations after the migration

Following the migration, we observed a substantial decrease in overall CPU requirements, with CPU cores operating at just 50% of their previous levels. While the memory footprint experienced a moderate reduction, it was not as pronounced as the CPU. Encouragingly, our continuous API performance suite demonstrated no degradation, particularly in the 99th percentile latency results. Additionally, our EC2 costs exhibited a slight decline, attributed to the cost-effectiveness of Graviton nodes compared to their AMD counterparts. This holistic improvement underscores the efficacy of our transition to Graviton in optimizing resource utilization and reducing operational costs.

Conclusion

In our journey to optimize performance and reduce costs, the decision to migrate our container workloads from AMD to ARM processors proved to be a pivotal one. By leveraging the efficiency of ARM architecture, we achieved significant reductions in CPU requirements, leading to a remarkable 50% decrease in CPU core utilization. While the impact on memory footprint was noticeable, the most substantial gains were observed in overall EC2 costs, thanks to the cost-effectiveness of Graviton nodes compared to their AMD counterparts.

The process of transitioning to ARM processors was streamlined through the use of powerful tools like Docker Buildx and Ko, allowing us to create and deploy multi-arch images seamlessly. The speed and efficiency of this migration, coupled with the sustained performance indicators in our continuous API performance suite, underscore the effectiveness of this strategic shift.

Our experience underscores the importance of choosing the right tools for the job and embracing emerging technologies. The reduced resource requirements post-migration not only optimize our current infrastructure but also provide the scalability headroom required for future growth. For those embarking on their cloud journey or considering a migration, we wholeheartedly recommend exploring ARM64 processors. If you find yourself on AMD processors, the benefits we’ve realized make a compelling case for making the switch to ARM.

It’s worth noting that while ARM processors present numerous advantages, some complex instruction dependencies, such as floating-point instructions, may require careful consideration. Despite this, the benefits we’ve reaped in terms of performance, cost, and scalability make the transition well worth the effort.

In conclusion, the success of our migration to ARM processors reinforces the importance of staying abreast of advancements in technology. We invite you to embrace this paradigm shift and unlock the potential benefits it may bring to your infrastructure.

References

AWS online talk on how to run containerized workloads on Graviton-based EC2 instance: https://www.youtube.com/watch?v=iwSQRLzDwHA

RISC vs CISC: https://cs.stanford.edu/people/eroberts/courses/soco/projects/risc/risccisc/

Building multi-arch image using docker buildx: https://www.docker.com/blog/multi-arch-build-and-images-the-simple-way/

Building multi-arch image using ko: https://ko.build/

Retagging multi-arch image: https://docs.docker.com/engine/reference/

commandline/buildx_imagetools_create/

Introduction to the AWS Graviton Processor: ARM Processor – AWS Graviton Processor

If you are interested in policy-based security and governance for Kubernetes, try the Nirmata Policy Manager.