For a number of years now, we have been using docker-compose to define our multi-container docker applications. Almost always, that has been in the context of local development or automation testing as part of our continuous integration and deployment pipelines. Perhaps we used docker-compose in proof-of-concepts and small non-critical applications hosted on a single server but generally this is where docker-compose ended its use in the development lifecycle. An extremely useful tool for local development and testing. When running our containers in upstream environments, we look to orchestration engines and managed cloud services such as Kubernetes, Amazon ECS and Amazon EKS.
Built-in integration with Amazon ecs
Recently, Docker announced built-in integration with Amazon ECS. Now, using native Docker Compose CLI commands, we can deploy our multi-container applications to Amazon ECS. What is neat about this integration is that the Docker tooling provisions the ECS environment for us (more on that below) and makes it (almost) seamless using a new `context` command line option. Using `context` we can switch between local and our ECS environment(s) and still use our docker-compose definitions and constructs.
Does this mean we should use this in production?
Well hold on there :face_with_cowboy_hat:!!
While this does extend the use of docker-compose off the local development environment (or CI/CD pipeline) It is not meant to replace production grade container orchestration engines like K8s, EKS or ECS. What it does excel at is allowing us to extend our developer toolchain into the cloud and accelerate the testing of our integration with other AWS services earlier in the development lifecycle.
Ok. So how does all this work?
To demonstrate this, I went and grabbed a typical Docker Compose 3-tiered demo app from Docker’s GitHub repository and made a few tweaks to make the demo easier. You can grab the demo solution from my GitHub repository if you want to follow along at home (https://github.com/ScottScovell/docker-ecs-demo/tree/feature/part-1).
The demo solution consists of a React.js frontend with a Node.js backend API and MySQL database.
version: “3.7”
services:
backend:
build:
args:
– NODE_ENV=development
context: backend
command: npm run start-watch
environment:
– DATABASE_DB=example
– DATABASE_USER=root
– DATABASE_PASSWORD=/run/secrets/db-password
– DATABASE_HOST=db
– NODE_ENV=development
– PORT=80
ports:
– 80:80
secrets:
– db-password
volumes:
– ./backend/src:/code/src:ro
– ./backend/package.json:/code/package.json
– ./backend/package-lock.json:/code/package-lock.json
– back-notused:/opt/app/node_modules
networks:
– public
– private
depends_on:
– db
db:
image: mysql:8.0.19
command: ‘–default-authentication-plugin=mysql_native_password’
restart: always
secrets:
– db-password
volumes:
– db-data:/var/lib/mysql
networks:
– private
environment:
– MYSQL_DATABASE=example
– MYSQL_ROOT_PASSWORD_FILE=/run/secrets/db-password
frontend:
build:
context: frontend
target: development
ports:
– 3000:3000
volumes:
– ./frontend/src:/code/src
– /code/node_modules
networks:
– public
depends_on:
– backend
networks:
public:
private:
volumes:
back-notused:
db-data:
secrets:
db-password:
file: db/password.txt
For the un-initiated, the power of docker-compose comes from the ability to spin “up” even a moderately complex application stack like this on your local machine in seconds and start developing. Let’s do that now and “up” our local development environment.
docker-compose –context default -f docker-compose.local.yaml up –build -d
You may notice two things jump out about the command in the line above. Firstly the use of the `context` parameter. This new parameter works akin to profiles in the AWS-CLI if you are accustomed to that. Here we are explicitly setting the Docker CLI context to the default (or local) context. Secondly, we are explicitly specifying the docker compose yaml file to use rather than the default `docker-compose.yaml`. More on that below.
First time through you’ll likely need to pull the base Docker images down so it may take a while (after that it only takes a few seconds). Our local docker-compose file tells the docker engine to build each of our docker images if they do not already exist locally in our cache. When complete you should see something like this
Successfully tagged docker-ecs-demo_frontend:latest
Creating docker-ecs-demo_db_1 … done
Creating docker-ecs-demo_backend_1 … done
Creating docker-ecs-demo_frontend_1 … done
From a browser you also should be able to navigate to the React frontend using localhost:3000 and invoke the backend API service on port 80.
Just as easily, we can also tear down the environment using the docker toolchain
Stopping docker-ecs-demo_frontend_1 … done
Stopping docker-ecs-demo_backend_1 … done
Stopping docker-ecs-demo_db_1 … done
Removing docker-ecs-demo_frontend_1 … done
Removing docker-ecs-demo_backend_1 … done
Removing docker-ecs-demo_db_1 … done
Removing network docker-ecs-demo_private
Removing network docker-ecs-demo_public
Note: I have pulled together a number of related commands into targets of a Makefile just to make the demo easier. This allows us to be explicit regarding with Docker context and corresponding docker compose yaml file to use without the lengthy command.
make dev-up
make dev-down
Using Docker to Integrate with Amazon ECS
Now that we have the local development environment running fine, let us look into using Docker to integrate with our AWS Account and “up” the same environment in Amazon ECS.
Pre-requisite:
- It is assumed you have the AWS CLI installed and configured. The profile being used must have permission to create IAM, VPC, EC2, ECS and other services (see below). This may be an issue if you are using your organisations AWS Account with restrictive permissions.
Create a New Context
The first step is to create a new `context` for the docker toolchain to use. Docker comes with a `default` context of type ‘moby’ which is the docker engine running on your local development engine. You manage your Docker contexts using the new Docker context command. Let’s list the existing contexts and note the active context in use (denoted using the asterisk)
You should see something like the following. Note the currently selected context
Create a new context of type `ecs`
In the line above, I have created a new ECS context named `aws`. The docker CLI will prompt you for an AWS-CLI profile and region. You can also create a new profile using the Docker CLI as well. You can name this context whatever you like. I used `aws` here and refer to it in the Makefile so make sure you change it in there if you are following my demo code and name your context something else. While you are at it, ensure you change the AWS account and region to match the AWS profile you are using as well.
If we list docker context again we see the following:
At this stage we have only setup the context. We have not deployed any resources into AWS. Only when we “up” our environment using Docker Compose do our resources get provisioned. Likewise, when we “down” our environment, the docker toolchain will clean up those resources so we are no longer charged.
Prepare Docker Compose file
Before we “up” the environment in AWS, we need to do some preparation to our Docker Compose file to align it with our target context.
version: “3.7”
services:
backend:
image: .dkr.ecr..amazonaws.com/docker-ecs-demo-backend:latest
command: npm run start-watch
environment:
– DATABASE_DB=example
– DATABASE_USER=root
– DATABASE_PASSWORD=/run/secrets/db-password
– DATABASE_HOST=db
– PORT=80
ports:
– 80:80
secrets:
– db-password
networks:
– public
– private
depends_on:
– db
db:
image: mysql:8.0.19
command: ‘–default-authentication-plugin=mysql_native_password’
restart: always
secrets:
– db-password
volumes:
– db-data:/var/lib/mysql
networks:
– private
environment:
– MYSQL_DATABASE=example
– MYSQL_ROOT_PASSWORD_FILE=/run/secrets/db-password
frontend:
image: .dkr.ecr..amazonaws.com/docker-ecs-demo-frontend:latest
ports:
– 3000:3000
networks:
– public
depends_on:
– backend
networks:
public:
private:
volumes:
db-data:
secrets:
db-password:
file: db/password.txt
Comparing the above with our local docker composer yaml file we have made changes to the following:
- Build: ECS can’t build our image as the docker engine does locally so we need to reference already built and tagged container images residing in a container registry like DockerHub or, in our case, Elastic Container Registry (ECR). We replace the build definitions with references to images we will push as part of our workflow.
- Volumes: During local development, we used persistent volumes to allow sharing of source code between our local IDE and the application code residing in the container. In ECS this is no longer required so we can remove this for our frontend and backend services. The database service (MySQL) will still need a persistent volume, so we don’t lose data between restarts of our container.
To preview the AWS resources that Docker will create for us we run the following docker command
docker –context aws compose convert -f docker-compose.ecs.yaml
Let us take a closer look at those resources created and how they relate back to our docker compose file
AWS Resource | Docker Compose | Description |
ECS Cluster | (common) | ECS cluster provisioned to run our multi-container application stack CloudMap namespace to manage our service discovery endpoints for each container service LoadBalancer for our exposed service endpoints |
ECS Service | service: | ECS service and task definitions for each container service
Sidecars containers for
Load balancer entries for ingress ports Service discovery entries for managing inter-service communications
|
– ports: | Security Group Ingress Rule for each exposed port defined for the service | |
– secrets: | (see secrets: below) | |
– networks: | Security Group for allowing traffic between configured networks (see networks: below) | |
Security Group | networks: | Public network security group Private network security group |
Elastic File Service | volumes: | EFS file system for persisting and sharing MySQL database files across multiple instances of our database service |
Secrets Manager Secret | secrets: | Secrets Manager secret for our MySQL database password |
Note: Docker will not create the ECR repositories referenced in your docker compose file. We will need to create these and push the referenced images into the repository, so it is available to ECS.
# Create ECR repositories
aws ecr create-repository –repository-name docker-ecs-demo-frontend
aws ecr create-repository –repository-name docker-ecs-demo-backend
# Login into ECR
aws ecr get-login-password –region <your aws region> | docker login –username AWS –password-stdin <your aws account>.dkr.ecr.<your aws region>.amazonaws.com
# Tag latest images with ECR repository name
docker –context default tag docker-ecs-demo_frontend:latest <your aws account>.dkr.ecr.<your aws region>.amazonaws.com/docker-ecs-demo-frontend:latest
docker –context default tag docker-ecs-demo_backend:latest <your aws account>.dkr.ecr.<your aws region>.amazonaws.com/docker-ecs-demo-backend:latest
# Push images to ECR
docker –context default push <your aws account>.dkr.ecr.<your aws region>.amazonaws.com/docker-ecs-demo-frontend:latest
docker –context default push <your aws account>.dkr.ecr.<your aws region>.amazonaws.com/docker-ecs-demo-backend:latest
The first time we run this, it will take some time to push each layer of our docker images into ECR. Subsequent changes to our images will only require those modified layers to be pushed.
"up" the multi-container application
Now let us “up” our multi-container application using our ECS docker compose file. We can either switch our current context to the one we created above (of type ecs) or we can just specify which context we want to apply the command to. I prefer the later just as a safe practice to ensure I don’t inadvertently spin up or send changes to the wrong context. If we wanted to switch context, we’d just use the `docker context use aws` command setting that context as the default.
Docker will generate the CloudFormation template we previewed above and apply it to the target environment as defined by the AWS Profile we configured against the context. Depending on the complexity of your application stack this may take 5-10 mins. We will see something like the following:
Note: Again, I’ve bundled the above workflow these into a handy make target, so we always remember to push the latest docker image into ECR before spinning up our ECS environment. Review the Makefile first and change to target your AWS environment.
When that completes, we should be able to navigate to our load balancer endpoint and verify our application stack is running in AWS. Find the DNS name of the load balancer endpoint for the stack we just created.
Open a browser and navigate to our DNS host name on port 3000. We should see our React application running in AWS. DNS may take some minutes after our AWS resources have been created to propagate as well so wait a few mins for that to update if you are having issues the first time.
Change to application and re-deploy
Now let us make a change to our application and re-deploy. We will make a simple change to the message being displayed from the backend. Currently it queries for and displays the name of the MySQL database used. Let’s make a small change to this text in the `server.js` file in our backend API.
Now that we have made our change, let’s test it locally using our local docker compose file and default docker context.
Happy with our local change, let’s push it to ECS using our ECS docker compose file and aws docker context
In the console we notice that our backend docker image has a new layer (change) that gets pushed to ECR and that a change to our ECS environment has also been triggered. A new ECS task definition version has been created informing ECS that a new container image is available and that instances of our backend service should be replaced with the new version.
Refreshing (or navigating to our AWS hosted application) should show the change in our ECS environment.
Clean up
To end, let’s clean up our resources so we are not running up costs when we are not using the resources. As mentioned before, we can easily do this the same way we do locally using the docker command line to tear down our environment.
Note: If you follow along with the demo project you can use ‘make clean’ after the above to clean up the ECR repositories created as part of this walkthrough
Also note that by default, EFS volumes are retained so you will need to clean up those resources separately if you are not intending to share them with other members of the development team.
Summary
In this post we have seen how we can apply familiar Docker toolchain commands and workflows used for local application development with containers to integrate with Amazon ECS. While not “seamless” by any notation of the word, we can see that integration with ECS is easy enough to configure and incorporate into development workflows. Having said that here are a few observations and suggestions on how this experience might be improved:
- It would be good not to maintain context specific docker-compose files That is, one for local dev and one for ECS dev. Perhaps some context specific attributes might be an approach here that are ignored by contexts that don’t support/need them.
- Add and display CloudFormation template outputs after we “up” our environment. This way we don’t need to go hunting for load balancer endpoints
- Ran into some issues defining different ports for host and container instance when using ECS context type (e.g. Expose port 80:3000)
In future posts we’ll explore further integration features and look at how we can share and re-use existing resources in our AWS account and how we might swap out containers used only for local mocking with cloud native services more representable of upstream production environments.
