Full Stack Serverless CI/CD On AWS With The Serverless Framework – Part 1

Cloud computing has certainly changed the way we build, deploy and run our application workloads. No longer do we want to be in the business of waiting for infrastructure to be provisioned, configuring web and application servers, bouncing back and forth with networking teams about firewall rules and SSL certificates…and we haven’t even written a line of code yet that delivers actual value to the business. If this broadly describes the way you still develop applications, then keep reading!

There are several cloud compute models that seek to address these concerns such as containerisation and platform-as-a-service. Many of them advance the cause somewhat, reducing (usually via automation) the infrastructure provisioning, operational cost, application deployment and networking configuration effort – and yet these models still require someone in the team to feed, water and take care of them. That is, manage, monitor, scale, build in high availability and fault tolerance.

Serverless computing abstracts these infrastructure concerns altogether, allowing organisations to just get on with delivering the pieces that provide value to the business…the code itself.

Serverless platforms provide:

• Provisioning, monitoring and management of the underlying infrastructure
• Automatic scaling as demand goes up/down
• High availability so that your application is always responsive to users even during upgrades and feature releases
• Fault tolerance so that your application is not impacted when service outages occur in the underlying infrastructure
• Cost reduction as we typically only pay for the resources consumed when our code is invoked

Serverless application frameworks provide an opinionated, repeatable and automated approach to build, deploy and run serverless applications. Many frameworks target just one public cloud platform (e.g. AWS Serverless Application Model or Apex/Up) or coding language, while others provide cross cloud platform and multi language support (e.g. Serverless Framework) and allow a consistent development toolchain no matter which cloud provider or language used.

In this post, we will walk through automating the delivery of a full stack serverless application on AWS using the Serverless Framework.

The sample application we’ll use consists of a single-page-application (SPA), backend APIs and a NoSQL data store. Our frontend will be hosted using AWS S3 static website hosting (minus a CloudFront CDN for simplicity) and business logic running on AWS Lambda sitting behind AWS API Gateway, our API governance and security layer. Our Lambda functions read/write from a DynamoDB data store and integrate with Twilio, a 3rd party API for sending SMS.

Note: Feel free to adapt to use your own application (it’s not the app we are interested in, it’s the approach)

BEFORE YOU BEGIN

You’ll need to have the following available to complete this walk through:

• AWS Account and AWS CLI client credentials
• AWS CLI installed and configured on your development machine
• Serverless Framework CLI installed on your development machine
• Twilio Account (full or trial) configured with a registered number to send SMS messages

STEP 1 – PREPARING OUR SOURCE CODE REPOSITORY

In this step we will prepare our source code repository to manage our code but also trigger our continuous integration pipeline as well. Here we’ll use AWS CodeCommit but feel free to adapt to suit other code repositories (e.g. GitHub).

1. Create an AWS CodeCommit repository for your app in the AWS console or use the following AWS CLI command from your development machine.

   aws codecommit create-repository --repository-name serverless-cicd --repository-description "Full stack serverless CI/CD on AWS with the Serverless Framework"
   

2. Git clone the new, empty repo to your development machine

   git clone 
   

STEP 2 – GENERATING APPLICATION SCAFFOLDING USING THE SERVERLESS FRAMEWORK

In this step we use the Serverless Framework to generate the application scaffolding using a ready-made template targeting a selected cloud provider and runtime (e.g. AWS and NodeJS).

1. In the terminal on your development machine, move to the directory where you cloned your new repo

2. Create the application scaffolding using the Serverless Framework CLI

   ”’ serverless create –template aws-nodejs –name chaos-button-app “’

In the step above we used the built-in serverless framework template relevant to a NodeJS serverless application running on AWS. The Serverless Framework provides several other templates that you can use to target specific coding language and cloud provider.

Run serverless create --help for a list of available templates or provide a URL to a git repository containing a custom template.

Note: For the purpose of this post, we’ll discard this scaffolding and use the following sample application that demonstrates the key points covered in the post.

STEP 3 – PROJECT WALKTHROUGH

1. Unzip the sample application into the directory you cloned your AWS CodeCommit repository to, overwriting the scaffolding code generated in the step above. https://github.com/ScottScovell/serverless-cicd

2. View the serverless.yml template in your favourite IDE (I’m using VS Code)

   Let’s walk through some of the points of interest in the file:

   • service – Not only the name of our serverless application but also a naming component of our AWS resources that will be provisioned by the framework.

   • plugins– Specifies that we want to use serverless-finch, one of the many community plugins, to extend the Serverless Framework to support static website hosting in AWS S3.

   • custom – Specifies some custom variables and configuration for custom components like serverless-finch.

   • provider – Specifies the cloud provider and application runtime we are targeting.

   • environment – Specifies the runtime configuration our application needs as we promote it through each environment. Here we are pulling in environment configuration settings from AWS System Manager’s parameter store where we have our AWS client credentials stored as well as API credentials for Twilio.

   • iamRoleStatements – The Serverless Framework creates an AWS IAM role for our Lambda functions but sometimes we need to grant the role additional permissions. Here we grant our functions permissions to a DynamoDB table (also provisioned in the next section)

   • resources – Specifies additional resources our stack needs using AWS CloudFormation template language. Here we reference a template fragment under the resources folder in our project.

   • functions – Describes our serverless functions and the event sources that trigger them. In our case, AWS Lambda functions with HTTP event triggers via AWS API Gateway.

3. The frontend of our sample application has a simple javascript single-page-application (SPA) residing in the client/dist folder of the project. The frontend calls our API Gateway so needs to know the API endpoint for the environment it is running in.

   We do not want to store these endpoints in code. Firstly, because it’s not good practice to do so and secondly, we don’t know what the endpoint will be as its created during deployment. To be truly serverless means we shouldn’t concern ourselves with the underlying infrastructure. We’ll let the framework and our continuous delivery pipeline take care of this for us.

4. Our backend API’s contain our business logic, and this is where we want to concentrate developer time and effort, not the underlying infrastructure. Our business logic resides in a NodeJS function (in the handler.js file) which connects to a NoSQL database to fetch data and also consumes a 3rd party API to send SMS messages.

   How we host and expose this business logic to our frontend is where the Serverless Framework comes in. Instead of the team standing up an API gateway, configuring a REST interface, wiring up the integration to a configured application and provisioning a persistent database server, we simply declare our runtime dependencies in our serverless.yml file and let the framework take care of the rest. Changes to any of these resources are managed alongside our source code ensuring no environment drift occurs.

5. To support automated deployment, our project contains several artefacts that our CI/CD pipeline needs.

   • buildspec.yml – specifies what the build server needs in order to invoke the Serverless Framework CLI to package our serverless application.

   • buildspec-deploy.yml and deploy.sh – enables the deployment stage of our pipeline to call the Serverless Framework CLI to provision the specified environment and inject environment settings into our SPA during deployment.

6. Commit your changes to AWS CodeCommit as we’ll need the project in our source repository to kick-off our CI/CD pipeline.

In this post we introduced the benefits of a serverless architecture, setup our project and walked through the project structure and Serverless Framework artefacts. serverless deploy and serverless client deploy are the only commands we need to provision our entire serverless stack and we can do this right from our development machine now if we like…again, not good practice. In the next post, we’ll automate this using a continuous integration and delivery pipeline.