Implementing a Serverless Flask REST API using AWS SAM

By Adam McQuistan in Serverless  12/02/2020 Comment

Introduction

In this article I demonstrate how to build a serverless REST API using the AWS Serverless Application Model (SAM) framework along with the popular Python based Flask micro web framework. SAM is a fantastic developer productivity booster which serves to wrap and extend CoudFormation as well as provide a CLI on top of the standard AWS CLI. Collectively SAM enables smooth, repeatable, and automated dev-test-deploy cycles within the AWS Cloud.

The complete code for this tutorial's project is on my GitHub account available in this repo.

Setup of AWS IAM, CLI and SAM

The first necessary to get started is to create an AWS IAM user with programmatic access and sufficient privilege policies to run the SAM CLI. A simple way to do this is to create a IAM user with AdministratorAccess policy. However, this is arguably a heavy handed approach so, please act in accordance with your organization's policies and procedures. The IAM user I'll be using for this article is named serverless-admin.

Aside from the note about creating an appropriate IAM user for running your SAM CLI to interact with your AWS account the SAM Docs on AWS do a very thorough job of describing how to setup the standard AWS CLI along with the SAM CLI. Please consult the linked docs for the remainder of the initial setup then rejoin in the next section where I walk through scafolding out the SAM project.

Building Out the SAM Project Structure and Template

AWS Serverless Application Model (SAM) utilizes simplified templating Infrastructure as Code (IoC) abstraction ontop of the standard CloudFormation specification as described in the SAM docs. In this section I build out the basic structure of my SAM project structure by piggy backing off the sam init command, specifying a valid Python3 runtime I have on my local machine(Python 3.8), give the app a name of lol-api, then select the Hello World Example template in the interactive terminal session as shown below.

$ sam init --runtime python3.8 --name lol-api
Which template source would you like to use?
	1 - AWS Quick Start Templates
	2 - Custom Template Location
Choice: 1

Cloning app templates from https://github.com/awslabs/aws-sam-cli-app-templates.git

AWS quick start application templates:
	1 - Hello World Example
	2 - EventBridge Hello World
	3 - EventBridge App from scratch (100+ Event Schemas)
	4 - Step Functions Sample App (Stock Trader)
	5 - Elastic File System Sample App
Template selection: 1

-----------------------
Generating application:
-----------------------
Name: lol-api
Runtime: python3.8
Dependency Manager: pip
Application Template: hello-world
Output Directory: .

Next steps can be found in the README file at ./lol-api/README.md

Now if I inspect the directory structure that was created withthe help of the unix tree command I see the contents shown below.

$ tree lol-api
lol-api
├── README.md
├── events
│   └── event.json
├── hello_world
│   ├── __init__.py
│   ├── app.py
│   └── requirements.txt
├── template.yaml
└── tests
    └── unit
        ├── __init__.py
        └── test_handler.py

4 directories, 8 files

The Hello World exmaple provides a pretty good functional example and structure for my project. If you are following along feel free to take a quick look at the files but, I'll be quickly moving along to customize this boiler plate starter project to serve my end goal of having a Python/Flask serverless REST API.

To start I rename the hello_world directory to flask_api then I update the requirements.txt to include the dependencies specific to building a Flask based Lambda API which are Flask-Lambda, requests and boto3 as shown below.

# requirements.txt
flask-lambda
boto3
requests

For the moment I ignore the contents of the app.py file and instead move on to the template.yaml, meant to contain the IaC for the app, replacing it's contents with that which are shown next.

# template.yaml

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
  This is a SAM template for lol-api which is a Serverless API for 
  composing and managing Lots of Lists.

# More info about Globals: https://github.com/awslabs/serverless-application-model/blob/master/docs/globals.rst
Globals:
  Function:
    Timeout: 10

Parameters:
  ExecEnv:
    Type: String
    AllowedValues:
      - local
      - dev
      - stage
      - prod
    Default: prod
  LotsOfListsTableName:
    Type: String
    Default: lots-of-lists

Resources:
  LotsOfListsFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: flask_api/
      Handler: app.app
      Runtime: python3.8
      Environment:
        Variables:
          TABLE_NAME: !Ref LotsOfListsTableName
          REGION_NAME: !Ref AWS::Region
          EXEC_ENV: !Ref ExecEnv
      Policies:
        - DynamoDBCrudPolicy:
            TableName: !Ref LotsOfListsTable
      Events:
        FetchLists:
          Type: Api
          Properties:
            Path: /lists
            Method: get
        CreateList:
          Type: Api
          Properties:
            Path: /lists
            Method: post
        FetchList:
          Type: Api
          Properties:
            Path: /lists/{id}
            Method: get
        UpdateList:
          Type: Api
          Properties:
            Path: /lists/{id}
            Method: put
        DeleteList:
          Type: Api
          Properties:
            Path: /lists/{id}
            Method: delete

  LotsOfListsTable:
    Type: AWS::DynamoDB::Table
    Properties:
      TableName: !Ref LotsOfListsTableName
      AttributeDefinitions:
        - AttributeName: listId
          AttributeType: S
        - AttributeName: userId
          AttributeType: S
      KeySchema:
        - AttributeName: userId
          KeyType: HASH
        - AttributeName: listId
          KeyType: RANGE
      ProvisionedThroughput:
        ReadCapacityUnits: 2
        WriteCapacityUnits: 2


Outputs:
  # ServerlessRestApi is an implicit API created out of Events key under Serverless::Function
  # Find out more about other implicit resources you can reference within SAM
  # https://github.com/awslabs/serverless-application-model/blob/master/docs/internals/generated_resources.rst#api
  LotsOfListsApi:
    Description: "API Gateway endpoint URL for Prod stage for Lots Of Lists function"
    Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/lists"
  LotsOfListsFunction:
    Description: "Lots Of Lists Lambda Function ARN"
    Value: !GetAtt LotsOfListsFunction.Arn
  LotsOfListsFunctionIamRole:
    Description: "Implicit IAM Role created for Lots Of Lists function"
    Value: !GetAtt LotsOfListsFunctionRole.Arn

One of the best things about both the SAM templating syntax and the CloudFormation spec that it is built on is the clear expressiveness and self documenting nature of Infrastructure as Code templates. I feel like anyone with a modest understanding of common AWS Services can read a SAM / CloudFormation template and generally understand what the corresponding stack will produce. That being said, I will not being into the details of the syntax other than to say that the SAM templating spec allows for direct usage of standard CloudFormation so, if there is something that the regular SAM Resource specification does not cover that you require then simply drop down to the standard CloudFormation template language and you are set.

Coding the Flask REST API

With the infrastructure defined in the template.yaml along with the basic project file structure and Python dependencies specified what remains is the fun part. Writing App Code!

Heading over to a suitable code editor I open flask_api/app.py and delete all the initial content provided by the Hello World SAM initialization template and move on by importing dependencies, instantiating a instance of Flask-Lambda then define a few constants representing the environment variables I specified in in the last section's template.yaml file. After that its really just implementing the standard Flask API for building REST API and interacting with AWS DynamoDB via the AWS Python Boto3 SDK as shown below. It's worth meantioning at this point that FlaskLambda is rather simple wrapper that simply intercepts and shuffles around requests and responses between formats that are compatible with Lambda functions and the popular Flask web microframework to provide developers the ability to use the familar API of Flask.

import os
import uuid

import boto3
from boto3.dynamodb.conditions import Key

from flask import request, jsonify
from flask_lambda import FlaskLambda

EXEC_ENV = os.environ['EXEC_ENV']
REGION = os.environ['REGION_NAME']
TABLE_NAME = os.environ['TABLE_NAME']


app = FlaskLambda(__name__)

if EXEC_ENV == 'local':
    dynamodb = boto3.resource('dynamodb', endpoint_url='http://dynamodb:8000')
else:
    dynamodb = boto3.resource('dynamodb', region_name=REGION)


def db_table(table_name=TABLE_NAME):
    return dynamodb.Table(table_name)


def parse_user_id(req):
    '''When frontend is built and integrated with an AWS Cognito
       this will parse and decode token to get user identification'''
    return req.headers['Authorization'].split()[1]


@app.route('/lists')
def fetch_lists():
    try:
        user_id = parse_user_id(request)
    except:
        return jsonify('Unauthorized'), 401

    tbl_response = db_table().query(KeyConditionExpression=Key('userId').eq(user_id))
    return jsonify(tbl_response['Items'])


@app.route('/lists', methods=('POST',))
def create_list():
    list_id = str(uuid.uuid4())
    try:
        user_id = parse_user_id(request)
    except:
        return jsonify('Unauthorized'), 401

    list_data = request.get_json()
    list_data.update(userId=user_id, listId=list_id)
    tbl = db_table()
    tbl.put_item(Item=list_data)
    tbl_response = tbl.get_item(Key={'userId': user_id, 'listId': list_id})
    return jsonify(tbl_response['Item']), 201


@app.route('/lists/<string:list_id>')
def fetch_list(list_id):
    try:
        user_id = parse_user_id(request)
    except:
        return jsonify('Unauthorized'), 401

    tbl_response = db_table().get_item(Key={'userId': user_id, 'listId': list_id})
    return jsonify(tbl_response['Item'])


@app.route('/lists/<string:list_id>', methods=('PUT',))
def update_list(list_id):
    try:
        user_id = parse_user_id(request)
    except:
        return jsonify('Unauthorized'), 401

    list_data = {k: {'Value': v, 'Action': 'PUT'}
                for k, v in request.get_json().items()}
    tbl_response = db_table().update_item(Key={'userId': user_id, 'listId': list_id},
                                          AttributeUpdates=list_data)
    return jsonify()


@app.route('/lists/<string:list_id>', methods=('DELETE',))
def delete_list(list_id):
    try:
        user_id = parse_user_id(request)
    except:
        return jsonify('Unauthorized'), 401

    db_table().delete_item(Key={'userId': user_id, 'listId': list_id})
    return jsonify()

After the code is written what remains is to then build and deploy the SAM application.

To build I use the following build command of SAM CLI to build the project using a AWS CLI profile configured for my serverless-admin IAM user and destined for the US-East-2 Ohio region of the AWS Cloud.

sam build --profile serverless-admin --region us-east-2

Then to deploy I use the deploy SAM CLI command in guided mode. This is an interactive command that allows me to save my inputs for automated deployment parameters in subsequent deployments.

sam deploy --stack-name lol-api \
	--profile serverless-admin \
	--region us-east-2 \
	--guided

Now on subsequent changes of the template or code the same build and an abbreviated deploy command shall be used as shown below.

sam build --profile serverless-admin --region us-east-2
sam deploy

Testing with the HTTPie Client

One of the outputs from this build command will be the endpoint for the REST API which for me is https://93dl1r3gzk.execute-api.us-east-2.amazonaws.com/Prod/lists but, this endpoint will go away at then of this tutorial when I clean up the CloudFormation stack so please don't rely on it being available and instead create your own.

I can try out the REST API using the a CLI program such as HTTPie or a GUI program like PostMan as well as the API GateWay dashboard in the AWS Console.

Below I use the httpie program to generate some list data for a mock user represented by a bare Authorization token value representing a userId of 12345. Note that this is just a weird stop-gap implementation in the REST API that will be changed once the frontend app is built and integrated with AWS Cognito service will issue a suitable JWT token.

To create a list for mock user with id 12345.

http -j POST https://93dl1r3gzk.execute-api.us-east-2.amazonaws.com/Prod/lists \
  "Authorization:bearer 12345" \
  name=Groceries \
  items:='[{"name":"milk", "completed":false},{"name":"cookies", "completed":false}]'

Example output.

HTTP/1.1 201 Created
Connection: keep-alive
Content-Length: 167
Content-Type: application/json
Date: Wed, 02 Dec 2020 05:43:44 GMT
Via: 1.1 61729b32280fd6715c2a3b0dbb7e571a.cloudfront.net (CloudFront)
X-Amz-Cf-Id: pacrTjVome7UGBq8vQL6zdCKOk3APZSA2DBQj8pGnuh-OnSP757oVA==
X-Amz-Cf-Pop: SFO5-C1
X-Amzn-Trace-Id: Root=1-5fc72990-405ddd7e4e1444ca3eba9437;Sampled=0
X-Cache: Miss from cloudfront
x-amz-apigw-id: W6NuhFVQiYcFcYA=
x-amzn-Remapped-Content-Length: 167
x-amzn-RequestId: c16e1614-3702-4177-bd31-62348daeb570

{
    "items": [
        {
            "completed": false,
            "name": "milk"
        },
        {
            "completed": false,
            "name": "cookies"
        }
    ],
    "listId": "160471cf-67a6-4e8f-aefb-512100cfd0ab",
    "name": "Groceries",
    "userId": "12345"
}

To fetch lists for a mock user of id 12345.

http -j GET https://93dl1r3gzk.execute-api.us-east-2.amazonaws.com/Prod/lists "Authorization:bearer 12345"

Example output.

HTTP/1.1 200 OK
Connection: keep-alive
Content-Length: 169
Content-Type: application/json
Date: Wed, 02 Dec 2020 05:44:51 GMT
Via: 1.1 f1234553b388306d833e1a4591227882.cloudfront.net (CloudFront)
X-Amz-Cf-Id: KKQ5L6pfbyGHfZiOLPbcDFivED2Fw6X8BEGNfZdt1gNhZTCJCXeLzQ==
X-Amz-Cf-Pop: SFO5-C1
X-Amzn-Trace-Id: Root=1-5fc729d3-64b0bcdd073a6aa11fed2831;Sampled=0
X-Cache: Miss from cloudfront
x-amz-apigw-id: W6N5GHG-iYcFe-w=
x-amzn-Remapped-Content-Length: 169
x-amzn-RequestId: cc0423ed-0763-40aa-a35f-e5f190e8d19d

[
    {
        "items": [
            {
                "completed": false,
                "name": "milk"
            },
            {
                "completed": false,
                "name": "cookies"
            }
        ],
        "listId": "160471cf-67a6-4e8f-aefb-512100cfd0ab",
        "name": "Groceries",
        "userId": "12345"
    }
]

To update a list item the previous list, assuming it has an id of 160471cf-67a6-4e8f-aefb-512100cfd0ab.

list_id=160471cf-67a6-4e8f-aefb-512100cfd0ab
http -j PUT https://93dl1r3gzk.execute-api.us-east-2.amazonaws.com/Prod/lists/$list_id \
  "Authorization:bearer 12345" \
  items:='[{"name":"milk", "completed":true},{"name":"cookies", "completed":true}]'

Example output.

HTTP/1.1 200 OK
Connection: keep-alive
Content-Length: 3
Content-Type: application/json
Date: Wed, 02 Dec 2020 05:49:53 GMT
Via: 1.1 afb9be97319013ab1a18f338fce40f2a.cloudfront.net (CloudFront)
X-Amz-Cf-Id: ypQRoQKRsOq8qN184L7BwdxbgroiU_QkdtnLR6Vj8kpNqoxbNWBozg==
X-Amz-Cf-Pop: SFO5-C1
X-Amzn-Trace-Id: Root=1-5fc72aff-622f1960600fa6b21342b352;Sampled=0
X-Cache: Miss from cloudfront
x-amz-apigw-id: W6On9HYuiYcFd1A=
x-amzn-Remapped-Content-Length: 3
x-amzn-RequestId: 165ca484-1d96-4b72-b532-2828f066f531

{}

Similarly, to fetch a single list with id 160471cf-67a6-4e8f-aefb-512100cfd0ab for a mock user with id 12345.

list_id=160471cf-67a6-4e8f-aefb-512100cfd0ab
http -j GET https://93dl1r3gzk.execute-api.us-east-2.amazonaws.com/Prod/lists/$list_id \
  "Authorization:bearer 12345"

Example output.

HTTP/1.1 200 OK
Connection: keep-alive
Content-Length: 165
Content-Type: application/json
Date: Wed, 02 Dec 2020 05:50:32 GMT
Via: 1.1 89d40f9555bb19bc571952b32ca87399.cloudfront.net (CloudFront)
X-Amz-Cf-Id: yejrLrOX9uavz8beTyG8NyJt5qhWy46EGPDvbkIn_FDj_aNXE5MXBw==
X-Amz-Cf-Pop: SFO5-C1
X-Amzn-Trace-Id: Root=1-5fc72b28-57faa3394fde74a876e8fe0e;Sampled=0
X-Cache: Miss from cloudfront
x-amz-apigw-id: W6OuWHKgCYcFTJw=
x-amzn-Remapped-Content-Length: 165
x-amzn-RequestId: b08737c8-bebf-4ed8-a102-4262a55e6f00

{
    "items": [
        {
            "completed": true,
            "name": "milk"
        },
        {
            "completed": true,
            "name": "cookies"
        }
    ],
    "listId": "160471cf-67a6-4e8f-aefb-512100cfd0ab",
    "name": "Groceries",
    "userId": "12345"
}

Then to delete the same list item.

list_id=160471cf-67a6-4e8f-aefb-512100cfd0ab
http -j DELETE https://93dl1r3gzk.execute-api.us-east-2.amazonaws.com/Prod/lists/$list_id \
  "Authorization:bearer 12345"

Example output.

HTTP/1.1 200 OK
Connection: keep-alive
Content-Length: 3
Content-Type: application/json
Date: Wed, 02 Dec 2020 05:51:05 GMT
Via: 1.1 32814ee4b53f3642b74e20a0ba5944f7.cloudfront.net (CloudFront)
X-Amz-Cf-Id: FPL4JIZJw9Cwx4J_iY2uTUFtcKrISjHX_XdtUY7qi6IyGNZFu2L2Jg==
X-Amz-Cf-Pop: SFO5-C1
X-Amzn-Trace-Id: Root=1-5fc72b49-1ca904da3daadd3b4fb9b143;Sampled=0
X-Cache: Miss from cloudfront
x-amz-apigw-id: W6OzhFmbCYcF0SQ=
x-amzn-Remapped-Content-Length: 3
x-amzn-RequestId: a89553fc-41c9-4993-9e5c-e33307a620da

{}

Cleaning Up a SAM Project's Resources (aka Removing the CF Stack)

To tear down all of a SAM applications resources you can either delete it in the CloudFormation dashboard with the AWS Console or use the AWS CLI and it's cloudformation subcommand like so.

aws cloudformation delete-stack \
  --stack-name lol-api \
  --profile serverless-admin \
  --region us-east-2

Running SAM Project Locally to Shorten the Development Test Fix Feedback Loop

Often times the functionality that we are asked to build as developers is not quite so straight forward as the simple CRUD functionality of this sample app which simply shuffles data between request to the database and back. As complexity grows so does iterative cycle of develop a piece of code, run the piece of code in the system to test the change, fix or modify then repeat. Even though we have the ability to deploy to a dev or staging environment in the AWS cloud in a pretty simple and straight forward way it is often best to shorten the feedback loop and run all or portions of the system locally. Using SAM along with Docker we can do just that. 

For simple Lambda functions and REST APIs that do not interact with a database directly (ie, operates on the data its given or sources data from another API) the SAM CLI provides the following two commands.

# run a function by logical ID 
sam local invoke FunctionName

# run an API locally
sam local start-api

For the demo app I've presented in this tutorial I have a dependency for a DynamoDB table so running the SAM REST API locally requires a bit more setup work. I have to pull down and run the Amazon DynamoDB Local docker image which enables me to interact with a one or more DynamoDB tables locally without incuring costs and at a faster development pace.

Luckily the setup for doing this is fairly easy.

First I create a docker network to run the Amazon DynamoDB local docker image along with the SAM REST API project in as well.

docker network create dynamodb-local

 After that I fire up the Amazon DynamoDB docker image running it within the dynamodb-local network just created.

docker run --network dynamodb-local --name dynamodb -p 8000:8000 amazon/dynamodb-local

With the DynamoDB docker container running on my system I then create my lots-of-lists table in it as specified in the template.yaml template to be used for local running and development of my SAM project. For this I can use the AWS CLI or one of the available Higher Level Programming SDKs like boto3. For this project I use the AWS CLI and specify the --endpoint-url to be the localhost and port I forwarded traffic to when starting up the DynamoDB container.

aws dynamodb create-table --table-name lots-of-lists \
  --attribute-definitions AttributeName=userId,AttributeType=S AttributeName=listId,AttributeType=S \
  --key-schema AttributeName=userId,KeyType=HASH AttributeName=listId,KeyType=RANGE \
  --provisioned-throughput ReadCapacityUnits=2,WriteCapacityUnits=2 \
  --endpoint-url=http://localhost:8000

Then I can issue the following SAM CLI command to run the project locally within a docker container connected to the same network I created and am running the DynamoDB container on. The output of this command will indicate that the REST API is available on localhost port 3000

sam local start-api --parameter-overrides ExecEnv=local \
    --docker-network dynamodb-local

You'll notice the --parameter-overrides flag being used in the above command which sets the parameter ExecEnv to local effectively overriding the default value of prod specified in the template.yaml file. This value is then referenced in the LotsOfListsFunction as an environment variable EXEC_ENV and used to dictate which DynamoDB resource to connect to via the Boto3 SDK as shown in the code snippet below from app.py

import os
import uuid

import boto3
from boto3.dynamodb.conditions import Key

from flask import request, jsonify
from flask_lambda import FlaskLambda

EXEC_ENV = os.environ['EXEC_ENV']
REGION = os.environ['REGION_NAME']
TABLE_NAME = os.environ['TABLE_NAME']


app = FlaskLambda(__name__)

if EXEC_ENV == 'local':
    print('Configuring local dynamodb access')
    dynamodb = boto3.resource('dynamodb', endpoint_url='http://dynamodb:8000')
else:
    dynamodb = boto3.resource('dynamodb', region_name=REGION)

What remains is to hop over in yet another terminal I can again use the HTTPie client to test out the locally running SAM Flask REST API.

$ http -j POST http://localhost:3000/lists \
  "Authorization:bearer 12345" \
  name=Groceries \
  items:='[{"name":"milk", "completed":false},{"name":"cookies", "completed":false}]'
HTTP/1.0 201 CREATED
Content-Length: 167
Content-Type: application/json
Date: Thu, 03 Dec 2020 03:18:04 GMT
Server: Werkzeug/1.0.1 Python/3.8.6

{
    "items": [
        {
            "completed": false,
            "name": "milk"
        },
        {
            "completed": false,
            "name": "cookies"
        }
    ],
    "listId": "72dddc16-0543-4489-83e9-1c8071800936",
    "name": "Groceries",
    "userId": "12345"
}

Resources to Learn More About Serverless Architecture and Computing

thecodinginterface.com earns commision from sales of linked products such as the book suggestions above. This enables providing high quality, frequent, and most importantly FREE tutorials and content for readers interested in Software Engineering so, thank you for supporting the authors of these resources as well as thecodinginterface.com

Conclusion

In this article I have demonstrated the ease and simplicity of using the AWS Serverless Application Model framework for developing a serverless Python based Flask REST API. In articles to follow I will be rounding out the Lots of Lists application by similarly demonstrating how to build and deploy a complete React web application capable of registering and authenticating users with the AWS Cognito service as well as utilize the REST API built in this article.

As always, thanks for reading and please do not hesitate to critique or comment below.

Share with friends and colleagues

[[ likes ]] likes

Community favorites for Serverless

theCodingInterface