Invoking Lambda with Kinesis locally with AWS SAM

SAM (AWS Serverless Application Model) is an open-source framework build on top of AWS infrastructure as code (IaC) service CloudFormation specific for serverless applications. One of its advantages is providing a way to run your serverless applications locally for debugging and testing without charges. In this tutorial I will show how to use SAM to invoke a Lambda function locally with a custom Kinesis Data Stream event.

SAM Template

SAM utilizes a template file named template.yaml to define the application resources. Here is a basic template with a Lambda function.

SAM template

In this template I’m specifying a MyLambda resource of type AWS::Serverless::Function which references a Lambda function. This resource has many properties that define its behavior. In this case a main file on the bin path written in Go with 128 Mb of memory.

Lambda Function Code

Lambda code

The above code is the Lambda function used as example in this tutorial. It basically receives a Kinesis event and print the record data to the console. Full repository here.

Run Lambda locally

To run a Lambda function defined on a SAM template locally install SAM and use the following command.

sam local invoke MyLambda

This command invokes the function without an event. To add an event trigger you need to specify the path to a JSON file with the event to the --event (-e) parameter.

sam local invoke MyLambda -e event.json

Each Lambda function event source (Kinesis, SQS, SNS…) has its own structure. You can check the documentation to find out the structure of the desired event source or generate a sample event.

Generating Events

To generate a event structure use the command sam local generate-event. In this tutorial we’ll use a Kinesis Data Strem event.

sam local generate-event kinesis get-records

Here is the output of the above command:

Output of generate-event command

This event can contain many Records where the content is stored in the data property as a base64-encoded string (https://docs.aws.amazon.com/kinesis/latest/APIReference/API_Record.html). This output can be directly passed to the invoke function.

sam local generate-event kinesis get-records | sam local invoke -e — MyLambda

The -e value - inputs JSON via stdin

Bellow is the invocation output in the console showing the default data decoded value “Hello, this is a test 123.”.

Lambda invocation with default data

Generating Events with Custom Data

You can change the data parameter in the JSON file (using base64 string) and call sam local invoke again but it is easier to generate a new event with custom data using the --data parameter which receives a normal text.

sam local generate-event kinesis get-records --data “My custom content”

Generally the data is a JSON string but using this format in the command line is not productive. My solution is to minify and stringify the content of the JSON event file using the popular command line JSON processor jq and pass the output to the --data parameter.

sam local generate-event kinesis get-records --data “$(jq -c < event.json)” | sam local invoke -e - MyLambda

This way you can quickly make changes in the JSON file and rerun the command with the updated event data. Bellow is the invocation output in the console showing the data decoded value as a JSON string.

Lambda invocation with custom JSON data

Extra

A common practice with Go applications is to use Makefile which is a file that contains a set of tasks that can be executed. I like to add a invoke target to invoke the Lambda function locally with the latest compiled version. This way you can simply run make invoke.

Makefile code

Conclusion

You can use SAM to test and debug serverless applications locally without charges and quickly generate custom integration data for resources.