What is Lambda Event Filtering?
At AWS re: Invent 2021, AWS announced the ability to filter events when creating event source mapping for SQS, Kinesis & DynamoDB streams.
For example, if you have a Lambda function triggered by the Kinesis stream, you can define filter criteria to only process events with a VALID status.
How does Event Filtering work in AWS Lambda?
When an event occurs at the event source, such as a new message coming through into SQS, AWS Lambda picks up this event data and passes it to the Lambda function.
However, in the case where filtering is enabled, the event is first filtered based on the criteria defined in the event source mapping. Only events with matching criteria and passed to the Lambda function.
Why is it important?
Customers running production workloads, or wanting to reduce overall costs by processing only what is needed, can access a solution that requires zero downtime and little to no code changes of the Lambda function.
How can I implement event filtering?
There are three ways to do this, which are:
- Attaching filter criteria to an event source using the AWS console
- Using the AWS CLI
- Using SAM (Serverless Application Model)
You can also build filters with other IaC tooling too such as the AWS SDK or Pulumi.
In the example below, a SAM template is built out with support for event filtering.
You can read more AWS documentation on how to set up using other ways here.
How do I write a filter?
Understanding the filter rule syntax is key to writing correct filters. You can have up to five filters set up per event source.
For example, if you wanted to filter all events of type ‘group’ to be processed by the Lambda function, you could write the filter like so.
Filter Criteria
It is important to know that Lambda uses exact character-by-character matching without case-folding or any other string normalisation. For numbers, Lambda also uses string representation. For example, 300, 300.0, and 3.0e2 are not considered equal.
Filter rule syntax
Use the filter rule syntax to set up your filters to parse through an event to allow rules to govern whether the event is to be filtered or not.
Example 1: Filter all events with a UserID = null
Example 2: Filter all events where the card type is credit or debit
Pro Tip
The event filter rule syntax supported by Lambda is the same as it is for the Amazon EventBridge.
Conclusion
Event filtering is a powerful feature of AWS Lambda that allows you to select and process only the events that are relevant to your functions. It can help keep any filtering logic out of your Lambda function(s), and potentially help with reducing your overall Lambda costs.
