Deploying an Open-Source Vector Database on AWS – Part 2
In this lab, we will create a Kafka producer application using AWS Lambda, which sends fake taxi ride data into a Kafka topic on Amazon MSK. A configurable number of the producer Lambda function will be invoked by an Amazon EventBridge schedule rule. Therefore, we are able to generate test data concurrently based on the desired volume of messages.
Deploying an Open-Source Vector Database on AWS – Part 1
In this lab, we will create a Kafka producer application using AWS Lambda, which sends fake taxi ride data into a Kafka topic on Amazon MSK. A configurable number of the producer Lambda function will be invoked by an Amazon EventBridge schedule rule. Therefore, we are able to generate test data concurrently based on the desired volume of messages.
Real Time Streaming with Kafka and Flink – Lab 1 Produce data to Kafka using Lambda
In this lab, we will create a Kafka producer application using AWS Lambda, which sends fake taxi ride data into a Kafka topic on Amazon MSK. A configurable number of the producer Lambda function will be invoked by an Amazon EventBridge schedule rule. Therefore, we are able to generate test data concurrently based on the desired volume of messages.
Orchestrating Snowflake Data Transformations with DBT on Amazon ECS through Apache Airflow- Part 2
We explore how AWS CloudWatch math metrics provide a powerful way to derive insights and take automated actions based on custom calculations.
Orchestrating Snowflake Data Transformations with DBT on Amazon ECS through Apache Airflow- Part 1
We explore how AWS CloudWatch math metrics provide a powerful way to derive insights and take automated actions based on custom calculations.
Real Time Streaming with Kafka and Flink – Introduction
This series updates a real time analytics app based on Amazon Kinesis from an AWS workshop. Data is ingested from multiple sources into a Kafka cluster instead and Flink (Pyflink) apps are used extensively for data ingesting and processing. As an introduction, this post compares the original architecture with the new architecture, and the app will be implemented in subsequent posts.
Exploring The Power of Vector Databases (Part 2)
This blog explores how we can use vector databases to keep LLM knowledge up to date, minimise hallucinations and enhance user experiences.
Dynamic Table Usage in Snowflake: Implementing Type 2 Slowly Changing Dimensions (SCD) with Flexibility and Efficiency
We explore how AWS CloudWatch math metrics provide a powerful way to derive insights and take automated actions based on custom calculations.
Vector Databases: The What, The How and The Why
In this blog, we explore the transformative power of modern data solutions, and the benefits they can provide organisations.
Kafka, Flink and DynamoDB for Real Time Fraud Detection – Part 2 Deployment via AWS Managed Flink
This series re-implements a simple fraud detection application that is discussed in an AWS workshop titled AWS Kafka and DynamoDB for real time fraud detection. In part 1, I demonstrated how to develop the application locally, and the app will be deployed via Amazon Managed Service for Apache Flink in this post.
