Deploy ETL Data Pipelines in Amazon Web Services using Azure DevOps

Discover streamlined ETL pipeline deployment in AWS using Azure DevOps, ensuring reliability and efficiency for data-driven decision-making.

Multi-account Deployment of An Open-Source Vector Database on AWS 

Learn how deploying Weaviate on AWS EKS across multiple accounts enhances security, control and innovation while safeguarding operations.

Deploying an Open-Source Vector Database on AWS – Part 2

In this lab, we will create a Kafka producer application using AWS Lambda, which sends fake taxi ride data into a Kafka topic on Amazon MSK. A configurable number of the producer Lambda function will be invoked by an Amazon EventBridge schedule rule. Therefore, we are able to generate test data concurrently based on the desired volume of messages.

Deploying an Open-Source Vector Database on AWS – Part 1

In this lab, we will create a Kafka producer application using AWS Lambda, which sends fake taxi ride data into a Kafka topic on Amazon MSK. A configurable number of the producer Lambda function will be invoked by an Amazon EventBridge schedule rule. Therefore, we are able to generate test data concurrently based on the desired volume of messages.

Real Time Streaming with Kafka and Flink – Lab 1 Produce data to Kafka using Lambda

In this lab, we will create a Kafka producer application using AWS Lambda, which sends fake taxi ride data into a Kafka topic on Amazon MSK. A configurable number of the producer Lambda function will be invoked by an Amazon EventBridge schedule rule. Therefore, we are able to generate test data concurrently based on the desired volume of messages.

Orchestrating Snowflake Data Transformations with DBT on Amazon ECS through Apache Airflow- Part 2

We explore how AWS CloudWatch math metrics provide a powerful way to derive insights and take automated actions based on custom calculations.

Orchestrating Snowflake Data Transformations with DBT on Amazon ECS through Apache Airflow- Part 1

We explore how AWS CloudWatch math metrics provide a powerful way to derive insights and take automated actions based on custom calculations.

Real Time Streaming with Kafka and Flink – Introduction

This series updates a real time analytics app based on Amazon Kinesis from an AWS workshop. Data is ingested from multiple sources into a Kafka cluster instead and Flink (Pyflink) apps are used extensively for data ingesting and processing. As an introduction, this post compares the original architecture with the new architecture, and the app will be implemented in subsequent posts.

Exploring The Power of Vector Databases (Part 2)

This blog explores how we can use vector databases to keep LLM knowledge up to date, minimise hallucinations and enhance user experiences.

Dynamic Table Usage in Snowflake: Implementing Type 2 Slowly Changing Dimensions (SCD) with Flexibility and Efficiency

We explore how AWS CloudWatch math metrics provide a powerful way to derive insights and take automated actions based on custom calculations.