Providing quality software engineering content in the form of tutorials, applications, services, and commentary suited for developers.
In this article I present an example of how one can use Kafka and the Confluent ksqlDB stream processing database to process a simplified dataset of fake stock quotes. The ultimate goal of this excercise will be to user ksqlDB to inspect a stream of stock quotes for individual companies in 1 minute windows and identify when a window has introduced a new daily high or low stock price.
In this How To article I demonstrate setting up a Docker Compose based implementation of the Community Components of the Confluent Platform complete with the kafka-connect-datagen plugin for Kafka Connect to generate test and/or developement data useful for working with Kafka.
Here I present an end-to-end example of a Serverless event driven architecture using Confluent Cloud for stream processing paired with AWS Lambda for event responsive logic using the Serverless Application Model (SAM) framework. Together this architecture will compose a system for fictitious financial stock quote email alerting.
In this article I go over how to use Apache Flink Table API in Python to consume data from and write data to a Confluent Community Platform Apache Kafka Cluster running locally in Docker.
In this article I present a minimal Java Gradle project that utilizes Apache Avro serialization and integrates with the Confluent Schema Registry for managing message data formats used by Apache Kafka producers and consumers.
In this article I show how to implement a data pipelining solution utilizing Dockerized Kafka Connect and the Snowflake Sink Connector to pull events out of Apache Kafka and into the Snowflake Cloud Data Warehouse.
In this article I capture a plethora of Kafka CLI commands that I've found useful. It is my goal to capture them here to serve as a cheatsheet of commands for myself and others to draw from.
In this article I review the concepts of Command Query Responsibility Segregation (CQRS) emphasizing the value and flexibility that this software architecture provides through indpendentent optimization of writing and reading data access patterns.
Schemas and schema registries are the governance tools that ensure vitality of data communications through configurable compatibility guardrails as patterns and rules to evolve schemas in a controlled fashion. In this article I will demonstrate how Confluent Schema Registry can be used in conjunction with JSON Schema to govern and enforce rules for schema evolution in the default backwards compatibility mode.
By