KSQL is an easy-to-use streaming SQL engine for Apache Kafka built using Kafka Streams. The ability to write streaming pipelines with SQL makes Apache Kafka available to a whole range of new developers and potential use cases.
Inspired by a customer project now in production, experience the lifecycle of a streaming application developed using KSQL and Kafka Streams. With Apache Gradle as our build framework, we’ll explore the open-source Gradle plugin we built to improve developer efficiency and automate the deployment of KSQL pipelines. We’ll also see Gradle build and automate our KSQL user-defined functions and our Kafka Streams microservices, providing an end-to-end deployment framework for stream processing with Kafka. We’ll demonstrate a full deployment live, and discuss design decisions around incorporating SQL-based processes into an overall streaming application.