Kafka SQL or KSQL is a streaming SQL engine that provides SQL interface to the streams in Apache Kafka. It is developed by Confluent Inc. and is built on the Kafka Streams API. As opposed to systems like Spark Streaming which require using Java/Scala for development, KSQL provides a completely interactive, SQL only interface improving the ease of access.[03][01]
- Source Code
- https://github.com/confluentinc/ksql[02]
- Developer
- Country of Origin
- US
- Start Year
- 2017 [04]
- Project Type
- Open Source
- Written in
- Java
- Supported Languages
- SQL
- Compatible With
- Spark SQL
Kafka SQL or KSQL is a streaming SQL engine that provides SQL interface to the streams in Apache Kafka. It is developed by Confluent Inc. and is built on the Kafka Streams API. As opposed to systems like Spark Streaming which require using Java/Scala for development, KSQL provides a completely interactive, SQL only interface improving the ease of access.[03][01]
Data Model[03]
The basic unit of storage in Kafka is a message, which consists of a key, value, timestamp, partition number and its offset in the partition. Key and value are just arrays of bytes, hence there is no restriction on the type of values they can hold. A schema can be associated with each topic, which is imposed upon the value part of the message.
ToDo: Verify who validates the schema i.e. can schema be applied on top of an existing Kafka topic.
Citations
4 sources- Database Streaming with ksqlDB | Confluent confluent.io
- GitHub - confluentinc/ksql: The database purpose-built for stream processing applications. · GitHub github.com
- https://openproceedings.org/2019/conf/edbt/EDBT19_paper_329.pdf openproceedings.org
- Introducing KSQL: Streaming SQL for Apache Kafka | Confluent confluent.io