Loading...

CodeCrafters

Catalog Pricing Roadmap

Build your own Kafka

Learn about TCP servers, the Kafka wire protocol and more.

kafka

kafka

kafka

kafka

kafka

kafka

kafka

kafka

kafka

kafka

kafka

Build your own Kafka

This challenge requires a CodeCrafters Membership.

You're welcome to study the full challenge structure and stage instructions for free. A membership is needed for submitting code and viewing solutions.

Want to try a free challenge? Try Build your own Shell — it's free this month.

Try Build your own Shell → View Membership Plans →

Apache Kafka is a distributed event streaming platform often used for high-performance data pipelines. In this challenge, you'll build your own Kafka broker that's capable of serving basic requests.

Along the way you'll learn about TCP servers, the Kafka wire protocol and more.

Stages

Send Correlation ID

Parse Correlation ID

Parse API Version

Handle ApiVersions requests

Concurrent Clients

Serial requests

Concurrent requests

Listing Partitions

Include DescribeTopicPartitions in ApiVersions

List for an unknown topic

List for a single partition

List for multiple partitions

List for multiple topics

Consuming Messages

Include Fetch in ApiVersions

Fetch with no topics

Fetch with an unknown topic

Fetch with an empty topic

Fetch single message from disk

Fetch multiple messages from disk

Producing Messages

Include Produce in ApiVersions

Respond for invalid topic or partition

Respond for valid topic and partition

Produce a single record

Produce multiple records

Produce to multiple partitions

Produce to multiple partitions of multiple topics

RECENT ATTEMPTS 0

“ There are few sites I like as much that have a step by step guide. The real-time feedback is so good, it's creepy!

Ananthalakshmi Sankar Automation Engineer at Apple “ I think the instant feedback right there in the git push is really cool. Didn't even know that was possible!

Patrick Burris Senior Software Developer, CenturyLink