Course Overview

This Kafka Introduction course prepares Developers and Architects in applying Kafka to their integrations and micro-services.

Course Prerequisites

Students should be familiar with Java or Scala. Familiarity with traditional messaging systems (ActiveMQ, RabbitMQ, …) is helpful, through not required.

Outline

Introduction

• Introduction to Event Streaming

• Discuss use cases for Kafka

• Introduce Apache Kafka

• Introduce Event Sourcing

• Understand the concept of journaling

• Understand Kafka's Architecture

• Push vs Pull architectures

• Introduce main concepts (event, producers, partitions, topics, brokers ,logs )

• Compare Kafka to MQ/JMS (traditional Message Oriented Middleware)

• Discuss alternatives

• Overview of the different APIs in Kafka

Installing & Running Kafka Cluster

• Planning hardware for kafka
• Explore the installation directory
• Understand the current role of Apache Zookeeper
• Configure kafka brokers
• Define producer/consumer quotas
• Discuss best practices in cluster size
• Partitions and brokers
• Building a kafka cluster

Producing Messages

• Producing messages using the Producer API
• Configure producers (latency, throughput, timeouts)
• Appreciate the batching of messages
• Configure acknowledgments
• Use message compression
• Build resilient producers
• Idempotent messages and exactly-once delivery semantics
• Using ProducerInterceptor

Messages (Partitioning, Serialization, …)

• Recap message structure
• Using message headers
• List available Serializers/Deserializers (e.g., Protobuf)
• Defining extensible schemas using Apache Avro
• Planning schemas for change
• Using Schema Registries with Avro
• Building Custom Serializers/Deserializers
• Using different Partitioning strategies (e.g., key-based, round-robin, …)
• Building a custom Partitioner
• Introduction to Kafka Connect

Consuming Messages

• Producing messages using the Consumer API
• Determine the off-set to control the Consumer Position
• Partition-based scaling using consumer groups
• Configuring PartitionAssignor strategies
• Introduce Static membership
• Understand semantic guarantees (At most/least/exactly once)
• Managing the offset (e.g., committing)
• Partition ownership and rebalancing
• Using ConsumerRebalanceListener

Basic Stream Processing

• Introduce Kafka Streams
• Introduce the Streams API
• Stream architecture (topology, source and sink processors)
• Understand key concepts streams and tables
• Time and windowing
• Discuss different ****use-cases and patterns

Frequently asked questions