Apache Kafka for Python Programmers Training Course
Apache Kafka is an open-source stream-processing platform that provides a fast, reliable, and low-latency platform for handling real-time data analytics. Apache Kafka can be integrated with available programming languages such as Python.
This instructor-led, live training (online or onsite) is aimed at data engineers, data scientists, and programmers who wish to use Apache Kafka features in data streaming with Python.
By the end of this training, participants will be able to use Apache Kafka to monitor and manage conditions in continuous data streams using Python programming.
Format of the Course
- Interactive lecture and discussion.
- Lots of exercises and practice.
- Hands-on implementation in a live-lab environment.
Course Customization Options
- To request a customized training for this course, please contact us to arrange.
Course Outline
Introduction
Overview of Apache Kafka Features and Architecture for Python
- Core APIs (producer, consumer, streams, connector)
- Concepts and uses
Accessing Kafka in Python
- Available Python libraries for use
- Compression formats supported
Installing Apache Kafka
- Computer installation
- Virtual private server and virtual machine installation
Starting Kafka Broker Server
- Reading and editing using an IDE (Integrated Development Environment)
- Running Zookeeper
- Logs folder
Creating a Kafka Topic
- Connecting to a Kafka cluster
- Reading topic details
Sending Messages Using Producers
- Initiating a producer
- Examining incoming messages
- Running multiple producers
Consuming Messages
- Kafka Console Consumer
- Running multiple consumers
Troubleshooting
Summary and Conclusion
Requirements
- Experience with Python programming language
- Familiarity with stream-processing platforms
Audience
- Data engineers
- Data scientists
- Programmers
Open Training Courses require 5+ participants.
Apache Kafka for Python Programmers Training Course - Booking
Apache Kafka for Python Programmers Training Course - Enquiry
Apache Kafka for Python Programmers - Consultancy Enquiry
Consultancy Enquiry
Testimonials (5)
Sufficient hands on, trainer is knowledgable
Chris Tan
Course - A Practical Introduction to Stream Processing
During the exercises, James explained me every step whereever I was getting stuck in more detail. I was completely new to NIFI. He explained the actual purpose of NIFI, even the basics such as open source. He covered every concept of Nifi starting from Beginner Level to Developer Level.
Firdous Hashim Ali - MOD A BLOCK
Course - Apache NiFi for Administrators
The course was excellent. Our trainer Andreas was very prepared and answered all the questions that we asked. Also he helped us when we have troubles and explained in details when needed. The best course that i have ever been part of.
Bozhidar Marinov - Pejsejf B"lgaria EOOD
Course - Microservices with Spring Cloud and Kafka
That I had it in the first place.
Peter Scales - CACI Ltd
Course - Apache NiFi for Developers
Recalling/reviewing keypoints of the topics discussed.
Paolo Angelo Gaton - SMS Global Technologies Inc.
Course - Building Stream Processing Applications with Kafka Streams
Upcoming Courses
Related Courses
Unified Batch and Stream Processing with Apache Beam
14 HoursApache Beam is an open source, unified programming model for defining and executing parallel data processing pipelines. It's power lies in its ability to run both batch and streaming pipelines, with execution being carried out by one of Beam's supported distributed processing back-ends: Apache Apex, Apache Flink, Apache Spark, and Google Cloud Dataflow. Apache Beam is useful for ETL (Extract, Transform, and Load) tasks such as moving data between different storage media and data sources, transforming data into a more desirable format, and loading data onto a new system.
In this instructor-led, live training (onsite or remote), participants will learn how to implement the Apache Beam SDKs in a Java or Python application that defines a data processing pipeline for decomposing a big data set into smaller chunks for independent, parallel processing.
By the end of this training, participants will be able to:
- Install and configure Apache Beam.
- Use a single programming model to carry out both batch and stream processing from withing their Java or Python application.
- Execute pipelines across multiple environments.
Format of the Course
- Part lecture, part discussion, exercises and heavy hands-on practice
Note
- This course will be available Scala in the future. Please contact us to arrange.
Building Kafka Solutions with Confluent
14 HoursThis instructor-led, live training (online or onsite) is aimed at engineers who wish to use Confluent (a distribution of Kafka) to build and manage a real-time data processing platform for their applications.
By the end of this training, participants will be able to:
- Install and configure Confluent Platform.
- Use Confluent's management tools and services to run Kafka more easily.
- Store and process incoming stream data.
- Optimize and manage Kafka clusters.
- Secure data streams.
Format of the Course
- Interactive lecture and discussion.
- Lots of exercises and practice.
- Hands-on implementation in a live-lab environment.
Course Customization Options
- This course is based on the open source version of Confluent: Confluent Open Source.
- To request a customized training for this course, please contact us to arrange.
Building Data Pipelines with Apache Kafka
7 HoursApache Kafka is a distributed streaming platform. It is de facto a standard for building data pipelines and it solves a lot of different use-cases around data processing: it can be used as a message queue, distributed log, stream processor, etc.
We'll start with some theory behind data pipelines in general, then continue with fundamental concepts behind Kafka. We'll also discover important components like Kafka Streams and Kafka Connect.
Apache Flink Fundamentals
28 HoursThis instructor-led, live training in Guatemala (online or onsite) introduces the principles and approaches behind distributed stream and batch data processing, and walks participants through the creation of a real-time, data streaming application in Apache Flink.
By the end of this training, participants will be able to:
- Set up an environment for developing data analysis applications.
- Understand how Apache Flink's graph-processing library (Gelly) works.
- Package, execute, and monitor Flink-based, fault-tolerant, data streaming applications.
- Manage diverse workloads.
- Perform advanced analytics.
- Set up a multi-node Flink cluster.
- Measure and optimize performance.
- Integrate Flink with different Big Data systems.
- Compare Flink capabilities with those of other big data processing frameworks.
A Practical Introduction to Stream Processing
21 HoursIn this instructor-led, live training in Guatemala (onsite or remote), participants will learn how to set up and integrate different Stream Processing frameworks with existing big data storage systems and related software applications and microservices.
By the end of this training, participants will be able to:
- Install and configure different Stream Processing frameworks, such as Spark Streaming and Kafka Streaming.
- Understand and select the most appropriate framework for the job.
- Process of data continuously, concurrently, and in a record-by-record fashion.
- Integrate Stream Processing solutions with existing databases, data warehouses, data lakes, etc.
- Integrate the most appropriate stream processing library with enterprise applications and microservices.
Distributed Messaging with Apache Kafka
14 HoursThis course is for enterprise architects, developers, system administrators and anyone who wants to understand and use a high-throughput distributed messaging system. If you have more specific requirements (e.g. only system administration side), this course can be tailored to better suit your needs.
Stream Processing with Kafka Streams
7 HoursKafka Streams is a client-side library for building applications and microservices whose data is passed to and from a Kafka messaging system. Traditionally, Apache Kafka has relied on Apache Spark or Apache Storm to process data between message producers and consumers. By calling the Kafka Streams API from within an application, data can be processed directly within Kafka, bypassing the need for sending the data to a separate cluster for processing.
In this instructor-led, live training, participants will learn how to integrate Kafka Streams into a set of sample Java applications that pass data to and from Apache Kafka for stream processing.
By the end of this training, participants will be able to:
- Understand Kafka Streams features and advantages over other stream processing frameworks
- Process stream data directly within a Kafka cluster
- Write a Java or Scala application or microservice that integrates with Kafka and Kafka Streams
- Write concise code that transforms input Kafka topics into output Kafka topics
- Build, package and deploy the application
Audience
- Developers
Format of the course
- Part lecture, part discussion, exercises and heavy hands-on practice
Notes
- To request a customized training for this course, please contact us to arrange
Confluent KSQL
7 HoursThis instructor-led, live training in Guatemala (online or onsite) is aimed at developers who wish to implement Apache Kafka stream processing without writing code.
By the end of this training, participants will be able to:
- Install and configure Confluent KSQL.
- Set up a stream processing pipeline using only SQL commands (no Java or Python coding).
- Carry out data filtering, transformations, aggregations, joins, windowing, and sessionization entirely in SQL.
- Design and deploy interactive, continuous queries for streaming ETL and real-time analytics.
Apache NiFi for Administrators
21 HoursIn this instructor-led, live training in Guatemala (onsite or remote), participants will learn how to deploy and manage Apache NiFi in a live lab environment.
By the end of this training, participants will be able to:
- Install and configure Apachi NiFi.
- Source, transform and manage data from disparate, distributed data sources, including databases and big data lakes.
- Automate dataflows.
- Enable streaming analytics.
- Apply various approaches for data ingestion.
- Transform Big Data and into business insights.
Apache NiFi for Developers
7 HoursIn this instructor-led, live training in Guatemala, participants will learn the fundamentals of flow-based programming as they develop a number of demo extensions, components and processors using Apache NiFi.
By the end of this training, participants will be able to:
- Understand NiFi's architecture and dataflow concepts.
- Develop extensions using NiFi and third-party APIs.
- Custom develop their own Apache Nifi processor.
- Ingest and process real-time data from disparate and uncommon file formats and data sources.
Spark Streaming with Python and Kafka
7 HoursThis instructor-led, live training in Guatemala (online or onsite) is aimed at data engineers, data scientists, and programmers who wish to use Spark Streaming features in processing and analyzing real-time data.
By the end of this training, participants will be able to use Spark Streaming to process live data streams for use in databases, filesystems, and live dashboards.
Microservices with Spring Cloud and Kafka
21 HoursThis instructor-led, live training in Guatemala (online or onsite) is aimed at developers who wish to transform traditional architecture into a highly concurrent microservices-based architecture using Spring Cloud, Kafka, Docker, Kubernetes and Redis.
By the end of this training, participants will be able to:
- Set up the necessary development environment for building microservices.
- Design and implement a highly concurrent microservices ecosystem using Spring Cloud, Kafka, Redis, Docker and Kubernetes.
- Transform monolithic and SOA services to microservice based architecture.
- Adopt a DevOps approach to developing, testing and releasing software.
- Ensure high concurrency among microservices in production.
- Monitor microservices and implement recovery strategies.
- Carry out performance tuning.
- Learn about future trends in microservices architecture.