Is the training Qualiopi certified?

Yes, Learni is a Qualiopi-certified training organization. This certification guarantees the quality of our courses and enables funding through OPCO and other mechanisms.

How can I fund my training?

Our courses are eligible for OPCO funding. Learni supports you through the funding process. A personalized quote is available upon request.

What are the prerequisites for this training?

The prerequisites for the Training Dataflow - Mastering Scalable Data Pipelines training are: Knowledge of Java or Python, basics of Google Cloud Platform and Apache Beam. A preliminary interview validates your eligibility.

Is the training available in person?

Yes, the Training Dataflow - Mastering Scalable Data Pipelines training is available in distanciel. The format can be adapted to your needs (inter, intra, custom).

Training Dataflow - Mastering Scalable Data Pipe... | Learni

The story of Learni

Founded by passionate advocates of learning and innovation, Learni set out to make professional training accessible to everyone, everywhere in the world. Our team works in the largest cities such as Paris, Lyon, Marseille, and internationally, to support talents and organizations in their skills development.

Configure your training

Response within 24hNo commitment100% free

9 spots remaining for the next session

FormatParticipantsDateDetails

Which format do you prefer?

Methods, approaches, and teaching resources

The Training Dataflow - Mastering Scalable Data Pipelines training is delivered in-person or remotely (blended-learning, e-learning, virtual classroom, remote in-person). At Learni, a Qualiopi-certified training organization, each program is designed to maximize skills acquisition, regardless of the training mode chosen.

The trainer alternates between demonstrative, interrogative, and active methods (through practical exercises and/or real-world scenarios). This pedagogical approach ensures concrete and directly applicable learning in the workplace.

Teaching resources provided

To ensure the quality of the Training Dataflow - Mastering Scalable Data Pipelines training, Learni provides the following teaching resources:

Mac or PC computers, high-speed fiber internet, whiteboard or flipchart, video projector or interactive touchscreen (for remote sessions)
Training environments installed on workstations or accessible online
Course materials, practical exercises, and supplementary resources
Post-training access to materials and teaching resources

For in-house training at a location external to Learni, the client ensures and commits to having all necessary teaching materials (IT equipment, internet connection...) for the proper conduct of the training action in accordance with the prerequisites indicated in the communicated training program.

* contact us for remote feasibility** ratio varies depending on the training followed

Assessment methods

The assessment of skills acquired during the Training Dataflow - Mastering Scalable Data Pipelines training is carried out through:

During training: case studies, practical exercises, and professional scenarios
At the end of training: self-assessment questionnaire and skills evaluation by the trainer
After training: training completion certificate detailing acquired skills

Training accessibility

Learni is committed to the accessibility of its professional training programs. All our training programs are accessible to people with disabilities. Our teams are available to adapt teaching methods to your specific needs. Do not hesitate to contact us for any accommodation request.

Training access terms and deadlines

Training objectives

Master Dataflow pipelines for professional batch and streaming processing

Develop efficient data transformations with Apache Beam

Implement scalable pipelines connected to Pub/Sub and BigQuery

Optimize the performance and costs of Dataflow jobs in enterprise settings

Design reusable templates for certifiable competencies

Deploy and monitor Dataflow applications in production

Training program

Module 1Dataflow Fundamentals: Batch Pipelines with Apache Beam (Java/Python, SDK, GCP Execution)

Installation of the Dataflow environment and creation of a first professional batch pipeline, exploration of Beam concepts like PCollection and DoFn, development of simple transformations on real enterprise datasets, execution of jobs on Google Cloud with initial log analysis, practical exercises to transform CSV files into BigQuery, production of a functional deliverable with code review by the trainer.

Module 2Dataflow Streaming: Real-Time Processing (Pub/Sub, Windowing, Source Integrations)

Configuration of streaming sources via Pub/Sub and Kafka for continuous flows, implementation of time windows and advanced aggregations in Beam, real-time data processing with joins and custom states, simulation of business use cases like IoT log analysis, interactive debugging with Dataflow Runner, realization of a complete pipeline ingesting events into BigQuery for immediate insights.

Module 3Dataflow Optimization: Performance and Costs (Fusion, Sampling, Stackdriver Monitoring)

Analysis of Dataflow metrics to identify bottlenecks, optimization techniques such as transformation fusion and adaptive sampling, GCP cost reduction up to 40% via autoscaling and machine types, advanced monitoring with Stackdriver and custom alerts, exercises on a large enterprise dataset, production of a quantified optimization report proving gains in efficiency and scalability.

Module 4Advanced Dataflow: Templates and Deployments (CI/CD, Side Inputs, Streaming Graphs)

Design of reusable Dataflow templates for quick team deployments, integration of side inputs and complex streaming graphs, automation via Cloud Build for professional CI/CD, unit tests and continuous integration on business pipelines, production deployment with high availability, finalization of the capstone project with presentation and action plan for the enterprise.