Is the training Qualiopi certified?

Yes, Learni is a Qualiopi-certified training organization. This certification guarantees the quality of our courses and enables funding through OPCO and other mechanisms.

How can I fund my training?

Our courses are eligible for OPCO funding. Learni supports you through the funding process. A personalized quote is available upon request.

What are the prerequisites for this training?

The prerequisites for the Training TensorRT-LLM - Optimizing LLM Inference in Production training are: Expertise in PyTorch/TensorFlow, CUDA programming, and deployment of LLM models such as Llama or GPT. A preliminary interview validates your eligibility.

Is the training available in person?

Yes, the Training TensorRT-LLM - Optimizing LLM Inference in Production training is available in distanciel. The format can be adapted to your needs (inter, intra, custom).

Training TensorRT-LLM - Optimizing LLM Inference… | Learni

The story of Learni

Founded by passionate advocates of learning and innovation, Learni set out to make professional training accessible to everyone, everywhere in the world. Our team works in the largest cities such as Paris, Lyon, Marseille, and internationally, to support talents and organizations in their skills development.

Configure your training

Response within 24hNo commitment100% free

9 spots remaining for the next session

FormatParticipantsDateDetails

Which format do you prefer?

Methods, approaches, and teaching resources

The Training TensorRT-LLM - Optimizing LLM Inference in Production training is delivered in-person or remotely (blended-learning, e-learning, virtual classroom, remote in-person). At Learni, a Qualiopi-certified training organization, each program is designed to maximize skills acquisition, regardless of the training mode chosen.

The trainer alternates between demonstrative, interrogative, and active methods (through practical exercises and/or real-world scenarios). This pedagogical approach ensures concrete and directly applicable learning in the workplace.

Teaching resources provided

To ensure the quality of the Training TensorRT-LLM - Optimizing LLM Inference in Production training, Learni provides the following teaching resources:

Mac or PC computers, high-speed fiber internet, whiteboard or flipchart, video projector or interactive touchscreen (for remote sessions)
Training environments installed on workstations or accessible online
Course materials, practical exercises, and supplementary resources
Post-training access to materials and teaching resources

For in-house training at a location external to Learni, the client ensures and commits to having all necessary teaching materials (IT equipment, internet connection...) for the proper conduct of the training action in accordance with the prerequisites indicated in the communicated training program.

* contact us for remote feasibility** ratio varies depending on the training followed

Assessment methods

The assessment of skills acquired during the Training TensorRT-LLM - Optimizing LLM Inference in Production training is carried out through:

During training: case studies, practical exercises, and professional scenarios
At the end of training: self-assessment questionnaire and skills evaluation by the trainer
After training: training completion certificate detailing acquired skills

Training accessibility

Learni is committed to the accessibility of its professional training programs. All our training programs are accessible to people with disabilities. Our teams are available to adapt teaching methods to your specific needs. Do not hesitate to contact us for any accommodation request.

Training objectives

Master TensorRT-LLM to accelerate LLM inference in professional production environments

Develop optimized TensorRT engines for certifiable enterprise competencies

Implement quantization techniques and layer fusion with TensorRT-LLM

Design scalable and high-performance inference pipelines for AI applications

Optimize GPU performance with custom kernels and advanced profiling

Deploy TensorRT-LLM solutions integrated with Triton Inference Server

Evaluate and tune models to minimize latency and costs in enterprise environments

Training program

Module 1TensorRT-LLM Fundamentals: Installation and Engine Building (NVIDIA Docker, HuggingFace)

Complete installation of the TensorRT-LLM environment on NVIDIA GPU, hands-on with Docker and CUDA Toolkit tools to create a first engine from a HuggingFace model like Llama-2, practical exercises on HF to TensorRT conversion, verification of initial performance with integrated benchmarks, generation of latency and throughput reports, setup of an ongoing enterprise LLM project to measure immediate gains.

Module 2Advanced TensorRT-LLM Optimization: Quantization and Fusion (FP8, GEMM plugins)

Exploration of INT4/INT8/FP8 quantization techniques in TensorRT-LLM, automatic fusion of attention and MLP layers to reduce GPU memory, development of custom GEMM plugins to accelerate matrix calculations, practical cases on models like Mistral with hyperparameter tuning, profiling with Nsight Systems to identify bottlenecks, production of an optimized engine delivering up to 3x speedup, documentation of technical choices for production replication.

Module 3TensorRT-LLM Deployment: Integration and Scaling (Triton Server, KV cache)

Integration of TensorRT-LLM with NVIDIA Triton Inference Server for multi-model deployments, management of KV cache for long-contextual streaming inferences, configuration of dynamic batching and tensor parallelism on GPU clusters, exercises on horizontal scaling with Kubernetes, load testing with Locust to simulate 1000+ requests/second, resolution of real enterprise error cases, deployment of a secure production-ready REST API service.

Module 4TensorRT-LLM Performance Tuning: Profiling and Real Cases (Nsight, multi-GPU)

In-depth performance analysis with NVIDIA Nsight Compute and DCGM to tune TensorRT-LLM kernels, multi-GPU optimization with tensor and pipeline parallelism on 70B parameter LLMs, resolution of challenges like Out-Of-Memory via intelligent paging and swapping, workshops on concrete enterprise cases like scalable chatbots, finalization of the ongoing project with live Q&A, evaluation of final gains (latency <50ms, 70% cost reduction), delivery of reusable certifiable templates.