Is the training Qualiopi certified?

Yes, Learni is a Qualiopi-certified training organization. This certification guarantees the quality of our courses and enables funding through OPCO and other mechanisms.

How can I fund my training?

Our courses are eligible for OPCO funding. Learni supports you through the funding process. A personalized quote is available upon request.

What are the prerequisites for this training?

The prerequisites for the Training TensorRT-LLM - Accelerate LLM Inference in Production training are: Mastery of Python, PyTorch or TensorFlow, basics in LLMs and CUDA. A preliminary interview validates your eligibility.

Is the training available in person?

Yes, the Training TensorRT-LLM - Accelerate LLM Inference in Production training is available in distanciel. The format can be adapted to your needs (inter, intra, custom).

Training TensorRT-LLM - Accelerate LLM Inference... | Learni

The story of Learni

Founded by passionate advocates of learning and innovation, Learni set out to make professional training accessible to everyone, everywhere in the world. Our team works in the largest cities such as Paris, Lyon, Marseille, and internationally, to support talents and organizations in their skills development.

Configure your training

Response within 24hNo commitment100% free

9 spots remaining for the next session

FormatParticipantsDateDetails

Which format do you prefer?

Methods, approaches, and teaching resources

The Training TensorRT-LLM - Accelerate LLM Inference in Production training is delivered in-person or remotely (blended-learning, e-learning, virtual classroom, remote in-person). At Learni, a Qualiopi-certified training organization, each program is designed to maximize skills acquisition, regardless of the training mode chosen.

The trainer alternates between demonstrative, interrogative, and active methods (through practical exercises and/or real-world scenarios). This pedagogical approach ensures concrete and directly applicable learning in the workplace.

Teaching resources provided

To ensure the quality of the Training TensorRT-LLM - Accelerate LLM Inference in Production training, Learni provides the following teaching resources:

Mac or PC computers, high-speed fiber internet, whiteboard or flipchart, video projector or interactive touchscreen (for remote sessions)
Training environments installed on workstations or accessible online
Course materials, practical exercises, and supplementary resources
Post-training access to materials and teaching resources

For in-house training at a location external to Learni, the client ensures and commits to having all necessary teaching materials (IT equipment, internet connection...) for the proper conduct of the training action in accordance with the prerequisites indicated in the communicated training program.

* contact us for remote feasibility** ratio varies depending on the training followed

Assessment methods

The assessment of skills acquired during the Training TensorRT-LLM - Accelerate LLM Inference in Production training is carried out through:

During training: case studies, practical exercises, and professional scenarios
At the end of training: self-assessment questionnaire and skills evaluation by the trainer
After training: training completion certificate detailing acquired skills

Training accessibility

Learni is committed to the accessibility of its professional training programs. All our training programs are accessible to people with disabilities. Our teams are available to adapt teaching methods to your specific needs. Do not hesitate to contact us for any accommodation request.

Training objectives

Master TensorRT-LLM to optimize LLM inference in enterprise settings

Develop high-performance and scalable inference engines

Implement quantization techniques and layer fusion

Design TensorRT-LLM serving pipelines with Triton Inference Server

Optimize latency and throughput for certified production deployments

Deploy accelerated LLM applications on professional NVIDIA GPUs

Training program

Module 1TensorRT-LLM Fundamentals: Installation and First Engine Builds (NVIDIA Docker, LoRA, Containers)

Quick installation of the TensorRT-LLM environment on NVIDIA GPUs via official Docker containers, hands-on with build tools to convert Hugging Face models into optimized engines, practical exercises on Llama and Mistral with FP8 and AWQ quantization, creation of your first functional engine, real-time latency tests, and performance gain analysis for concrete enterprise use cases.

Module 2Basic TensorRT-LLM Optimizations: Layer Fusion and KV Cache (Plugins, Multi-Head Attention)

Exploration of TensorRT-LLM's automatic optimization passes such as GEMM fusion and multi-head attention, manual implementation of plugins for custom KV cache, exercises on LLMs from 7B to 70B parameters with paging and prefix caching, precise acceleration measurements via TensorRT benchmarks, development of a reusable build script for professional projects, and group validation of deliverables.

Module 3Advanced TensorRT-LLM Techniques: Serving and Scaling (Triton, Multi-GPU, Dynamic Batching)

Integration of TensorRT-LLM with Triton Inference Server for scalable serving, multi-GPU configuration with tensor parallelism and pipeline parallelism, implementation of dynamic batching and in-flight batching to handle variable requests, practical cases on enterprise chatbots with token streaming, GPU resource optimization using NVIDIA DCGM tools, and production of actionable monitoring metrics.

Module 4Production Deployment of TensorRT-LLM: Monitoring and Real Cases (Kubernetes, Security, CI/CD)

Deployment in Kubernetes clusters using Helm charts for TensorRT-LLM, securing endpoints with TLS and rate limiting, CI/CD integration via GitHub Actions for automated engine rebuilds, resolution of real-world issues like OOM and performance drift, final red thread project on a customized LLM in simulated production, delivery of performance reports, and action plan for certified enterprise skills.