Is the training Qualiopi certified?

Yes, Learni is a Qualiopi-certified training organization. This certification guarantees the quality of our courses and enables funding through OPCO and other mechanisms.

How can I fund my training?

Our courses are eligible for OPCO funding. Learni supports you through the funding process. A personalized quote is available upon request.

What are the prerequisites for this training?

The prerequisites for the Training TensorRT-LLM - Accelerate LLM Inference x10 in Production training are: Proficiency in PyTorch/TensorFlow, advanced CUDA, optimized C++/Python, GPU architectures. A preliminary interview validates your eligibility.

Is the training available in person?

Yes, the Training TensorRT-LLM - Accelerate LLM Inference x10 in Production training is available in distanciel. The format can be adapted to your needs (inter, intra, custom).

Training TensorRT-LLM - Accelerate LLM Inference... | Learni

The story of Learni

Founded by passionate advocates of learning and innovation, Learni set out to make professional training accessible to everyone, everywhere in the world. Our team works in the largest cities such as Paris, Lyon, Marseille, and internationally, to support talents and organizations in their skills development.

Configure your training

Response within 24hNo commitment100% free

8 spots remaining for the next session

FormatParticipantsDateDetails

Which format do you prefer?

Methods, approaches, and teaching resources

The Training TensorRT-LLM - Accelerate LLM Inference x10 in Production training is delivered in-person or remotely (blended-learning, e-learning, virtual classroom, remote in-person). At Learni, a Qualiopi-certified training organization, each program is designed to maximize skills acquisition, regardless of the training mode chosen.

The trainer alternates between demonstrative, interrogative, and active methods (through practical exercises and/or real-world scenarios). This pedagogical approach ensures concrete and directly applicable learning in the workplace.

Teaching resources provided

To ensure the quality of the Training TensorRT-LLM - Accelerate LLM Inference x10 in Production training, Learni provides the following teaching resources:

Mac or PC computers, high-speed fiber internet, whiteboard or flipchart, video projector or interactive touchscreen (for remote sessions)
Training environments installed on workstations or accessible online
Course materials, practical exercises, and supplementary resources
Post-training access to materials and teaching resources

For in-house training at a location external to Learni, the client ensures and commits to having all necessary teaching materials (IT equipment, internet connection...) for the proper conduct of the training action in accordance with the prerequisites indicated in the communicated training program.

* contact us for remote feasibility** ratio varies depending on the training followed

Assessment methods

The assessment of skills acquired during the Training TensorRT-LLM - Accelerate LLM Inference x10 in Production training is carried out through:

During training: case studies, practical exercises, and professional scenarios
At the end of training: self-assessment questionnaire and skills evaluation by the trainer
After training: training completion certificate detailing acquired skills

Training accessibility

Learni is committed to the accessibility of its professional training programs. All our training programs are accessible to people with disabilities. Our teams are available to adapt teaching methods to your specific needs. Do not hesitate to contact us for any accommodation request.

Training objectives

Master TensorRT-LLM for ultra-fast LLM inferences in enterprise environments

Develop optimized engines with plugins and custom kernels

Implement distributed multi-GPU inferences for professional projects

Design scalable and certified inference pipelines

Optimize TensorRT-LLM performance, reducing latency by 90%

Deploy secure and monitored production solutions

Training program

Module 1TensorRT-LLM Fundamentals: Building Engines and Advanced Configuration (NVIDIA Tools, CUDA)

Dive into expert installation of TensorRT-LLM on GPU clusters, configure Docker environments with CUDA 12+, build first TRT-LLM engines from HuggingFace models like Llama2, test real-time batching inferences, analyze profiling logs to identify initial bottlenecks, produce your first optimized deliverable ready for scaling, with exercises on real business cases to anchor professional certifying skills.

Module 2TensorRT-LLM Kernel Optimizations: Custom Plugins and Ops Fusion (TensorRT Plugins, GEMM)

Develop custom kernels via TensorRT plugins to accelerate FlashAttention and Rotary Embeddings, fuse matrix operations GEMM with cuBLAS, benchmark gains on GPT-J and Mistral, implement INT8/FP8 quantizations to reduce VRAM memory by 50%, apply techniques to red thread projects, generate detailed performance reports, transform your skills into immediate enterprise assets through intensive practices.

Module 3Distributed TensorRT-LLM Inference: Multi-GPU and Tensor Parallelism (NCCL, Pipeline Parallelism)

Master tensor and pipeline parallelism on multi-GPU A100/H100 setups, configure NCCL for ultra-fast communications, deploy in-shard inferences for 70B+ models, test fault-tolerance and load-balancing, integrate with Kubernetes for orchestration, simulate high production loads on concrete e-commerce AI cases, produce scalable deliverable architectures, boost your professional projects with ready-to-use certifying expertise.

Module 4Integrated TensorRT-LLM Pipelines: Serving and Model Chaining (Triton Inference Server, gRPC)

Design end-to-end pipelines with Triton Server for chaining TensorRT-LLM and embeddings, expose secure gRPC/REST APIs, manage token streaming and KV-cache for real-time chatbots, integrate Prometheus/Grafana monitoring, test high availability on real workloads, develop CI/CD deployment scripts, finalize red thread project with clear ROI metrics, leverage your enterprise skills through production-ready and certifying solutions.

Module 5Tuning and Production Deployment of TensorRT-LLM: Performance Tuning and Security (Profiling, Helm Charts)

Optimize final latencies to the extreme via advanced TensorRT-LLM profiling, implement OWASP security measures for sensitive inferences, deploy via Helm on EKS/GKE, analyze cloud vs on-prem costs with ROI calculators, conduct expert code reviews on red thread projects, prepare for professional skills certifications, depart with a complete toolbox to transform your enterprise AI infrastructures into performance leaders.