vLLM Inference Meetup Bengaluru

Join us for a deep dive into the engine room of vLLM and llm-d AI inferencing, where we will focus on the architecture, optimizations, and raw engineering required to run inference at scale.

​Whether you’re looking to squeeze every last token out of your GPU cluster or you're curious about the latest commits to the vLLM and llm-d ecosystems, this is the room you want to be in.

​​What to Expect ​Deep Technical Sessions: Hear directly from the maintainers and core committers of vLLM and llm-d

​Scale in Production: Learn from industry leaders about deploying LLMs in production

​Hands-on learning & demos

​Networking: Stick around for food and soft drinks. It’s a great chance to chat with the speakers and exchange ideas with fellow developers and engineers.

​​Who Should Attend ​vLLM and llm-d users and contributors

​ML and infra engineers working on inference and serving

​Platform teams running GenAI in production

​Anyone curious about efficient inference across local, cloud, and Kubernetes

​​Event Agenda ​1:00 PM – 1:30 PM | Registration & Welcome Registration, networking, and welcome refreshments.

2:00 PM – 2:30 PM | vLLM & llm-d Updates Prasad Mukhedkar, Red Hat

​An overview of the latest updates and developments across vLLM and llm-d.

​2:30 PM – 3:00 PM | llm-d for Sovereign & Agentic Workloads Pravein Govindan Kannan, IBM

​An overview of llm-d’s distributed inference stack, along with ongoing optimization and benchmarking efforts for sovereign and agentic workloads.

​3:00 PM – 3:30 PM | Distributed Inference on ROCm with WideEP on vLLM & llm-d by Chaitanya Sri Krishna Lolla & Sirra Ajith, AMD

​Explore distributed inference on AMD ROCm using WideEP with vLLM and llm-d.

​3:30 PM – 4:00 PM | Break & Networking

​4:00 PM – 4:30 PM | Inside vLLM Semantic Router: Intelligent Routing for LLM Inference Aayush Saini, Red Hat

​Explore how vLLM Semantic Router uses intelligent, cache-aware routing based on request semantics, model capabilities, workload characteristics, and infrastructure constraints to optimize inference.

​4:30 PM – 5:00 PM | Scaling Inference at NxtGen Using the vLLM Ecosystem Abhishek Kumar, NxtGen

​Learn how NxtGen is leveraging the vLLM ecosystem to scale LLM inference workloads.

​5:00 PM – 5:15 PM | Break

​5:15 PM – 6:15 PM | Hands-on Workshop: vLLM Inference with AMD GPUs - Get hands-on experience running vLLM inference on AMD GPUs using open-source ROCm platforms.

​​ Important information ​Registration closes 24 hours before the event. We cannot admit unregistered attendees.

​Bring your laptop with SSH installed (GPU instances provided by the organizers)

​Please bring a photo ID to verify your registration on arrival.