vLLM Server Using OpenAI API on Gaudi 3 | AI with Guy

Channel:
United States Intel Software
Subscribers:
256,000
Published on ● Video Link: https://www.youtube.com/watch?v=55tKpLRwq2I


Duration: 0:00
155 views
4

In this episode of AI with Guy, learn how to build a real-world retrieval-augmented generation (RAG) system using vLLM, the OpenAI API, and Intel Gaudi 3 across multiple AWS instances. This demo shows how to deploy a scalable, production-ready Gen-AI application using the OPEA framework.

Whether you're a developer, ML engineer, or AI architect, this walkthrough covers:
Setting up a vLLM server
Connecting to the OpenAI API for inference
Deploying across AWS EC2 using Intel Gaudi 3
Coordinating workloads using OPEA for high performance and cost efficiency

Resources:

OPEA Documentation: https://opea.dev/
ChatQnA Gen-AI Example: https://opea-project.github.io/latest/GenAIExamples/ChatQnA/README.html
Intel Cloud Dev Tools: https://cloud.intel.com/

Tech Stack:
-vLLM
-OpenAI API
-Intel Gaudi 3 (AWS DL1/DL2 instances)
OPEA
AWS EC2

About Intel Software:
Intel® Developer Zone is committed to empowering and assisting software developers in creating applications for Intel hardware and software products. The Intel Software YouTube channel is an excellent resource for those seeking to enhance their knowledge. Our channel provides the latest news, helpful tips, and engaging product demos from Intel and our numerous industry partners. Our videos cover various topics; you can explore them further by following the links.

Connect with Intel Software:
INTEL SOFTWARE WEBSITE:https://intel.ly/2KeP1hDD
INTEL SOFTWARE on FACEBOOK:http://bit.ly/2z8MPFFF
INTEL SOFTWARE on TWITTER:http://bit.ly/2zahGSnn
INTEL SOFTWARE GITHUB:http://bit.ly/2zaih6zz
INTEL DEVELOPER ZONE LINKEDIN:http://bit.ly/2z979qss
INTEL DEVELOPER ZONE INSTAGRAM:http://bit.ly/2z9Xsbyy
INTEL GAME DEV TWITCH:http://bit.ly/2BkNshuu

#intelsoftware

vLLM Server Using OpenAI API on Gaudi 3 | AI with Guy | Intel Software



Other Videos By Intel Software


2025-06-13Discover Robotic AI Innovation at the Edge | Intel Software
2025-06-13Run Ollama + Web-UI on Your AI PC | AI With Guy
2025-06-13Get a GPU VM in One Click | AI With Guy
2025-06-11The Heart of HPC Today: Heterogeneous Computing | Intel Software
2025-06-10FAMU-FSU: Up-to-the-Minute Lessons in AI with the help of the Educator Program by Intel
2025-06-10Cornell University: A Support System to Optimize Curriculum, Course Materials and Student Engagement
2025-06-10Cal Poly: Breaking New Ground in Programming Curriculum, Without Reinventing the Wheel
2025-06-10RAG Pipeline Using Standard Libraries and OPEA | AI with Guy |
2025-06-09Run PyTorch 2.7 on Intel GPUs: A Step-by-Step Setup | AI with Guy
2025-06-06GPU Coding Using Triton Compiler | AI with Guy
2025-06-05vLLM Server Using OpenAI API on Gaudi 3 | AI with Guy
2025-06-04Build a Gen-AI Application Across Multiple AWS Instances with OPEA | AI with Guy
2025-06-03OPEA vs. NVIDIA NIM: What’s Best for Your GenAI Deployment?
2025-05-28PyTorch Export Quantization with Intel GPUs | Intel Software
2025-05-23Unlocking Gen AI: From Experimentation to Production with Red Hat & Intel | Intel Software
2025-05-23Overcoming Deployment Challenges: Scaling AI in Edge Computing w/ Red Hat AI & Intel Edge Platforms
2025-05-23Discover AI Innovations at Red Hat Summit with Intel: RHEL AI, OpenShift AI & Edge AI
2025-05-22Explore OpenVINO Model Hub – Instantly Compare AI Model Performance Across Devices | AI with Guy
2025-05-22Build a RAG Chatbot with OPEA on AWS | AI with Guy | Intel Software
2025-05-20Enterprise AI Inference with Intel: Bill Pearson on Infrastructure & Standards | Intel Software
2025-05-20Automatically Quantize LLMs with AutoRound | Intel Software