Deploying AI Assistants with OpenVINO™ on the AI PC | Technical Talk | Innovation Selects
10Learn about power efficient AI at the edge by building local assistants with Intel® Core™ Ultra. Create personalized AI assistants with Intel® Core™ Ultra. Discover how to leverage LLMs, speech transcription, and optimization techniques for efficient deployment.
In this session, we explore how to build and deploy local LLM-based assistants on the AI PC or at the edge. Our pipeline leverages a real-time speech transcription model (Distil-Whisper), and Large Language Model (LLama3) powered chatbots leveraging Retrieval Augmented Generation (RAG), to personalize user interactions via text generation and summarization over prior interaction history. We discuss how the Intel® Core™ Ultra enables efficient deployment of LLMs on the CPU, iGPU, and NPU, through optimization techniques such as quantization and OpenVINO™ compression libraries. Live demos will be presented throughout the session to ensure developers can see the work in action.
Intel Innovhttps://intel.ly/46ixwbA46ixwbA
Intel Ehttps://intel.ly/4dCbLWb4dCbLWb
Intel Architecture Set Extensions Programming Reference Sheet Dowhttps://intel.ly/4eArOF84eArOF8
Optimizing Software for Intel® Hybrid Prochttps://intel.ly/482cr6g/482cr6g
Processor Power Management Options | Microsofthttps://intel.ly/3U4cCbo/3U4cCbo
Process Threads API: Win32 apps | Microsofthttps://bit.ly/3Y5te3M/3Y5te3M
System CPU Set Information | Win32 Apps | Microsofthttps://intel.ly/3zHAcDU/3zHAcDU
Microsoft Media Foundation - Win32 apps | Microsofthttps://intel.ly/47Waxnz/47Waxnz
Custom AI Assihttps://intel.ly/3zXm75m/3zXm75m
About Intel Software:
Intel® Developer Zone is committed to empowering and assisting software developers in creating applications for Intel hardware and software products. The Intel Software YouTube channel is an excellent resource for those seeking to enhance their knowledge. Our channel provides the latest news, helpful tips, and engaging product demos from Intel and our numerous industry partners. Our videos cover various topics; you can explore them further by following the links.
Connect with Intel Software:
INTEL SOFTWARE https://intel.ly/2KeP1hDy/2KeP1hD
INTEL SOFTWARE on Fhttp://bit.ly/2z8MPFFy/2z8MPFF
INTEL SOFTWARE on http://bit.ly/2zahGSny/2zahGSn
INTEL SOFTWAREhttp://bit.ly/2zaih6zy/2zaih6z
INTEL DEVELOPER ZONE Lhttp://bit.ly/2z979qsy/2z979qs
INTEL DEVELOPER ZONE INhttp://bit.ly/2z9Xsbyy/2z9Xsby
INTEL GAME DEVhttp://bit.ly/2BkNshuy/2BkNshu
#intelsoftware #innovation
Deploying AI Assistants with OpenVINO™ on the AI PC | Technical Talk | Innovation Selects