Get Started Post-Training Dynamic Quantization | AI Model Optimization with Intel® Neural Compressor

Channel:

Intel Software

Subscribers:

256,000

Published on June 28, 2023 1:00:08 PM ● Video Link: https://www.youtube.com/watch?v=5xHKe4wWLes

Duration: 4:30

9,523 views

16

Learn the basics of dynamic quantization. Then see how it’s applied to a GPT2-based headline generation application using Intel Neural Compressor.

Dynamic quantization is the simplest method for quantizing AI models for efficient deployment. This technique quantizes the model weights of a pre-trained model and inserts functions into the model to quantize activations during inference. While this adds runtime overhead, it can also adapt the scale factor dynamically as the input ranges change.

Intel® Neural Compressor works across PyTorch*, TensorFlow*, and ONNX* Runtime. Learn how to implement dynamic quantization using a GPT2-based headline generation AI Reference Kit. The demonstration discusses options to customize the dynamic quantization process and shows the resulting speedup for this application.

Intel® Neural Compressor: https://bit.ly/3Nl6pVj

Intel® Neural Compressor https://GitHub: bit.ly/3NlBgkH

About the AI Model Optimization with Intel® Neural Compressor Series:
Learn how to choose and get started with AI model optimization techniques. Get started with examples using Intel® Neural Compressor, which works within PyTorch*, TensorFlow*, and ONNX* Runtime

About Intel Software:
Intel® Developer Zone is committed to empowering and assisting software developers in creating applications for Intel hardware and software products. The Intel Software YouTube channel is an excellent resource for those seeking to enhance their knowledge. Our channel provides the latest news, helpful tips, and engaging product demos from Intel and our numerous industry partners. Our videos cover various topics; you can explore them further by following the links.

Connect with Intel Software:
INTEL SOFTWARE WEBSITE: https://intel.ly/2KeP1hD
INTEL SOFTWARE on FACEBOOK: http://bit.ly/2z8MPFF
INTEL SOFTWARE on TWITTER: http://bit.ly/2zahGSn
INTEL SOFTWARE GITHUB: http://bit.ly/2zaih6z
INTEL DEVELOPER ZONE LINKEDIN: http://bit.ly/2z979qs
INTEL DEVELOPER ZONE INSTAGRAM: http://bit.ly/2z9Xsby
INTEL GAME DEV TWITCH: http://bit.ly/2BkNshu

#intelsoftware #ai #oneapi

Get Started Post-Training Dynamic Quantization | AI Model Optimization with Intel® Neural Compressor