AIMET Model Zoo: Highly accurate quantized AI models are now available

8-bit integer models using the AI Model Efficiency Toolkit.

Making neural network models smaller is crucial for the widespread deployment of AI. Qualcomm AI Research has been developing state-of-the-art quantization techniques that enable power-efficient fixed-point inference while preserving model accuracy, such as Data Free Quantization (DFQ) and AdaRound, which are post-training techniques that achieve accurate 8-bit quantization without data.

To make this research more accessible and contribute to the open-source community, Qualcomm Innovation Center (QuIC) launched the AI Model Efficiency Toolkit (AIMET) on GitHub in May 2020. AIMET’s goal is to enable power efficient integer inference by providing a simple library plugin for AI developers to utilize for state-of-the-art model efficiency performance. The AIMET project is flourishing with regularly updated quantization techniques based on work from Qualcomm AI Research and active use by the broader AI community, including multiple mobile OEMs, ISVs, and researchers in academia.

QuIC is now taking it a step further by contributing a collection of popular pre-trained models optimized for 8-bit inference to GitHub in the form of “AIMET Model Zoo.” Together with the models, AIMET Model Zoo also provides the recipe for quantizing popular 32-bit floating point (FP32) models to 8-bit integer (INT8) models with little loss in accuracy. The tested and verified recipes include a script that optimizes TensorFlow or PyTorch models across a broad range of categories from image classification, object detection, semantic segmentation, and pose estimation to super resolution, and speech recognition.

This will allow researchers and developers direct access to highly accurate quantized models, saving them time in achieving performance benefits like reduced energy consumption, latency, and memory requirements for on-target inference. For example, imagine you are a developer wanting to do semantic segmentation for image beautification or autonomous driving use cases by using DeepLabv3+ model. AIMET Model Zoo provides an optimized DeepLabv3+ model using the DFQ and Quantization Aware Training (QAT) features from AIMET. The corresponding AIMET Model Zoo recipe points to this optimized model and provides proper calls to the AIMET library to run INT8 simulation and assess performance. In fact, the AIMET quantized version has a Mean Intersection over Union (mIoU) score of 72.08%, which is virtually equivalent to the 72.32% provided by the original FP32 model. The image below visually shows how the quantized model in AIMET Model Zoo results in accurate semantic segmentation.

This is one example. The AIMET Model Zoo has many INT8 quantized neural network models that provide accurate inference comparable to FP32 models. With this initial contribution of 14 INT8 models to AIMET Model Zoo, we are easing the hurdles for the ecosystem in using quantized models in their AI workloads and thus marching toward making fixed-point power-efficient inference ubiquitous. You can get the best of both worlds — the high accuracy of a floating-point model and the model efficiency of 8-bit integer models.

 

Did this article help you? If so, please tell me in a comment what do you think about it.

Don’t miss any of our future video tutorials, follow us on Youtube. Like us on Facebook. Watch our photo albums on Flickr. Subscribe now to our newsletter.

Post a comment


VOOPOO, a global leader in the vaping industry, proudly announces the launch of its latest innovation, the VOOPOO ARGUS A. This new …

Jul 12, 2024

Immerse in Pure Sound, Anywhere, With Edifier W830NB Wireless Over Ear Headphones.   Main Features – Hi-Res Audio with LDAC. The W830NB …

Jul 12, 2024

Coupon Code Alert! You can now buy [EU Direct] KuKirin G3 18Ah 52V 1200W 10.5in Folding Moped Electric Scooter from Banggood.com for …

Jul 12, 2024

Coupon Code Alert! You can now buy KuKirin G2 MAX Electric Scooter 20Ah 48V 1000W 10in Folding Moped Electric Scooter 60-80KM Mileage …

Jul 12, 2024

This firmware is for Boxput iATV Q5 TV Box with Allwinner H316 as CPU. BTW, you can buy Boxput iATV Q5 TV …

Jul 12, 2024

We are pleased to introduce the latest firmware upgrade for the OBSBOT UVC to NDI Adapter. This upgrade brings enhanced streaming flexibility …

Jul 11, 2024

ASUS announced the ASUS NUC 14 Pro+, an ultracompact PC that offers best-in-class performance thanks to the power of the new Intel® …

Jul 11, 2024

FiiO, a known name in the HiFi Audio realm has released a brand new desktop DAC/AMP, the FiiO K11 R2R. It comes …

Jul 11, 2024

50% Off for PISEN 140W Mega Charging Hub @ Kickstarter. Highlights 2x 140W & 1x 100W USB-C 1x USB-A 2x AC Outlets …

Jul 11, 2024

The Most Advanced Snapdragon Mobile Platform, Snapdragon 8 Gen 3 for Galaxy, Powers the New Samsung Galaxy Z Series Globally. Snapdragon 8 …

Jul 11, 2024

Coupon Code Alert! You can now buy [EU Direct] YISORA I8 Corded Vacuum Cleaner 6M Long Cord 23Kpa Vacuum Cleaner for Pet …

Jul 11, 2024

Coupon Code Alert! You can now pre-order the new Xiaomi Redmi bluetooth Speaker Portable Wireless Speaker Dual Drivers Stereo Bass TWS IP67 …

Jul 11, 2024