Listen to this Post
The landscape of AI on mobile and edge devices just got a major boost. Google AI Edge, known for enabling small language models (SLMs) to run directly on devices like smartphones and browsers, has announced significant expansions to its model offerings. Building on last year’s launch of four initial models, the latest update introduces over a dozen new models—including the powerful Gemma 3 and its multimodal sibling Gemma 3n. These new models bring not only enhanced performance but also the ability to process multiple types of data—text, images, video, and audio—right on the device, opening doors to more versatile and privacy-conscious AI applications. Supported by new Retrieval Augmented Generation (RAG) and Function Calling libraries, developers now have more tools than ever to create intelligent, interactive AI features that work without relying on cloud connectivity.
Google AI Edge initially debuted with support for four small language models that could run directly on Android, iOS, and web platforms. This year, the portfolio has expanded dramatically, now featuring more than a dozen models accessible through the LiteRT Hugging Face community. Among these, Gemma 3n stands out as the first multimodal on-device SLM, capable of handling text, images, video, and audio inputs. This is a significant leap, as it enables truly rich, interactive AI experiences even when offline or on limited connectivity.
Developers can easily download and run these models on devices with just a few lines of code. Models are fully optimized for mobile and web environments, with comprehensive documentation and fine-tuning guides available—including a Colab notebook for training and quantizing Gemma 3 1B. The latest quantization advances in Google’s tools allow models to shrink significantly—up to 4 times smaller—while improving speed and reducing memory usage, thanks to int4 post-training quantization. This means powerful AI can now fit comfortably within the limited resources of everyday devices.
Gemma 3 1B, introduced earlier this year, exemplifies this balance of size and speed: at just 529MB, it can process over 2,500 tokens per second on mobile GPUs, translating to near-instant handling of a full page of text. The new Gemma 3n models, available in 2 billion and 4 billion parameter versions, extend this capability to handle multiple data types, ideal for complex enterprise scenarios. Imagine field technicians snapping photos of parts for instant analysis or warehouse workers updating inventory via voice commands while hands are busy.
A game-changer among the new features is the Retrieval Augmented Generation (RAG) library. RAG enables small language models to access and incorporate vast amounts of specific data—such as thousands of pages or photos—without needing expensive retraining. This selective retrieval mechanism ensures AI responses stay highly relevant to the task or user context. The RAG library is already available on Android, with plans for broader platform support, empowering on-device AI with real-time access to customized information.
Complementing this, the Function Calling library adds interactive intelligence by allowing language models to invoke predefined app functions or APIs directly on the device. For example, in healthcare apps, voice input can be converted to structured data to fill out patient forms automatically. The library also integrates with a Python tool simulation library, which helps developers train their models to accurately call functions using synthetic data, improving accuracy and usability.
Google’s commitment to advancing on-device AI means continuous updates and expansions for these libraries and models. New LiteRT APIs and an AI Edge Portal service for benchmarking further support this ecosystem, providing developers with tools to measure and improve performance on real devices.
What Undercode Say:
Google AI Edge’s latest announcement marks a pivotal shift in on-device AI, making sophisticated language models more accessible, versatile, and efficient. The introduction of multimodal models like Gemma 3n is especially significant because it mirrors how humans interact with the world—through multiple senses—thus paving the way for more natural AI experiences on smartphones and embedded devices.
This expansion addresses a key challenge in AI today: balancing model performance with resource constraints. By leveraging advanced quantization techniques and efficient model design, Google ensures that these models remain compact and fast enough for real-world mobile applications. The ability to process text, images, video, and audio without cloud reliance offers substantial privacy benefits, reduces latency, and cuts operational costs for developers and users alike.
The Retrieval Augmented Generation library is another standout innovation. It eliminates the need for costly fine-tuning by dynamically retrieving the most relevant information for each query, allowing small models to punch well above their weight. This flexibility is crucial for enterprise applications where domain-specific knowledge must be incorporated quickly and frequently without downtime or retraining.
Function calling pushes interactivity to a new level by enabling language models to trigger real-world actions directly on the device. This seamless integration bridges the gap between AI understanding and practical application, from filling out forms to managing IoT devices. The synergy with Python tool simulation for synthetic data generation also empowers developers to customize these models for specific workflows, enhancing accuracy and robustness.
Overall, Google’s approach exemplifies the future of AI: decentralized, multimodal, and tightly integrated into everyday tools. This vision aligns with growing demand for privacy-preserving AI that works offline or in low-connectivity environments while still delivering high-quality, context-aware results.
Developers and enterprises adopting these technologies can expect faster innovation cycles, improved user engagement, and more personalized experiences. However, the complexity of managing multimodal inputs and ensuring security on-device will require continued refinement. Google’s ongoing updates and community support through LiteRT Hugging Face and AI Edge Portal will be essential to sustain momentum and address challenges.
As mobile hardware evolves, supporting even larger and more complex models, we can anticipate a wave of new applications—ranging from intelligent personal assistants that understand speech and images to advanced field service tools that operate entirely offline. Google AI Edge’s commitment to open, extensible libraries and optimized models gives it a strong competitive edge in this rapidly growing market.
Fact Checker Results:
Google AI Edge has officially expanded its support for small language models with over a dozen new options, including the multimodal Gemma 3n.
The newly introduced RAG and Function Calling libraries are available now on Android, with broader platform support planned.
Int4 quantization significantly reduces model size and latency, improving performance on mobile GPUs.
Prediction:
As on-device AI grows more powerful and versatile, expect a surge in applications that rely on multimodal inputs, such as voice-controlled smart home devices, offline translation tools, and context-aware enterprise assistants. Google’s advancements in small language models and retrieval technologies will likely accelerate adoption across industries that demand privacy and low latency. Over the next few years, on-device AI could become the norm, reshaping how users interact with their devices and enabling smarter, more personalized digital experiences even in remote or connectivity-limited environments.
References:
Reported By: developers.googleblog.com
Extra Source Hub:
https://www.quora.com
Wikipedia
Undercode AI
Image Source:
Unsplash
Undercode AI DI v2
