On-Device Generative AI Integration In Android Applications: Architectural Patterns For Scalable And Efficient AI Systems

Abstract

The growing availability of powerful and scalable mobile system-on-chip (SoC) architecture platforms has enabled intelligent applications to embed generative artificial intelligence (GenAI) within apps without relying on cloud resources, addressing latency, connectivity requirements, and privacy concerns. This paper provides a systematic architectural study of integrating Generative AI on device in the Android environment. We instantiate five common architecture patterns (Direct-Embedded, Client-Server Hybrid, Delegate-Based Offload, Modular Pipeline, and Adaptive Quantized Runtime) and test each on three device levels, using a controlled experimental testbed featuring 5 inference runtimes (LiteRT, ONNX Runtime Mobile, MLC-LLM, MediaPipe GenAI Tasks, and PyTorch Mobile/ExecuTorch). Inference execution time, peak memory costs, energy consumption, and test-set accuracy across quantization levels (FP32 to INT4) demonstrate that Adaptive Quantized Runtime architectures deliver the best responsiveness and energy savings, while Direct-Embedded designs provide the best offline accuracy and strong privacy and reliability assurances at a higher storage cost. The Client-Server Hybrid is well-suited to large-parameter models but has inherent network-dependent latency variance, making it unsuitable for interactive generative experiences. Results are presented as an abstraction of the layered architecture and a decision tree for developers and systems architects to choose and connect patterns based on application-specific requirements, such as latency, energy, privacy, and agility for model updates. The paper concludes by presenting a set of open challenges related to cross-vendor hardware heterogeneity, standardized benchmarking, and on-device continuous personalization, which warrant further research.