Run AI on Your Android Phone Without an Internet

This article delves into how to install and run Large Language Models (LLMs) locally using the MLC Chat app.

 

Run AI on Your Android Phone Without an Internet Connection

Unleash the Power of AI on Your Android: A Guide to Running LLMs Locally

The world of Artificial Intelligence is rapidly evolving, and now you can experience the power of AI assistants directly on your Android phone, even without an internet connection! This article delves into how to install and run Large Language Models (LLMs) locally using the MLC Chat app.

Introducing MLC Chat: Your Gateway to Local AI

While various applications like LM Studio and GPT4All enable local LLM execution on computers, options for Android phones have been limited. Fortunately, MLC LLM has developed the innovative MLC Chat app, empowering you to download and run AI models directly on your Android device. This app offers a variety of small AI models (ranging from 2B to 8B parameters) including popular choices like Llama 3, Gemma, Phi-2, and Mistral.

Getting Started with MLC Chat

Here's a step-by-step guide to set up MLC Chat and experience local AI on your Android phone:

1. Download the MLC Chat App: First, you'll need to download the MLC Chat app. Keep in mind that this is a free app, but you'll download the APK file (approximately 148MB) and install it manually.
2. Explore the Available AI Models: Once launched, the MLC Chat app will showcase a list of compatible AI models. This includes the latest iteration, Llama 3 8B, alongside other popular options like Phi-2, Gemma 2B, and Mistral 7B.
3. Download and Run Your Preferred Model: Choose the AI model that best suits your needs. For this example, we'll download Microsoft's Phi-2 model, known for its compact size and efficiency.
4. Engage in Local AI Chat: After downloading the selected model, simply tap the "chat button" next to it. This unlocks the ability to converse with the AI model directly on your Android phone, eliminating the need for an internet connection.

Important Considerations and Performance

While MLC Chat offers a groundbreaking solution for local AI interaction on Android devices, it's crucial to acknowledge some key factors:

• Limited On-Device NPU Utilization (Currently): At present, MLC Chat doesn't fully leverage the on-device Neural Processing Unit (NPU) present on most Snapdragon processors. As a result, the inference process relies primarily on the CPU, potentially impacting speed.
• Model Performance and Hardware Dependency: Testing revealed that the Phi-2 model functioned smoothly on the test device. However, other models like Gemma might not run as effectively, and the larger Llama 3 8B exhibited slower processing speeds.
• Processing Speed and Hardware Specifications: The token generation speed plays a significant role in user experience. During testing on a OnePlus 7T powered by the Snapdragon 855+ SoC (a five-year-old chip), the Phi-2 model generated output at a rate of approximately 3 tokens per second.

The Future of Local AI on Android

While MLC Chat paves the way for local AI experiences on Android, the future holds even greater possibilities:

• Enhanced NPU Integration: Anticipate future versions of MLC Chat to capitalize on the on-device NPU present in Snapdragon processors. This will potentially unlock significant performance improvements by utilizing the combined capabilities of CPU, GPU, and NPU.
• Matching Performance on Apple Devices: Apple's MLX framework already empowers iPhones with local inference speeds close to 8 tokens per second. We can expect Android devices to follow suit, offering comparable performance with better NPU support.
• Qualcomm's Snapdragon 8 Gen 2 Potential: Qualcomm itself claims that the Snapdragon 8 Gen 2 can generate up to 8.48 tokens per second when running a larger 7B model. This number is expected to rise further for smaller, quantized models.

Deepen Your AI Knowledge with Discounted Chat GPT Books

As the realm of AI continues to expand, staying informed is crucial. Explore a vast selection of Chat GPT books at incredible discounts on Amazon (link: https://amzn.to/3xQu8aI)! Delve into the intricacies of Chat GPT, explore its potential applications, and gain insights from leading experts in the field. These discounted books can equip you with the knowledge to navigate the exciting world of AI with confidence.

The ability to run LLMs locally on Android devices opens doors to a plethora of possibilities.