AI and Machine Learning in Android: Gemini, ML Kit, TFLite

AI features are now common in apps: chatbots, smart replies, text recognition and recommendations. There are two ways to add them. A cloud model (Gemini, OpenAI and others) runs on a server and is very capable. An on-device model (ML Kit, TensorFlow Lite) runs on the phone, works offline, is fast and keeps data private, but is smaller. Pick based on what the feature needs.

Generative AI: how to call a model safely

A generative AI API takes a prompt (and a conversation history) and returns text. Calls cost money and need an API key, so the key must never be in the app. Anyone can extract it from the APK and run up your bill. The safe design is: the app calls your own server, which holds the key, checks the user, limits usage and calls the model. Firebase offers an AI service that avoids shipping a key and supports App Check; check its current documentation. For learning, you can put a key in local.properties and read it through BuildConfig, but remember that this is for testing only and the key is still extractable.

// the app talks to YOUR server, which talks to the model
interface ChatApi {
    @POST("chat")
    suspend fun send(@Body body: ChatRequest): ChatReply
}

@Serializable data class ChatMessage(val role: String, val text: String)   // "user" or "model"
@Serializable data class ChatRequest(val messages: List<ChatMessage>)
@Serializable data class ChatReply(val text: String)

Streaming responses and history

Models can send the answer in pieces, so the user sees words appear as they are generated. On Android you expose this as a Flow: each emitted chunk is appended to the last message. Keep the conversation history as a list of messages and send the recent ones with each request, because the model has no memory between calls. Store it in Room if users should see old chats.

fun reply(history: List<ChatMessage>): Flow<String> = flow {
    var text = ""
    api.stream(ChatRequest(history)).collect { chunk ->     // your server streams chunks
        text += chunk
        emit(text)                                         // UI shows the growing text
    }
}.flowOn(Dispatchers.IO)

ML Kit on device

• Text recognition (OCR): read printed text from an image.
• Barcode and QR scanning: used in project 17 with CameraX.
• Face detection: find faces and landmarks (not identification).
• Language identification and translation: detect a language and translate offline after a model download.
• Smart reply, image labelling, object detection and entity extraction are also available.

val recognizer = TextRecognition.getClient(TextRecognizerOptions.DEFAULT_OPTIONS)
val image = InputImage.fromBitmap(bitmap, 0)
recognizer.process(image)
    .addOnSuccessListener { result -> show(result.text) }
    .addOnFailureListener { e -> showError(e) }

TensorFlow Lite and recommendations

TensorFlow Lite (now also called LiteRT) runs your own trained model on the phone, for example an image classifier. You add the .tflite file to assets, resize the input to what the model expects, run it and read the scores. Train the model in Python and convert it. For recommendations you can start simple: show items similar to what the user viewed, or popular in the same category, and move to a server-side model when you have data. For AI-powered search, send a query to a server that uses embeddings, and show the ranked results.

Responsible AI

• Tell users when they are talking to an AI, and that answers may be wrong.
• Do not send private data to a model unless the user agreed and your privacy policy says so.
• Never let the model make a final decision about money or safety without a human check.
• Filter and limit input, protect the key on a server, rate-limit each user and watch costs.
• Test for unwanted outputs and give users a way to report a bad answer.

Your task

1. Make a chat screen with a list of messages and a fake streaming repository that emits words slowly.
2. Use ML Kit text recognition on a photo of a printed page.
3. Write the server contract (request and response) your app expects, without the key.
4. Write a short in-app notice that explains the AI feature and its limits.

How to check it worked

• The key is not in the app or in the repository.
• Text grows on screen as chunks arrive.
• You can explain cloud versus on-device AI.