Ghulam Qadir
Engineering2 min read

Search on the device, not only in the prompt

EmbeddingGemma 2 maps text, images, audio, and video into one small embedding space. That is a product decision about where private data is allowed to go.

Search on the device, not only in the prompt

On 6 October 2026 Google released EmbeddingGemma 2, a 740 million parameter embedding model under Apache 2.0. It maps text, code, images, audio, and video into one space, and it is sized to run on a laptop or a phone. The details are in Google’s post. The first EmbeddingGemma, text only, had already passed 20 million downloads. This one is the version I would actually plan a feature around.

The generator was never the search engine

A lot of AI features in products are a prompt with a pile of documents pasted in. That works until the pile is a year of call recordings, a design archive, or a codebase. Embeddings are the unglamorous half: turn each item into a vector, retrieve a few, then let a generative model talk.

EmbeddingGemma 2 is interesting because the retrieval can stay on the machine. Google’s numbers put code retrieval well up from the previous model on MTEB Code, and the same weights cover images, documents, video, and audio. A voice memo can point at a clip. A sentence can point at a function. You do not need a separate specialist model for each.

Where I would use it

  • A local search over a repo before a coding agent reads files.
  • A client portal where contracts should not leave the laptop just to answer which PDF mentioned the penalty.
  • A media library: find a frame or a sentence without uploading the archive to a chat product.

It pairs with Gemma 4 for the answer step, and they share a text tokenizer and audio encoder, so the combined footprint is smaller than bolting two unrelated models together. That matters on a device. It matters less on a server you already pay for.

What I would not pretend

On-device does not mean correct. A 740 million parameter model will miss a clause a careful reader would catch. I would show the hit, the timestamp, or the file path, and let the person open it. Retrieval you can inspect is more honest than a paragraph with no source.

For a product studio, this release is a reminder. The model people talk about writes the sentence. The model that makes the product feel like it knows your files is the one that embeds them, and it can live next to the files.