Visual Product Search on a Laptop with DINOv2 (No Vector DB Required)

A client wanted something simple to describe: a salesperson takes a photo of a product, sends it to a Telegram bot, and the bot replies with the matching item from the catalogue.

The MVP scope was about 30 products and ~700 photos. When people hear "image search," they reach for a vector database, a GPU server, and a fine-tuning pipeline. For this size, you need none of them. It ran on a MacBook.

Why DINOv2

DINOv2 is a self-supervised vision model from Meta. "Self-supervised" is the important part: it wasn't trained to output labels like "cat" or "chair," it was trained to produce embeddings where visually similar things land close together. That's exactly what search needs, and it works well out of the box — no fine-tuning, no labelled data.

The small variant (ViT-S/14) produces a 384-dimensional vector per image and runs fine on a laptop.

Step 1: Embed every catalogue photo

# index.py
from pathlib import Path

import torch
import torch.nn.functional as F
from PIL import Image
from torchvision import transforms

device = "mps" if torch.backends.mps.is_available() else "cpu"
model = torch.hub.load("facebookresearch/dinov2", "dinov2_vits14").to(device).eval()

preprocess = transforms.Compose([
    transforms.Resize(256, interpolation=transforms.InterpolationMode.BICUBIC),
    transforms.CenterCrop(224),
    transforms.ToTensor(),
    transforms.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225)),
])


@torch.inference_mode()
def embed(paths: list[Path], batch_size: int = 32) -> torch.Tensor:
    chunks = []
    for i in range(0, len(paths), batch_size):
        batch = torch.stack([
            preprocess(Image.open(p).convert("RGB")) for p in paths[i : i + batch_size]
        ]).to(device)
        # L2-normalise so a dot product == cosine similarity
        chunks.append(F.normalize(model(batch), dim=-1).cpu())
    return torch.cat(chunks)


if __name__ == "__main__":
    # catalogue/<product_id>/<photo>.jpg
    paths = sorted(Path("catalogue").glob("*/*.jpg"))
    product_ids = [p.parent.name for p in paths]
    torch.save({"vectors": embed(paths), "product_ids": product_ids}, "index.pt")

700 images × 384 floats is about 1 MB. That's the whole "database."

Step 2: Search is one matrix multiply

Because every vector is normalized, cosine similarity is just a dot product. For one query against 700 vectors, that's instant:

# search.py
import torch

from index import embed

index = torch.load("index.pt")
VECTORS, PRODUCT_IDS = index["vectors"], index["product_ids"]
MIN_SCORE = 0.55  # tune on your own data


def search(query_path, k: int = 3) -> list[tuple[str, float]]:
    q = embed([query_path])[0]
    scores = VECTORS @ q  # (N,)

    # Several photos per product -> keep each product's best-matching photo
    best: dict[str, float] = {}
    for score, pid in zip(scores.tolist(), PRODUCT_IDS):
        if score > best.get(pid, -1.0):
            best[pid] = score

    ranked = sorted(best.items(), key=lambda kv: kv[1], reverse=True)[:k]
    return [(pid, s) for pid, s in ranked if s >= MIN_SCORE]

Two design choices that mattered more than the model:

Aggregate per product, not per photo. Each product has many photos from different angles. If you return the top 3 photos, you'll often get the same product three times. Taking the max score per product gives you three different candidates.

Have a "no match" answer. Without a threshold, the bot always returns something, even for a photo of someone's shoe. A minimum score lets it say "I couldn't find this in the catalogue," which users trust far more than a confident wrong answer. Pick the threshold by looking at real scores from real queries — there's no universal number.

Step 3: Put it where users already are

The interface is a Telegram bot: receive a photo, download it, call search(), reply with the product name and its reference photo. The ML part is ~60 lines. The bot is ~60 more.

What actually affects quality

Once the pipeline worked, accuracy was mostly about data, not models:

• Catalogue photos should look like query photos. Studio shots on white backgrounds don't match phone photos on a counter. Adding a few "realistic" photos per product helped more than anything else.
• Multiple angles per product. The per-product max means more angles = more chances to match.
• Crop to the object. A small product in a large busy frame embeds the background as much as the item.

When this stops being enough

This brute-force approach is fine up to tens of thousands of images — a dot product over 50,000 × 384 is still milliseconds. Beyond that, or with many concurrent users, move the vectors to something like pgvector (you probably already run PostgreSQL) or FAISS. And if similar-looking products start getting confused, then consider fine-tuning.

But start here. A pretrained model, a normalized matrix, and a threshold will get you surprisingly far.