Grocery Sidekick
Live · iOS
An ingredient scanner that never says “safe.”
I built Grocery Sidekick end to end: the product, the catalog pipeline, the ingredient dictionary and parser, the results engine, the app, and the App Store launch. It reads the label instead of ranking it.
The short version
The product’s hardest job isn’t finding allergens — it’s never implying an absence it can’t support. Most of the system exists to keep “we couldn’t tell” distinct from “not found.”
- Role
- Founder / AI Product Builder, Yellow Labs Products
- Timeline
- 2026 – present
- Status
- Live on the App Store
- Live
- App Store ↗
Key signals
- 99.2% allergen recall from ingredient lists alone (9,612 declared allergens)
- Zero false “not found” results when the label’s declaration is read
- 400,000+ foods bundled on the phone for offline use
Capabilities shown
- Data normalization at scale
- Deterministic + AI hybrid
- Evaluation benchmarks
- Designing for uncertainty
- On-device OCR
- Consumer mobile
How it was built · I use AI-assisted development to work directly in the implementation layer, while owning the product decisions, architecture, evaluation criteria and production validation.
The answer is on the label — in 6pt type, written for regulators.
The answer to “can I eat this?” is printed on the back of the package in 6pt type, written for regulatory compliance rather than for a shopper, and it has to be decoded standing in an aisle holding a cart. Anyone avoiding an allergen, a seed oil, or an additive ends up translating an ingredient list on the spot — and most apps that offer to help hand back a letter grade instead of the facts.
The failure that matters is false reassurance.
A shopper with an allergy acts on “not found.” A missed allergen and a cheerful “looks fine” are the same failure from where they stand. So the product is built around what it can honestly say: present, possible, not found — and, often enough to matter, “couldn’t tell.”
The tradeoffs I made on purpose.
Never say “safe”
The word is banned in code, in every state. “Not found” is reported only when the label was read cleanly, nothing was truncated, and every ingredient resolved — otherwise the answer is “couldn’t tell.” “May contain” and facility warnings stay distinct from absence rather than collapsing into a clean result.
Fuzzy matching, measured and switched off
Edit-distance matching added under one point of coverage, and mapped “sodium citrate” to “sodium nitrate” and “vitamin E” to “ascorbic acid.” A coverage gain that quietly misidentifies ingredients isn’t a gain here, so it stays disabled; resolution is exact, alias-based, and repairs known OCR confusions deterministically.
A deterministic engine with AI on a short leash
What’s in a product is never decided by a model. AI has one runtime job — reading a brand and product name from a photo of the front of the package when a barcode fails — and a validator strips anything else it returns.
Bundle the clean source, not the biggest one
Bundling Open Food Facts would have added 12.1% more products — and 15.4% of the statements it added failed quality checks. The bundled catalog is USDA Branded Foods; Open Food Facts stays an online fallback.
Alternatives clear the same gate
A suggested alternative must clear each of the shopper’s allergens and diets under the same evidence gate as the scan. “Possible” or “couldn’t tell” never qualifies.
A deterministic engine, built for the aisle.
From barcode or photo to an answer the shopper can act on — most of it on the phone, and none of the judgment delegated to a model.
- 01
Identify
A barcode resolves against a catalog of 400,000+ foods bundled on the phone — so an aisle with no signal still works — with online lookups as fallbacks, and front-of-package identification as the last resort.
- 02
Read
A photo of the ingredient panel is read on the device by hand-written native modules over Apple Vision and ML Kit. Ingredient-label photos are never uploaded.
- 03
Parse and resolve
The statement is parsed and each ingredient resolved against a curated dictionary of 1,307 ingredients and 3,791 aliases, drawn from FDA, USDA and public-domain sources.
- 04
Judge
A results engine assigns each watched item a state — present, possible, not found or couldn’t tell — and names the ingredient responsible. The watch list covers the nine major US allergens (gluten standing in for wheat) plus seed oils, MSG, high-fructose corn syrup and artificial sweeteners.
- 05
Precompute
Evaluating a complex label live took up to four seconds on a phone, so the catalog carries flags precomputed at build time by the same engine — 2 MB — and a verification step re-evaluates a random sample live and fails on any disagreement.
For technical readers · Architecture & stack
The stack supports the product decisions above; it isn't the story.
- App
- ExpoReact NativeTypeScriptiOSOffline-first
- On-device OCR
- Hand-written native modulesApple VisionML Kit
- Data
- Bundled SQLite catalog (USDA Branded Foods)400,000+ foods · 430,000+ barcodes1,307-ingredient curated dictionaryBuild-time precomputed watch flags
- Interpretation
- Ingredient statement parserExact and alias resolution with OCR-confusion repairEvidence-gated absencePresent · possible · not found · couldn’t tell
- Scoped AI
- Front-of-package product identification via a hardened server proxyAgent-proposed dictionary repairs behind a deterministic safety check and an approval step
- Payments
- RevenueCatSubscription management
Recall, measured and held.
Recall on allergens is the number that matters, so it’s measured against a frozen benchmark and held there by the test suite.
Allergen benchmark
9,612 allergen declarations taken from manufacturers’ own CONTAINS statements. With the declaration withheld, the ingredient list alone yields 99.2% recall; with it read, as a real scan does, recall is 100% with zero false “not found” results.
A ratchet, not a target
Zero false “not found” results is enforced as an absolute in the test suite, and the strict-mode miss count may never rise.
MSG and seed-oil benchmark
A second frozen gate of 7,780 known-positive rows covers the non-allergen watch items.
Release audit
An audit of 1,180 “not found” answers on products named for an allergen surfaced 74 false clears, each fixed or downgraded.
Live, and built to work offline.
Grocery Sidekick is live on the App Store and built to work where shoppers actually are — in an aisle, often without signal.
- Offline-first: catalog, saved scans and label reading work without a network
- Ingredient labels read entirely on-device
- JavaScript and dictionary fixes ship over the air; native changes are gated by runtime version
- Stable product IDs preserved across catalog rebuilds
- Subscriptions through RevenueCat
Decisions the measurements changed.
Alternatives that still carried the allergen
Before the shared gate, 3.3% of suggested alternatives still carried an active allergen or diet conflict. After it: zero.
Fuzzy matching removed
Measured, found to cause dangerous mis-mappings for under a point of coverage, and switched off.
Precomputation after measuring latency
Live evaluation took 851ms for ranch dressing and four seconds for ice cream. Build-time flags made results instant for 2 MB.
A bigger catalog rejected
+12.1% products wasn’t worth 15.4% of new statements failing quality checks.
Refusing to answer is a feature. The trust a shopper places in “not found” is only deserved if the product is just as willing to say “couldn’t tell.”
What I’d measure next.
The product works in production today. These are the next layers of measurement and rigor I’d add as it matures.
- 01Precompute catalog flags for high-fructose corn syrup and artificial sweeteners, so alternatives can account for them too.
- 02Run the benchmark gates automatically on every change, not only in the local test suite.
