AI Share of Voice
- Node.js
- Express
- TypeScript
- PostgreSQL
- Prisma
- React 19
- Tailwind v4
- React Query
- Recharts
- OpenAI
- Anthropic
- Gemini
- Zod
GEO · Generative Engine Optimization · Multi-tenant SaaS
Rank tracking for the AI era. When someone asks ChatGPT "which SUV should I buy?", does your brand show up, get recommended, or get ignored? This platform measures that fuzzy signal across several models, turns it into Share of Voice with confidence intervals, and translates it into a concrete plan to improve it.
What problem does AI Share of Voice solve?
Traditional SEO assumes people click on search results. But increasingly, the buying decision runs through a generated answer: you ask an AI “which one should I get?” and it hands you a pre-chewed recommendation. In that world it matters less where you rank on Google, and more how you’re named — or whether you’re named at all — when the model answers. The industry started calling this GEO (Generative Engine Optimization) or AEO (Answer Engine Optimization).
The catch is that the signal is slippery. LLM answers are non-deterministic: the same question, twice, gives two different answers. There’s no stable “position 3”. And reading the answer isn’t enough: you need to know whether the brand is recommended, mentioned in passing, compared, or outright warned against — and what the model leaned on to say it.
I built AI Share of Voice as a tool for a GEO agency (INDI): a platform that systematically queries the models, measures each brand’s standing with statistical rigor, and closes the loop with an actionable diagnosis of why a brand shows up rarely and what to do about it. The first vertical is the Argentine automotive market.
How does the AI Share of Voice pipeline work?
The heart of the system is a pipeline that turns questions into metrics. It’s not one LLM call and done — it’s a chain of steps, each with its own design decision.
The most important structural decision lives at the ends of that chain. At the input, statistical sampling: because the same prompt yields different answers, each priority prompt runs N times against each model. At the output, honest aggregation: Share of Voice is computed per run in the correct unit — (prompt × provider) — and mean, standard deviation and a 95% confidence interval are stored using a real Student’s-t implementation. The number the strategist sees isn’t a point: it comes with an error bar.
The other interesting end is the parser. The model’s raw answer is prose; to measure it you have to structure it. A cheap second LLM, at temperature 0, extracts a rich per-mention schema from each answer: brand and model, position, whether it’s recommended, sentiment, framing type (recommended / mentioned / compared / warned against), price positioning, and the sources the model claims it used. That parser detects truncated JSON, strips code fences, and validates every enum with a fallback — because in production models return everything.
What technical decisions underpin AI Share of Voice?
The parts I care about most aren’t the pretty screens, they’re the decisions that make the numbers trustworthy and the tool say something actionable.
Share of Voice with error bars
Measuring positioning over a non-deterministic system without quantifying uncertainty is fooling yourself. So the scoring layer has its own Student's-t implementation (with table and interpolation), computes confidence intervals only when the sample is big enough (n≥2), and derives a visibility score = quality × confidence, where confidence grows with sample size.
- SOV aggregated in the right unit: (prompt × provider), not loose mentions
- Mean, SD and 95% CI persisted per sampling batch
- Sample-size planning for a target margin — how many runs you actually need
Three engines, one interface, graceful degradation
Each provider implements a common interface and is instantiated lazily inside the query — so the OpenAI/Anthropic SDKs don't blow up at import time when a key is missing (key to letting tests run without credentials). The system discovers at runtime which providers have a key and works with whatever's there; since Gemini's SDK lacks AbortSignal, it's emulated with a Promise.race.
- Queries run with web-search / grounding tools, returning structured citations
- Separate model roles: one to answer, a cheap one to parse, one to analyze
- Per-provider fallback: the tool doesn't fall over because one API key is missing
From prose to structured mentions
The resolver preloads brands and aliases once to kill N+1 lookups, matches case-insensitively by name or alias, registers unknown models as "pending" for review, and attaches a brand named without a product to a sentinel model so brand-level signal isn't lost. What doesn't match isn't dropped: it's saved with its raw JSON for later recovery.
- In-memory per-response resolver — no N+1 against the database
- Discovery of new models + a queue for unmatched mentions
- Every mention stores position, framing, sentiment and claimed sources
Entity audit → prioritized playbook
Measuring without saying what to do is a report, not a tool. The audit module runs four crawlers in parallel (Wikipedia, Wikidata, Schema.org, citations) under Promise.allSettled, scores the brand's entity presence, and reconciles findings into concrete playbook actions — each with its impact range, effort and estimated time — matched against a Markdown catalog of AEO tactics.
- Per-source entity score + overall score, with ok/warn/crit findings
- Actions ranked by impact × effort (create a Wikipedia article, a Q-id, FAQ schema, llms.txt…)
- Canary-based drift detection: fixed re-query + statistical outlier test (mean + 3σ)
AI visibility vs. the actual market
A brand can sound loud in the answers and sell little, or the other way around. The market module harvests real vehicle registrations, matches brands against the ones in the workspace, and computes the gap: where you're under-represented in the AIs versus your real market share. That contrast is the clearest opportunity signal for the strategist.
- Market harvest per period, alias-matched against the tracked brands
- AI-SOV vs. real-share gap, with an under/over-represented verdict
- Demand radar: keywords and questions harvested (SERP/PAA/marketplaces) → new prompts to measure
The architecture, briefly
Express + TypeScript backend on PostgreSQL/Prisma (39 models, 33 modules), genuinely multi-tenant: almost everything is workspace-scoped, with JWT auth + rotated, hashed refresh tokens, and a composite FK that guarantees at the database level that the primary brand belongs to the same workspace. Seven independent cron jobs (measurement runs, weekly digest, drift canaries, entity audit, market and demand harvests) startable at runtime from settings. React 19 + Vite + Tailwind v4 frontend with React Query and Recharts, in a Stripe-like visual language: light, precise, data at the center. The backend validates its config at boot and won’t start without a strong JWT and at least one LLM key; tests run against real Postgres with LLM clients instantiated lazily, so the suite doesn’t depend on credentials.
Closing
AI Share of Voice is the project where an idea I care about shows up most clearly: measuring well is a design decision, not a detail. Anyone can call an LLM and count how many times a brand appears; the hard part is doing it in a way the number survives — sampling against non-determinism, aggregating in the right unit, attaching a confidence interval, and only then turning it into a recommendation. Everything else in the platform — the parser, the entity matching, the audit, the playbook, the contrast with the real market — exists to close the path from “what do the AIs say about my brand?” to “and what do I do tomorrow to improve it?”.