Work · Products · 12
FirstChair
How do ChatGPT, Gemini and Perplexity describe your law firm versus your competitors?
- Type
- Products
- When
- 2026 · shipped
- Role
- Sole designer and developer. Measurement design, orchestration, storage, interface.
- Scale
- Three models tracked · repeated sampling rather than single-shot queries
In plain language
FirstChair watches how AI assistants talk about a law firm. When someone asks ChatGPT, Gemini or Perplexity to recommend a lawyer in their city, the answer they get is now part of that firm's reputation, and until recently nobody was measuring it. The tool asks each model the same questions repeatedly and reports what proportion of the time the firm shows up, where in the answer, and how it is described relative to named competitors.
The question
A firm can rank first on Google and still be invisible, or actively misdescribed, when a client asks an assistant instead. How do you measure something whose answer changes every time you ask?
Rough sketch
Where things sit on the screen, and in what reading order. Drawn, not screenshotted, so it stays true when the styling moves on.
Firm overview
Model detail
The problem
Clients increasingly ask an assistant before they ask a search engine. For a law firm, what the assistant says back is now part of the firm’s reputation, and nobody was measuring it.
SEO has twenty years of tooling. Generative engine optimisation has almost none. A firm can rank first on Google and still be invisible, or actively misdescribed, when someone asks ChatGPT for a recommendation in their city.
What it does
Tracks how ChatGPT, Gemini and Perplexity describe a given firm relative to named competitors: whether it gets mentioned at all, in what position, in what framing, and how that moves over time.
Why it is harder than a rank tracker
Model output is non-deterministic and unversioned. The same prompt returns different text run to run, and the underlying model changes without notice or changelog. A single measurement is noise.
So the product only means anything if it samples repeatedly and reports a distribution rather than a rank. “Mentioned in 7 of 10 runs, usually third” is a real answer; “ranked third” is a fabrication. Getting that right is most of the work, and it is the part a naive version of this product gets wrong.
Stack
Next.js and TypeScript with Supabase for storage and scheduled sampling, deployed on Vercel.
How it is put together
Left to right, the path a request or a record takes through the system.
01
Prompt set
- Firm and competitor names
- City and practice-area framing
- Held constant across runs
Fixed, so runs are comparable over time.
02
Sample
- ChatGPT
- Gemini
- Perplexity
- N runs per prompt
Repeatedly, not once.
03
Extract
- Mentioned or not
- Position in the answer
- Framing and sentiment
- Competitors named alongside
04
Aggregate
- Mention rate across runs
- Typical position
- Movement over time
A distribution, never a rank.
What happens, step by step
In plain language, in the order it happens.
Fix the questions
A firm, its named competitors, a city and a practice area become a fixed prompt set. Changing the questions between runs would make the time series meaningless, so they are held constant.
Ask every model, repeatedly
Each prompt goes to ChatGPT, Gemini and Perplexity, many times. One measurement of a non-deterministic system is an anecdote.
Read each answer
For every response: was the firm mentioned at all, in what position, in what framing, and who else appeared alongside it.
Report a distribution
Mentioned in 7 of 10 runs, usually third is a real answer. Ranked third is a fabrication, and it is the answer a naive version of this product would give.
Track the movement
Because the models change underneath without notice or changelog, the useful output is the trend across scheduled runs rather than any single reading.
Decisions, and what they cost
Every choice worth recording has a road not taken. Both are here.
Sample repeatedly and report a distribution
instead ofOne query, one rank, like an SEO tool
Model output is non-deterministic and the underlying model is unversioned. The same prompt returns different text run to run, and the model itself changes without a changelog. A single measurement is noise presented as a number. This is the whole product; getting it wrong makes everything else decoration.
Scheduled sampling on a server
instead ofOn-demand queries from the browser
The value is in the time series, which requires runs to happen whether or not anyone opens the dashboard. It also keeps model credentials off the client.
Track named competitors explicitly
instead ofAbsolute visibility scoring
There is no absolute scale for how visible a firm is inside a language model. Relative to a named set of competitors is a question that has a defensible answer.
Supabase for storage and scheduling
instead ofRolling my own queue and cron
The interesting problem here is measurement design, not infrastructure. Every hour spent on a job runner is an hour not spent on the sampling methodology, which is the part that decides whether the numbers mean anything.
What came out of it
- Live, tracking three assistants against a fixed prompt set.
- Reports a distribution: mention rate and typical position, rather than a fabricated rank.
- Built in a category with almost no tooling: SEO has twenty years of instruments, generative engine optimisation has close to none.
Still open
- Model versions are unobservable from the outside, so a shift in the numbers cannot yet be attributed to a model change versus a real reputation change.
- Sample sizes are a cost decision. There is a real trade between statistical confidence and per-run spend that is currently set by judgement.
Built with
- Next.js
- TypeScript
- Supabase
- LLM orchestration
- Vercel