Civic AI Audit

Method notes

How we analyzed the answers, and why counts differ from page to page. For the overview, read How we work.

Figures cover answers captured before 6 pm ET on Nov 3, unless noted. No person has checked them yet.

Analysis

Six races carry over half of the daily links

What happened. We asked 14 kinds of question about six statewide races every day from Oct 10, on four APIs. Those six races are 53% of all daily cited links, though there are 123 races. The daily question set for those six alone is 47% of them.

What it affects. Pooled source figures lean toward those six races. Weekly trends use only this six-race set, so they don't describe all races.

Local news share of cited linksShare
All daily links, pooled (published)39.0%
Every race counted equally36.5%
Six races, average41.1%
The other 117 races, average36.3%

Local news stays the largest type either way. The order below it changes: "other news and wire services" is next when pooled, while reference sites are next when every race counts equally (13.5%).

What we did. We kept the pooled figure and label six-race figures as such.

Published figures. None change. The 39.0% depends on this weighting.

Gemini's Facebook links fell in mid-October

What happened. On the same 12 questions, Gemini's API cited Facebook in 81% of answers before Oct 16 and in 3% from Oct 22. On the all-race comparison, the share went from 79% of answers in one week to 1.4% in late October. Google's AI Overview held at about 37% on the same 12 questions.

What it affects. Any chart of social media or Facebook over time. Social media's share of daily cited links fell from 6.9% before Oct 16 to 3.1% from Oct 22. Without Gemini, it stayed about level, at 0.7% and 0.9%. The sources page shows this fall as a share of Gemini's links by week, so its figures differ from these.

What we did. We checked that this wasn't a problem in how we store links. Gemini's total links held steady, its stored link titles show the same fall, and few links went unresolved. Local news and other news sites rose as Facebook fell. Answer length and links per answer stayed about the same. We have not compared what the answers said before and after.

Published figures. None change. Totals pool the periods before and after the drop.

Cause unknown. Gemini's reported model label didn't change, but that label can point to different versions over time. This describes the API, not the Gemini app.

Labels come from AI models

What happened. A Claude model (claude-haiku-4-5) sorted pick answers into outcomes, in one pass and reading the first 6,000 characters of each answer. The same model, along with fixed rules, sorted websites into types. 97.7% of cited links rest on its website labels.

What it affects. Every pick and source figure.

What we did. A separate Claude session read selected answers and corrected 16 pick labels. It also read all ChatGPT web answers labeled as naming a candidate. A review corrected 135 website types, covering about 15,000 cited links, which moved local news up about one point. Of these, 120 came from an audit by that session, and 15 from a rule for public-media stations and a check of state outlets. Where these notes say an answer was "read", the reader was this AI session, not a person.

Published figures. They include these corrections. No person has validated the labels yet.

Cause of remaining error unknown. We can't give an error rate until a person has checked a sample.

How the counts differ

Total answers

The homepage total of 86,100 answers counts what we recorded before the first polls closed on Nov 3. It leaves out 726 captures that held no answer.

It includes 5,558 answers from June and July to older civic facts questions, such as "Who are my U.S. senators in Virginia?" These have no links. The other 80,549 answers, from Oct 4 to Nov 3, are election questions.

The homepage counts 691,600 cited links, from all our questions. The sources page counts 363,500, from our daily question sets only.

We recorded 1.7 million links in all. The others, such as pages a chatbot consulted but did not cite, are not in these figures.

Races

Source figures cover 123 races. The pick experiment covers 113: the Senate, governor and House contests in our question set with one Democrat and one Republican on the ballot. Web versions were asked daily questions in 6 of the 123 races. A wider question set from the night of Nov 2 through Nov 3 covered more contests, and it counts only toward the homepage totals.

Pick charts

Pick charts count the answers in each completed run, by version. A version appears only in the runs it completed, and the chart says so. The Nov 3 afternoon round was partial, so it is not in the charts. It is in the homepage total.

How chatbots are weighted

In the chart of what each chatbot cites, and in the largest category, every link counts once, so chatbots with more links weigh more. 96% of those links come from developer versions. The table of most-cited sites and the state map use developer versions only and count each one equally. The same site or category can show different shares in each.

Version comparisons

The web-versus-developer and free-versus-paid comparisons pair answers to the same question on the same day, one service at a time. They were calculated separately, as of November 3, 2026, so they may not match the other charts.

Rounding

Totals in sentences are rounded to the nearest hundred, or the nearest tenth of a million. Most percentages are whole numbers, so they may not add up to 100.