Grok vs GPT partisan bias in election recommendations study

Convovo's test forced Grok 4.7 and GPT-6.1 Sol to name a single winner in nearly 500 upcoming midterm races, with no hedging allowed. Grok named the Republican candidate in 92% of its answers, GPT named the Democrat in 97% of its answers, and the two models agreed on the same candidate less than 10% of the time ^[1]. Neither model treated local political reality as a limit: GPT picked the Democrat in 57 of 67 heavily Republican districts, and Grok picked the Republican in 47 of 72 deep-blue districts ^[1], meaning each model's pick tracked its own internal pattern more than the makeup of the race itself.

The researchers isolated where that pattern comes from by rerunning a 30-race subset with live web search switched off. GPT still picked the Democrat in 24 of 30 races relying on training data alone, while Grok refused to pick a candidate at all in 27 of 30 races without search ^[2]. Parikh summarized the split as a difference in mechanism, not just outcome: "For GPT, it has mostly decided already...it's in the weights. For Grok, it really is the search pattern - the terms and the sources - that determine the result" ^[1].

If Grok's bias comes from search rather than training, the obvious next question is what, exactly, it's searching for and finding. Grok's answers drew on a Christian conservative voter guide in 39% of its responses, a Turning Point Action-affiliated site in 23%, and X itself in 41%, alongside nonpartisan references like Ballotpedia, OpenSecrets and VoteSmart. GPT, by contrast, leaned heavily on Wikipedia, the Associated Press, Axios and Reddit - outlets with which OpenAI has existing content licensing deals ^[3].

The two models were also asking different questions of the same races. GPT more often searched for candidates' positions on healthcare and abortion, while Grok focused on immigration, taxes, crime and energy ^[3]. That is not a small detail: if a model's eventual pick depends on what its search step surfaces, two models asking different questions of different sources will keep landing on different answers even when they are nominally evaluating the same candidates.

Separate from the search-versus-weights finding, Parikh's team discovered something odder sitting inside Grok's API traffic: every request carried roughly 1,240 tokens of extra input that the researchers never wrote and could not see or remove. Parikh called these potentially "hidden instructions," and said they are unlikely to originate from the model's weights, though their source remains unexplained ^[1].

That distinction matters because it sits outside both explanations offered for the bias itself - it is neither a trained-in lean nor a byproduct of which websites a live search happens to surface. It is an opaque addition to the request that neither the researchers nor, based on the available reporting, xAI itself has publicly accounted for, leaving open the question of what else might be added to a model's input without a user's or even a researcher's visibility.

The study's forced answers came from querying Grok's and GPT's developer APIs directly, not the consumer chat apps most people actually use; those consumer apps refused to name a candidate at all. An OpenAI spokesperson said that distinction is significant precisely because the consumer app, not the API, is how most people engage with the chatbots, while maintaining that "ChatGPT is designed to be objective by default, including in political conversations, and shouldn't have political bias in any direction" ^[1].

That still leaves a gap for anyone building on top of these models' raw APIs, outside the consumer guardrails, who could surface the same partisan recommendations the study found. Parikh separately warned that the risk isn't limited to someone directly asking who to vote for: the same leanings could surface in broader political conversations with these models. That concern echoes a separate finding from the Center for AI Safety, which has found that large language models can covertly steer users toward political positions through information selection, word choice, framing and sourcing ^[4]- the same levers this study caught Grok and GPT pulling in opposite directions.

The app-versus-API gap was also the detail that online discussion of the study kept circling back to, alongside a competing theory about where each model's lean actually originates: that GPT's pattern reflects the Reddit-heavy data it trained on, while Grok's reflects X data and post-training choices tied to its parent company. Reaction skewed more mocking toward Grok specifically, but real, pointed skepticism landed on both companies, including questions about whether the forced-choice design itself would reproduce consistently.

The forced binary choice used in the Convovo study is not the only way researchers have tried to measure AI political bias, and other methods have produced a milder picture. A Chalmers University of Technology and University of Gothenburg study that generated over 148,000 AI responses across six models, including Grok and GPT, on 107 Swedish policy issues found GPT had "the most neutral and balanced outputs," while Grok "deviated notably toward right-leaning positions" on migration, crime and gender - though after statistical correction, no model showed a significant bias toward any of Sweden's eight parliamentary parties ^[5].

That contrast matters for how to read the Convovo numbers: a forced, binary, US-election-specific prompt that strips out hedging produced a far starker partisan split than a broader policy-issue survey did, even with some of the same models involved. A single study's framing should be read as describing that framing's result, not a model's politics in the abstract.