Professor of Mathematics Steven Miller submitted a sworn court declaration on Nov. 21 that suggested the possibility of serious ballot irregularities in Pennsylvania. Over the previous week, Miller had written several iterations of the declaration, which first circulated on conservative media on Nov. 17. Since then, President Donald J. Trump amplified the statement in a tweet, the Chairman of the Federal Elections Commission (FEC) referenced the statement as indicative of fraud, and a conservative group prominently featured it in a legal brief seeking to overturn the Pennsylvania election results. 

Miller’s analysis, which interprets survey results collected by conservative data analyst Matt Braynard, suggests that tens of thousands of registered Pennsylvania Republicans may have received absentee ballots despite not requesting them, and tens of thousands more may have returned their absentee ballots but did not have those ballots counted. However, a number of statisticians and political scientists have denounced the statement as deeply flawed and misleading, and the underlying data upon which the analysis rests as unreliable. 

Among the statement’s critics is Associate Chair of Statistics Richard De Veaux, who is also the vice president of the American Statistical Association. De Veaux described Miller’s document as “completely without merit” and “both irresponsible and unethical” as part of a longer rebuttal that also accused Miller of violating at least seven out of the 10 guidelines for ethical statisticians laid out by the American Statistical Association. 

Miller’s analysis was also strongly criticized by George Marcus, professor of political science emeritus at the College, and Charles Stewart III, professor of political science at the Massachusetts Institute of Technology (MIT). 

All three professionals found Miller’s calculations themselves routine and uncontroversial, but took serious issue with Miller’s assumption that the data that he analyzed was accurate and representative. They pointed to the fact that the survey’s response rates were low, that the survey only reached out to registered Republicans and that voters often tell pollsters untrue statements on their voting history.

“I agree completely that this could have been done better and welcome the opportunity to help others not make these mistakes,” Miller told the Record in response to the criticism he has received. Miller also apologized via email to community members on Monday, saying that his dependence on the accuracy and reliability of the data he analyzed should have been spelled out more clearly. 

Miller did not recant his decision to conduct the analysis, and wrote in the email, “It is worthwhile to know how well [election] systems worked, which is why I felt it was important to do this calculation.” In other portions of the email, he adds that his analysis does not definitively state whether fraud occurred, and his results are at most suggestive.

Miller made his original declaration in an individual capacity and not on behalf of the College, as both he and Chief Communications Officer Jim Reische noted. “Williams faculty are free to pursue political and other personal interests outside of work without college interference,” Reische said. “Steven Miller doesn’t claim college endorsement of his election analysis, nor would we grant it.”

Miller’s report coincides with the Trump campaign issuing a string of lawsuits challenging President-elect Joseph R. Biden Jr.’s victory, many of which are riddled with factual errors and the vast majority of which have been struck down in court. Trump’s lawsuits have challenged absentee ballot procedures and claimed that poll watchers were not given close enough access to the vote count, among other allegations, and have often demanded that certification be delayed or some votes be excluded from the tally. Trump has used these lawsuits to justify baseless attacks on the democratic process, though Pennsylvania and multiple other states have certified their results in recent days. 

The statement’s origins and methodology

Miller first wrote the statement after being approached by a volunteer attorney to the Trump campaign litigation effort, Peter Haller of Hill Strategy LLC. Miller said Haller contacted him after Miller had previously conducted a variety of independent and non-public analyses on potential voter fraud in the weeks after the 2020 election. 

After hearing about Miller’s work, Haller asked him to analyze survey data gathered by Matt Braynard, a former Trump campaign data director and pollster. 

“Dr. Miller’s reputation of high academic integrity caused my interest. I had confidence that he would evaluate data without bias,” Haller told the Record. He added that he hopes to see Miller’s statement used in litigation to “help to evaluate what happened in this election.” 

In his interview with the Record, Miller said that he was willing to conduct analysis for anyone. “My motivation is to do the analysis and find out what’s going on,” he said. “If it’s useful to anybody on either side, I am willing to share.” He added that he welcomed his analysis appearing in court if it was useful to either party. His primary motivator, he said, was intellectual curiosity and a desire to discern whether or not substantial voter fraud occurred. He said he was not paid for any of his work. 

Miller is a self-described “conservative libertarian,” but claims that his politics played no role in his decision to write the document. 

Miller’s report performs direct analysis on data collected by Braynard, who interviewed a number of registered Republicans in Pennsylvania who requested absentee ballots but did not ultimately return them, according to official state records. Braynard had previously launched a nationwide “Voter Integrity Fund” that is staffed with former Trump campaign and administration affiliates, and called voters across the country to ascertain the likelihood of voter fraud. According to the Philadelphia Inquirer, Braynard’s attempts to fund the effort through GoFundMe was shut down for violating GoFundMe rules by trying to “spread misleading information about the election.” 

In his initial analysis, Miller extrapolates from Braynard’s data that over 40,000 Republicans most likely received an absentee ballot without requesting it, and over 40,000 Republicans likely returned absentee ballots that were not counted. Miller’s analysis rests on the assumption that Braynard’s underlying survey data is fully accurate and representative, which multiple experts in statistics and survey research found to be extremely unlikely for a litany of reasons.

De Veaux, Marcus and Stewart all pointed out that Braynard’s data is likely flawed in a number of ways, including that response rates were relatively low, that Braynard only reached out to registered Republicans and that voters often tell pollsters untrue statements on their voting history. There is also evidence that many voters in Pennsylvania did not realize they requested an absentee ballot, which could explain much of Braynard’s results. Based on these uncertainties, all three found Miller’s report to be likely inaccurate and therefore irresponsible and unethical to publish.

Braynard dismissed concerns that his survey design could introduce nonresponse bias and that voters might make untrue statements. In an interview with the Record, Braynard complimented Miller’s work and denied working to assist the Trump campaign, saying that his only goal was to evaluate the election’s “legitimacy.”

In Miller’s first sworn statement, which he signed under penalty of perjury on Nov. 17, he stated his conclusions without significant caveats regarding data quality, and his report erroneously stated the source of his data. Miller initially stated that his data came from “Matt Braynard and his firm, Election Data Services.” However, Braynard has no current affiliation with Election Data Services, and when the Record reached out to Kim Brace, the president of Election Data Services, he was unaware of Miller’s report and said that Braynard was terminated from the firm over 12 years ago.

Miller later claimed that this iteration of his sworn and signed statement was not ready to be released and was leaked by an unknown entity. In a later version of the statement, which Miller estimated had been through at least six drafts, he repeatedly includes the caveat that his conclusions are “based on the data I received being accurate and a representative sample of the population,” and removes all reference to Election Data Services. This is the version of the statement that has appeared as evidence in a Pennsylvania courtroom. 

The statement’s reach

Miller’s analysis was prominently featured in one Trump campaign lawsuit in Pennsylvania that was ultimately dismissed, in which plaintiffs sought to overturn the state’s election results. On Nov. 21, a group of Republican politicians and activists submitted a motion to intervene in Donald J. Trump for President v. Boockvar, in which Miller’s statement is listed as Exhibit A. Describing Miller as one of two “expert witnesses” featured in the case, the brief writes that “Such experts will identify significant and dispositive discrepancies/error or misconduct which would call into questions [sic] the results of the Presidential Election in Pennsylvania.”

U.S. District Judge Matthew W. Brann dismissed the case, writing that it used “strained legal arguments without merit and speculative accusations.” The Trump campaign has appealed the decision to the U.S. Court of Appeals for the Third Circuit. 

Another version of Miller’s statement was cited in numerous far-right media outlets, including Just the News, the Epoch Times and the Virginia Star. Just the News was the first publication to reference Miller’s statement, under the title “In sworn statement, prominent mathematician flags up to 100,000 Pennsylvania ballots.” The article was co-written by John Solomon, who previously spearheaded conspiracy theories regarding Joe Biden and Ukraine.

In the article, Just the News quotes Chairman of the FEC Trey Trainor as saying, “This data, which is provided by an expert witness, who would be qualified in almost any court in the country, adds to the conclusions that some level of voter fraud took place in this year’s election.” Trainor, the top elections official in the country and a Trump appointee, did not respond to the Record’s request for comment.

Another Just the News article citing Miller’s research was referenced favorably by Trump, who retweeted a post linking the article and added “Thank you!”

Widespread criticisms of the statement from political scientists, statisticians

Charles Stewart III, a professor of political science at MIT, found numerous flaws with Miller’s approach. Stewart is a self-described specialist in “elections forensics” who is the MIT director of the Caltech/MIT Voting Technology Project, and estimates that he has testified a dozen times in state and federal court relating to elections administration. 

In a phone call with the Record, Stewart expressed that the issue with Miller’s work was not his statistical calculations themselves, which he found relatively rudimentary and noncontroversial, but the underlying assumption that Braynard’s data is accurate and trustworthy.

Stewart emphasized that voters often do not tell the truth in responses to surveys, especially on whether and how they voted. In addition, surveyors who attempt to call the voter rolls do not always reach out to the correct phone numbers, which can jeopardize their findings.

Lastly, Stewart expressed his belief that the original survey data was collected by Braynard in bad faith with the intent to prove fraud, rendering the data suspect. 

“They’re trying to find evidence of fraud,” he said. “And they are not being careful to do things in a scientifically valid way. And there’s no evidence that any of them know how to do survey research in a scientific, valid way.” 

Miller responded to a similar critique in an interview with the Record by arguing that he was simply performing a basic mathematical analysis under clearly defined assumptions, and did not claim his statement to be conclusive but rather potentially indicative. 

“I am a strong believer that sunlight is the best disinfectant, that the best solution is to put things out there in the public,” Miller said. 

Nevertheless, Stewart found Miller’s involvement in analyzing the data set itself to be problematic. 

“I don’t know why you need a Yale and Princeton trained mathematician to calculate proportions and standard errors,” he said. “… Bringing in a highly credible, highly capable mathematician, to do calculations that thousands of survey researchers do every day — the only reason why you would do that is to lend credibility to a study that otherwise has no credibility.”

Williams Professor of Political Science Emeritus George Marcus reached similar conclusions after reviewing Miller’s original draft statement that circulated online. 

“As written it has a number of problems before it would be credible for anyone [who] is motivated to ask questions,” Marcus wrote to the Record. “It is meant for the credulous.” He pointed to what he saw as artificially precise numerical conclusions, the use of “shouting” red font, and declarative statements made without statistical caveats, before discussing at length the methodology behind Braynard’s polling.

“Who made the calls? How many? What was the training of these individuals?” he asked. “I can infer that no such efforts at achieving high quality data were made… All these elements can generate a sample that is very likely to be biased and unrepresentative.” 

Professor of Statistics Richard De Veaux expressed many of the same concerns as Marcus and Stewart.

“Most of the public will breeze past the caveats about assumptions and focus on the sensational and ridiculous conclusion, especially since [Miller] has been paraded as a ‘respected mathematician,’” De Veaux wrote. “This would be a comical mistake if it didn’t have real world consequences.”

De Veaux forwarded to the Record a conversation that took place among a number of academics and professors on a statistics-dedicated listserv. The conversation centered around Miller’s work, and was largely critical. Individual comments included “I am genuinely shocked that a trained mathematician would write this under penalty of perjury” and “Did he have a stroke before doing this analysis?”

Not all trained mathematicians took issue with Miller’s work, however. Professor of Mathematics Stewart Johnson wrote to the Record of the document, “This is well within his area of expertise, and he exercised due diligence in his analysis, including pointing out potential issues with the data. That this is controversial is a symptom of our highly politicized climate.” 

“I should have made a greater effort to go deeply into and share how the data was collected”

In his apology, Miller emphasized that he should have approached Braynard’s data with more skepticism. “Especially as the consequences are so important, I should have made a greater effort to go deeply into and share how the data was collected and not treat this solely as an independent calculation, and I apologize for not explicitly doing the following steps and asking additional questions,” he wrote. Miller also wrote in his apology that recent conversations with Braynard assuaged some of his concerns and made him more confident about the validity of the underlying data. 

“Though [Braynard] does have ties to the Trump campaign in the past, he did assure me that the data collection and results are consistent with other efforts by these national call centers,” Miller told the Record. 

For Stewart, Miller’s decision to analyze Braynard’s data, even when including caveats, effectively constitutes participation in a legal effort that is damaging to democratic principles. “No one has come forward with serious evidence of fraud. And so now, basically, there’s a group of people engaged in character assassination of American democracy,” Stewart said. “… As we can see already, that’s how [Miller’s] research is being used.”