don’t think it has the best reputation,” one expert says. A “disappointment,” says another, a Harvard alumnus. “A bit of a laughing stock,” a current student says.

The Harvard Center for American Political Studies-Harris Poll is a monthly public opinion survey of registered voters released by Harvard’s CAPS, the Harris Poll, and HarrisX, a polling firm. An unusual partnership between a Harvard research center and a private polling firm, the poll is used in government courses and provides data for research. It is also extremely divisive.

A public opinion poll seeks to understand respondents’ opinions on a series of questions. Universities and newspapers conduct public opinion polls to ‘take the pulse’ of Americans and make findings widely available in research journals or online news. These insights inform, and perhaps sway, the general public. Private polling firms conduct public opinion polls, too, but their insights often have more concrete goals: strategizing campaigns, designing advertisements, and substantiating policy arguments. These privately-contracted polls might lead a politician to prioritize a particular campaign promise based on what their voters find most urgent, or to use language that respondents find persuasive. These contracted pollsters’ results are not generally published.

Public opinion polls seek to accomplish an impossible task: to distill nuanced beliefs into hard statistics. It is an imperfect science. If a poll is poorly designed, as critics say the Harvard CAPS-Harris Poll is, it can lead to inaccurate results, misleading politicians and confusing the public. Pollsters tend not to comment upon each other’s polls, but professors, students, and pollsters have criticized it.

That is, except the pollsters who run it.

“I’m not going to say the best,” longtime pollster and political operative Mark J. Penn ’76 says, “but I think we have a super useful poll that a lot of people study very carefully.”

HarrisX Founder and CEO Dritan Nesho ’08 says “we’re one of the most accurate pollsters in the country,” adding that “we’re definitely top-tier.” Alongside government professor Stephen D. Ansolabehere, these men make up the Harvard CAPS-Harris Poll’s co-directing trio.

Most public opinion polls ask similar questions, follow similar methods, and collect similar data. Neutral question wording, probability sampling, and transparent methods have long been regarded as hallmarks of rigorous polling. But in the last 15 years, polls have seen some high-profile misses: most notably, underestimating Donald J. Trump’s support in the 2016 and 2020 presidential elections. The perception of polling accuracy has plummeted. Polling, many have said, is in crisis.

The Harvard CAPS-Harris Poll’s directors imagine their poll to be an innovator, combining the public-facing results of an academic poll with the questions and methods of a privately-run campaign or governance poll. Depending on who you ask, these departures represent a bold approach to understanding voters’ underlying behavior, or a misguided hybrid with damning flaws. A poll experimenting with new ideas, or one shirking decades of best practices. The Harvard CAPS-Harris Poll is different — whether that’s a good thing is, fittingly, a matter of opinion.

Industry vs. Academia

he story of the Harvard CAPS-Harris Poll starts with Harris, not Harvard. It starts in the 1960s with Louis Harris, a prominent public opinion pollster — and it ends with Harris as a “zombie brand,” as one pollster describes it, bringing a credible name best known for news polls to an organization that now conducts most of its work privately.

Researchers, politicians, and newspapers have run public opinion polls for nearly two centuries. First used to predict election results, polls have become ubiquitous because they seek to offer systematic measurements of public sentiment — as opposed to opinions expressed in a newspaper’s editorial pages or shared in public gatherings. In theory, polls can extrapolate how a population feels without talking to its every single member.

Over time, the science of polling has been continuously refined to more accurately capture the public’s opinions. George Gallup developed the method of surveying a sample of people meant to represent a larger population, and his first poll in 1932 correctly predicted a local election in Iowa. Gallup’s polls then expanded to cover a wider range of topics beyond electoral results, including policy issues and proposals, as well as political favorability.

Harris was a polling innovator: an early pioneer in using polling to directly advise political campaigns. He spent nearly a decade guiding more than 200 campaigns, most famously for the successful 1960 presidential campaign of John F. Kennedy ’40. But three years later, Harris determined he would no longer be employed by private political campaigns. Inspired by the Gallup Poll, he took his operation to the public.

He founded the Harris Poll in 1963, polling on everyday issues to take the pulse of the American public. Harris was the chief polling analyst at both CBS and ABC News during the 1960s and 70s, and his poll soon became one of the most well-known in America.

Penn was born into the Harris Poll’s golden era. He conducted his first poll at age 13 in his school cafeteria and brought that passion to Harvard as an undergraduate, where he took courses on public opinion in the Government department and served as The Crimson’s informal resident pollster. After graduation, Penn parlayed his experience into professional work, polling and strategizing for some of the world’s most prominent left-leaning political figures — from President Bill J. Clinton and Senator Hillary R. Clinton to British Prime Minister Tony Blair and billionaire Bill H. Gates.

Over decades in politics, Penn became a prominent campaign pollster. If you don’t know his name, you know his work; he coined the term ‘soccer mom’ and, according to his website, has helped elect more than 25 world leaders in Asia, Latin America, and Europe.

Penn’s career has also been marked by controversy. While polling for Hillary Clinton’s 2008 presidential campaign, he was accused of a conflict of interest after meeting with Colombia’s ambassador to the U.S. and subsequently stepped down from his role as chief strategist. Since then, Penn has become a well-known critic of the Democratic Party and offered political advice to President Donald J. Trump. He came to Trump’s defense in 2018, labeling former FBI Director Robert S. Mueller’s investigation into Trump as a “partisan, open-ended inquisition.”

In the early 2010s, Penn, having stopped day-to-day polling, made two moves in quick succession. First, he returned to Harvard to co-teach with Ansolabehere an undergraduate course modeled on one Penn took as a student. Government 1360: “American Public Opinion” was offered by 2014, giving Harvard undergraduates the opportunity to learn about polling from one of its foremost practitioners and an academic expert in Ansolabehere.

In addition to teaching, in 2015, Penn left a position at Microsoft to found the Stagwell Group, a private investment firm that has since expanded into marketing and market research under the name Stagwell Global. With Penn as CEO, it now controls several dozen companies. The conglomerate lends its services to companies like Amazon and Nike, sometimes conducting opinion polls and “message tests” that test the effectiveness of marketing campaigns. Stagwell also received scrutiny last year after news reports on leaked documents showed that Stagwell had performed research for Israel’s Ministry of Foreign Affairs to test messaging about the country’s international image.

Of Stagwell’s many subsidiaries, the crown jewel is the Harris Poll.

By the time Stagwell acquired the Harris Poll, it had been bought and sold by a series of private companies. Its owners had increasingly used it for private research, even as the brand remained widely recognized. Penn saw an opportunity. “I thought that it would be great to have the Harris Poll and its history,” he says. In January 2017, Stagwell purchased the Harris Poll, and Penn became its chairman.

Meanwhile, in the “American Public Opinion” course, Penn and Ansolabehere were contracting with outside panels to field student-designed surveys and collect data for in-class analysis. They envisioned a recurring, in-house poll that could also support the course. “We said, ‘Okay, let’s create a poll, a monthly poll that would be better than the polls out there, that would inform people,” Penn remembers. “We would just dump all the data and information online. Anybody could look at it. Anybody could analyze it.”

After Penn purchased the Harris Poll, Dritan Nesho, a longtime Penn employee, founded a sister company to the poll. HarrisX conducts more advanced data analytics, often collaborating with the Harris Poll on research. Penn offered the services of both HarrisX and the Harris Poll to CAPS, and the Harvard CAPS-Harris Poll was born.

Since debuting in February 2017, it has generally been published 11 times per year. Key results and raw data for more than 100 of its polls are publicly available on the Harvard CAPS website. Its results have been featured in the New York Times, the Washington Post, and the Wall Street Journal. In a 2020 post-election analysis, the Washington Post ranked the Harvard CAPS-Harris Poll as more accurate than polls from NBC News, SurveyUSA, and YouGov, among others.

Together, Penn, Ansolabehere, and Nesho comprise the Harvard CAPS-Harris Poll’s three directors, each with their own expertise. Penn has the hands-on experience with campaign polling and business. Ansolabehere offers the academic background. Nesho, twenty years their junior, is the self-described “millenial pollster,” bringing modern polling techniques to the collaboration. All three contribute to “American Public Opinion” — Ansolabehere and Penn as professor and lecturer, respectively, and Nesho as the teaching associate who administers the students’ surveys. Though students’ questions do not appear in Harvard CAPS-Harris Poll itself, as part of the partnership, students in GOV1360 field their surveys through HarrisX.

In the class, students develop a survey from start to finish. The assignment is to conduct a campaign-style poll; this semester, the class’s four sections are designing four sets of poll questions that will gather information from the public to be used by Pennsylvania Governor Josh D. Shapiro, President Trump, Vice President JD Vance, and House Minority Leader Hakeem S. Jeffries (D-N.Y.). They began workshopping questions for their sections’ surveys in September, and the results they collect will culminate in a paper advising their politician.

The Harvard CAPS-Harris Poll, like students’ polls in Gov 1360, is modeled more like a privately commissioned political poll than a traditional academic survey. “If you read through past polls done by anybody who’s consulting in the political world, their polls look more like what we’re doing,” Ansolabehere says.

Despite publishing their research online for academics and others to study, the Harvard CAPS-Harris pollsters are interested in going beyond traditional academic approaches. “Academics tend to have a more anesthetized version of what a poll ought to look like,” Ansolabehere says. Like an academic poll, the trio publishes results online for anyone to see, but the poll’s design pushes past the “anesthetized” questions they associate with academic polling. In their eyes, the results are more practical and actionable for politicians analyzing the results.

When Louis Harris created the Harris Poll in 1963, he told the New York Times that “it has always been my view that major contributions could be made in both the public and private areas of political research, but that the two in the end are of necessity mutually exclusive.”

But though Harvard CAPS-Harris Poll still bears Louis’s name, it combines industry and academic approaches to polling. Through its methodology, the poll strives to reflect those used to support the White House and other political campaigns. As Ansolabehere says, “This is a marriage of practical politics and the academic.”

A ‘Good’ Question

he Tsai Auditorium inside the Center for Government and International Studies seats 146 students, exactly double Gov 1360’s enrollment of 73. But only perhaps two-thirds of the class is present for Penn’s lecture in late September on question-writing, and half of these text friends, shop, or scroll through Instagram Reels as others take notes. The room is silent but for the clatter of keyboards and Penn, lecturing on question-writing.

Polls start with questions, and question writing is a dominant focus of Gov 1630.

Fundamentally, a poll seeks to understand respondents’ opinion on something — whether about an issue, a message, or a person. If a question sways, confuses, or otherwise confounds respondents’ true beliefs, the question will not capture public opinion of that something.

“There’s a lot of what I call leading questions out there,” Penn says in class. There is extensive literature on what constitutes a ‘good’ poll question, and no shortage of experts willing to join the discourse. Pollsters typically agree that questions should not be leading — pressuring the respondent to answer in a certain way. They should also not be double-barreled — asking about multiple topics in one question — or suggest a false binary, forcing a respondent to pick between only two extreme options when others exist.

Using these guidelines, pollsters try to construct questions that accurately capture trends — whether in voting intentions or underlying beliefs. One of Penn’s lecture slides states that questions should consider three elements when writing a question: “what you want to know, what the respondent actually knows, and what you might do to lead the respondent and get a false result.”

The Harvard CAPS-Harris Poll’s questions can be roughly split into two types. The first series of questions have been consistent throughout the poll’s lifespan, asking staples such as a president’s favorability, the outlook of the economy, and approval of both political parties. The question wording is fairly standard across major polls.

Where the Harvard CAPS-Harris poll differentiates itself is in its issue-based polling, which usually makes up anywhere from a third to half of its published results. These questions respond to current events or hot political topics, testing the appeal of ideas or language in a similar way that a campaigning politician might hire a pollster. They also receive much of the poll’s public attention and, in Penn’s eyes, fill a void left by other polls. “I know that people look to see how we’ve analyzed the issues and learn from it in just the way, frankly, I intended it,” he says.

One question from the August poll reads, “Do you think that the Democratic Party should embrace or reject people like Hasan Piker who have said things like America deserved 9/11?” A similar question follows about whether the Republican Party should do the same for people like conservative commentator Candace Owens “who have said things like the Jews are behind everything.” (Piker, a left-wing online political commentator, said in August 2019 that “America deserved 9/11” in a critique of U.S. foreign policy and Middle Eastern intervention. He has since apologized for his use of the word “deserved.”) Respondents could not answer that they didn’t know; they were forced to choose between “embrace” and “reject.”

Daniel Cassino, a Government professor at Fairleigh Dickinson University and executive director of its poll, says that the question conflates Piker and Owens with one of their statements and wrongly assumes the average respondent will know who they are. As a result, respondents may react less to the question’s subject — “people like Hasan Piker” — than the information — “who have said things like America deserved 9/11.”

“I would never ask that question,” he says, calling it “ridiculous.”

Academics consider it best practice for pollsters to ask respondents whether they know a person before asking about their favorability. “You can’t ask favorability or unfavorability or ask attitudes about someone if people don’t know who the heck the person is,” Cassino explains. The Harvard CAPS-Harris Poll sometimes asks respondents this, but not always, and not in this case.

Question wording has been the most frequent critique of the poll in the media. A 2023 Semafor article questioned the following: “Do you think that the long-term answer to the Israel-Palestinian dispute is for Arab states to absorb the Palestinians, for there to be

two states, Israel and Palestine, or for Israel to be ended and given to Hamas and the Palestinians?” From the same poll, Reason Magazine suggested the question “Do you think that Jews as a class are oppressors and should be treated like oppressors, or is that a false ideology?” presented a false dichotomy.

In 2024, an op-ed in The Crimson written by Government concentrator Maya A. Bodnick ’26, a former Crimson Editorial editor, called for the University to “break up” with HarrisX, citing leading questions that regurgitated conservative talking points. The long-winded example drawing the author’s ire:

“Donald Trump has recently called on the Senate not to pass a compromise bill on immigration and border security not [sic] to give the Biden administration any wins on an issue that he says they failed to do anything about and failed to enforce the law for over three years. The bill would allow up to 5000 migrants to come in each day before new [sic] measure kicked in and Trump says Republicans can solve the problem better after voters give them a mandate in the election. Do you think Trump is right and support his position against a compromise bill on immigration or is he standing in the way of solving a major issue?”

Penn doesn’t deny that his questions look different from most academic polls. In fact, he says that many polls’ ‘neutral’ language has really defaulted to terms used by Democrats, and as a result, pollsters “didn’t understand why half the country was Republican.” Harvard CAPS-Harris, in his mind, asks questions with both Democrat and Republican slants to “see what resonates.”

Ansolabehere says that questions like the Piker one can be used for just this purpose: “Who will say yes to a somewhat extreme position? We’re informing them of a position, so we’re saying here’s a message — would you support that message?” Questions that range “from super left to super right in their orientation,” Ansolabehere says, are “way more efficient” in determining beliefs.

In response to the August poll, 77 percent of respondents suggested that the Democratic Party should reject people like Hasan Piker, while 23 percent said that they should embrace people like him. Respondents could not opt out. By party affiliation, 79 percent of Republicans, 72 percent of Democrats, and 80 percent of independents and others responded in favor of rejection.

The poll’s summary of the Piker and Owens questions stated that “Voters are united across party lines that both Democrats and Republicans should reject extremists like Piker and Owens.” Headlines picked up the conclusion. Dave Rubin, a conservative political commentator, published an article stating that “even a huge number of Democrats reject Hasan Piker,” while the Times of Israel posted “US voters overwhelmingly reject anti-Israel figures Hasan Piker, Candace Owens.”

To some in the media, such as political reporter Mark E. Halperin ’87, the poll’s results are insightful. Halperin was involved in overseeing polling at ABC and Bloomberg, and he has repeatedly featured Penn and the Harvard CAPS-Harris Poll on his newest media venture, an online media platform called 2WAY.

The Harvard CAPS-Harris directors “think creatively about where the country is,” Halperin says. “A lot of the polls I read, every question is predictable, every question is derivative of static thinking, but they really focus again on original questions and issue-based questions as opposed to political questions.”

The issue-based questions are original, and crucial to how Penn imagines polling: a vehicle for issue-based inquiries that probe voters’ underlying views. “All of these polling operations now are mostly geared towards projecting the horse race,” Penn says. “We’re geared towards trying to understand what is underneath the horse race.”

Adjusting To Reality

fter a poll is finalized, pollsters start gathering respondents. This is called the sample: the group of people who, collectively, reflect the population whose opinion pollsters are attempting to ascertain. A good sample can be generalized to represent the full population; if a pollster only sampled Harvard students, for example, those results wouldn’t speak for the opinions of everyone in the country.

For most of polling’s history, public opinion pollsters have randomly selected respondents to ensure the sample is free from any biases researchers might introduce. Techniques such as random-digit dialing or address-based sampling in particular geographic regions were standard for much of the 20th century. But as answering the phone has become less common, it has become more difficult and expensive to reach a random group of people.

In the last 25 years, some polls, including the Harvard CAPS-Harris Poll, have turned to a faster, cheaper way of recruiting respondents for their survey: online opt-in panels. Respondents self-select into filling out surveys for small amounts of compensation. These panels can be run by pollsters, but are not necessarily; numerous third party organizations exist to solicit respondents for a given survey, or maintain a consistent base of respondents from survey to survey.

Although the specific composition differs month-to-month based on the poll’s needs, Harvard CAPS-Harris tends to purchase respondents from four to six online sample providers, and respondents are collected over a two to five day period. According to a HarrisX spokesperson, the poll uses several sampling companies, including Lucid, Pure Spectrum, Mfour, Dynata, Cint and Stagwell’s own panel, Unlock Surveys.

Opt-in panels offer a fast way to recruit large numbers of respondents. But there are a number of downsides to these panels that bring the technique’s reliability into question.

A Pew Research Center review compared the accuracy of opt-in surveys versus those conducted with random samples. Eleven studies concluded that opt-in panels were less accurate, while Ansolabehere co-authored one of the two studies that found that opt-in surveys were just as accurate.

One concern is that people may quickly click through multiple choice questions to get the compensation without taking the time to select the most accurate answers. Pollsters or third-party panels generally attempt to screen for response quality, but screening isn’t infallible. For example, when Pew Research Center conducted a 2022 study on bogus respondents that asked respondents whether they were licensed to operate a nuclear submarine, 5 percent of respondents said yes. In reality, nearly zero percent of Americans are licensed to operate nuclear submarines. The “bogus responses” were particularly severe from young and Hispanic respondents — roughly a quarter of Hispanics and 12 percent of people under 30 years old claimed they were licensed.

Even if bogus answers are controlled for, poll experts disagree on whether opt-in panels can be treated as representative.

Stanford social psychologist and Harvard alum Jon A. Krosnick ’80 is a vocal critic of non-random sampling. A co-founder of the Stanford Institute for Excellence in Survey Research, he’s also a strong voice in survey methodology, particularly sampling. Krosnick authored two of the studies in the Pew Report’s analysis that found opt-in panels to be less accurate than probability samples. In Krosnick’s description, the institute “identifies and disseminates best practices in survey research, data collection, analysis, and interpretation.”

Krosnick is not a fan of the Harvard CAPS-Harris Poll. He was a co-author of a 2023 San Francisco Chronicle op-ed criticizing its methods and calls the poll a “disappointment.” Although he takes umbrage with many of the poll’s tactics, he is most vehement about its use of opt-in panels, which he calls “fatally flawed” and scientifically “bankrupt.” According to him, randomness is required for representation, and inviting respondents to the poll biases data because the ‘type of person’ who responds to a survey is itself part of a fundamentally, irredeemably unrepresentative slice of the American populace.

The Harvard CAPS-Harris Poll is not alone in its use of opt-in panels. In 2023, Pew reported that, of 78 national polling organizations analyzed, the number using opt-in panels alone or alongside other methods rose from 10 to 47 between 2012 to 2020.

“These methods have been audited and reviewed by many large academic, corporate, and government clients,” a HarrisX spokesperson wrote, “and in addition to the panel controls, we conduct front end screening and back-end quality controls, including in address checks, capturing speeders and straight-liners, using trap questions and a decision-tree algorithm for low attention response patterns.” The spokesperson added that such methods are also a key part of the Harvard CAPS-Harris Poll’s decision to mirror political campaign polls that need to collect results quickly and regularly. “Gathering data over a month or many months using an academic methodology would be out of date and ultimately of little use,” they wrote.

One way that pollsters try to address the limitations of their samples is through weighting.

Whether randomly sampled or opt-in, polls sometimes end up with a sample that doesn’t match the demographic composition of the population they’re seeking to represent. When this is the case, pollsters compensate by adjusting their results using a process called weighting. Answers from respondents of certain demographics are scaled to count more or less than others so the full tally approximates the true demographics of the population. Even if the sample is not random, the statistical argument is that weighting can bring the sample closer to known population benchmarks.

Harvard CAPS-Harris weights its results to approximate the American population on the categories of age, gender, region, race/ethnicity, marital status, household size, income, employment, education, political party, and political ideology.

Many polls weight their samples along most of these axes, often to approximate the race, gender, education, and age demographics measured in the most recent U.S. census. Almost 70 percent of national polls now ask for party identification or reported vote choice, according to the political analysis site Good Authority. This practice has emerged in the last decade to combat growing partisan nonresponse bias — where Republicans are less likely to respond to polls than Democrats.

Krosnick says that weighting can improve the accuracy of a random sample, but with opt-in panels, “most often it actually hurts” a poll’s accuracy. To Krosnick, weighting any group doesn’t help when “it’s kind of arbitrary who we get in the door in the first place.” If Republican, Hispanic, and young respondents — groups that are consistently less likely to respond to polls — aren’t a good sample to begin with, weighting their responses up only amplifies bias.

In the Harvard CAPS-Harris Poll’s results, for most demographic groups, the unweighted base and the weighted base are fairly similar, with one unweighted response counting similarly to one weighted response. The demographic whose proportion consistently changes the most compared to its unweighted tally is Hispanic respondents. In the September poll, 127 unweighted responses from Hispanic people counted as 286 after weighting was applied, meaning one person’s answers counted for more than two. In August, one Hispanic respondent counted for more than three.

Outside of the proportions themselves, one of the categories on which the pollsters weight is disputed: political ideology.

One of the last questions on the Harvard CAPS-Harris poll asks respondents which most closely describes their political ideology: strong liberal, lean liberal, moderate, lean conservative, and strong conservative. (Pew Research Center typically uses a seven-point scale ranging from “extreme left” to “extreme right” to capture ideology.) The poll asks this because a respondent’s party registration and their ideology may not necessarily line up. One can be a conservative Democrat, or a progressive Republican — one doesn’t need to hold certain beliefs to register with a party, though the two are correlated. If a sample collects a disproportionate amount of moderate Democrats, then weighing just by party would allow an ideology bias by accounting for every Democrat the same way. Weighing by ideology theoretically accounts for more nuance.

But in order to weigh for ideology, pollsters must assume a “true” breakdown of ideology within a population, just like weighting for race means scaling to a population benchmark such as Census racial and ethnic data.

“I don’t know what population numbers to use to do that adjustment,” Columbia University government and statistics professor Andrew Gelman says of ideological weighting. Moreover, to Gelman, an additional weight on ideology may not add much value if a poll is already weighing on party affiliation.

The Harvard CAPS-Harris Poll does not publish a benchmark for ideological weighing or the source they use to determine it. In a statement, a HarrisX spokesperson wrote that they believe the U.S. is 24 percent liberal, 38 percent moderate and 38 percent conservative.

“We review the exit polls and determine the go-forward weights for the next cycle of polling,” the spokesperson wrote. “Party ID and ideology are probably among the two most important variables in voting today and so minor changes in these variables wave over wave could suggest non-existent change in public opinion.”

Cassino, who runs the Fairleigh Dickinson University poll, calls weighting based on ideology “really controversial.” In part, he says the problem is that determining a national spread of ideologies is a contested process — though categories like race or gender correspond to a person’s attributes, ideology corresponds to beliefs, and beliefs resist categorization. He argues that weighting by more readily measurable characteristics poses fewer problems.

“Ideological labels don’t mean a whole hell of a lot to most of the people in the public,” Cassino says. “These are just words they use, they don’t mean what we think it means, and people’s own estimation of what it means is wildly out of line.”

The HarrisX spokesperson wrote that they use a “rigorous process” to minimize variations in respondent demographics from one poll to the next. “Ideology is rarely a weight but a full review of the key demographics is done with an eye especially on age, race, region, party and ideology,” they added.

‘Free to Interpret’

n a 2018 interview with NPR, Deputy White House press secretary Hogan Gidley claimed that 80 percent of Americans wanted to close the U.S.-Mexico border, citing a Harvard CAPS-Harris Poll question. NPR later wrote that Gidley had misinterpreted the statistic — confusing wanting to close the border with wanting a secure one. The actual question asked, “Do you think we should have basically open borders or do you think we need secure borders?” 79 percent responded in favor of the later option.

As Gidley pointed to several more statistics, he concluded in the interview with NPR: “And this is a study from Harvard.”

Cassino, the director of the FDU poll, says that it is the responsibility of the person writing the question to consider possible interpretations and write questions that are hard to take out of context. Cassino also says polls should avoid asking questions just to generate headlines — especially when the name of a reputable university is attached to the results. “When you’re affiliated with the university, there is a higher standard you have to hold yourself to,” Cassino says.

Faculty of Arts and Sciences spokesperson James M. Chisholm wrote that “the University, the FAS and the Division of Social Sciences are not involved with the Harris Poll.” Yet Harvard’s name appears wherever the poll is cited.

Penn says most news agencies correctly interpret Harvard CAPS-Harris’s results, but that the data can be reported regardless. “I mean, we’re giving out the information,” he says. “They’re free to interpret it as they will.”

Journalists look at polling data for headlines, but pollsters review each other’s polls as well. Data transparency is an important part of a poll’s reputability. Unless they disclose how they obtained their results, it’s hard to determine what valid conclusions can be drawn. “Accuracy is something that people really look at, but I would say overall among pollsters, transparency is the thing that people tend to value most,” says John G. Cluverius, University of Massachusetts Lowell political science professor and director of survey research at the Center for Public Opinion.

Cluverius is not the only expert interested in transparency. “Lack of transparency is common. Lack of transparency does not necessarily mean malign intent,” says Gelman, the Columbia professor. “Rather, it takes effort to describe what you’ve done, and so lack of transparency typically means that the organization did not put in the effort to write very clear documentation, and that’s the way it goes.”

The American Association for Public Opinion Research, the dominant association for polling, has a transparency initiative to which many pollsters report their survey methods. Numerous polling organizations belong to the initiative, including several cited in this article. Although Harvard CAPS-Harris meets many of the initiative’s requirements, it is not enrolled.

“If they are claiming to be a university-based public polling organization, why are they not a member of the Transparency Initiative?” Cluverius asks. “That’s kind of a big question.”

Cassino says that a “seal of approval” from AAPOR membership shows that a poll is high-quality, above-board, and trustworthy. Cassino sees it as his job to be as transparent as possible.

A HarrisX spokesperson wrote that the poll meets the-released standards of the National Council on Public Polls, an association of public opinion organizations founded in 1969 by Gallup. “In terms of transparency the most important thing is disclosure of the exact questions,” the HarrisX spokesperson wrote. “We do that.”

Harvard CAPS-Harris publishes its findings in three ways. Key results are presented in a slideshow presentation, with general proportions, margin of error, and headlines based on survey questions. The “topline” contains the questions asked, sample size, and the proportion of respondents, while “cross tabs” states the number of respondents for each question in various demographic categories, both weighted and unweighted.

“You can get access to all the data, so that makes it more transparent than most polls,” Ansolabehere says.

Indeed, Harvard CAPS-Harris is more transparent than many corporate polls, which may keep their survey methodology and results private. The poll’s disclosure of toplines and cross-tabulations also allows independent, external parties to understand and interpret the data beyond the findings highlighted in press releases.

But beyond the raw numbers, the poll does not publicly disclose a number of methodological decisions. They also do not publicly disclose how they use weights to account for combining the data from various panels or how they validate that responses are genuine (as opposed to hastily completed by a person or robot). Harvard CAPS-Harris publishes both weighted and unweighted numbers, but it does not define the weights outright or explain what population-based measures they are referencing.

Nesho did not provide weighting calculations after multiple requests from The Crimson.

Life and Legacy

he CGIS buildings are nothing if not well-lit, and on the third floor of CGIS South in mid-October, sunlight streams in through the panel windows. Some 500 government concentrators will pass through this building this year. A substantial portion of those will encounter the Harvard CAPS-Harris poll.

Penn sits inside a sparsely decorated office. Six or seven books rest on a bookshelf; the desk bears no sign of regular use. The office does not have Penn’s name on the door. He commutes from Miami most Thursdays for lecture, but uses it when on campus. His suitcase and backpack rest at his feet.

Leaning forward, Penn shares his long-term hope for the Harvard CAPS-Harris Poll: to endow it to the University. “I just want it to continue,” he says. “I do it with Harvard because that’s where I learned how to poll, and that’s where I’m trying to give back by doing the class.”

Chisholm, the FAS spokesperson, declined to comment on Penn’s idea. “Harvard’s Gift Policy Guide outlines the established process for considering, accepting, and administering philanthropic gifts,” Chisholm wrote in a statement. “We do not comment on specific proposals or discussions that may be under consideration.”

The Harvard CAPS-Harris collaboration has enabled Harvard students to work hands-on with the poll. For students, the class is an accessible way to learn about public opinion from a practical standpoint. Former course affiliates such as Jason A. Sclar, a teaching fellow for Gov 1360 in 2015 and 2016, have leveraged the partnership professionally. Sclar now works as a senior research executive at HarrisX, focusing primarily on the organization’s corporate work.

“One winter I had decided that I didn’t want to stay in academia,” Sclar recalls, “and not 48 hours later, I heard from Mark, and an offer to work with HarrisX came through.” He has worked for Penn’s HarrisX for seven years.

Nonetheless, some critics argue that Harvard could do better. Krosnick, as an alumnus, certainly feels that way. “I want Harvard to thrive. I want to say Harvard is doing great things. In this regard, it simply is not,” he says. “Throughout the life of the Harvard CAPS-Harris Poll, Harvard could afford to fund A-plus quality research, and it has not.”

Penn hopes an endowment will secure the Harvard CAPS-Harris Poll’s future at the University. He thinks of it as his would-be “legacy.”

Looking out the window of his unfurnished office in CGIS, he seems satisfied with what the poll has accomplished. “We are detectives. We try to go in-depth on the issues and when we pick an issue, we try to give people the tools that they would have to to understand it,” Penn says. “If you don’t like my questions, ask your own.”

—Associate Magazine Editor Wyatt J. Meyer can be reached at [email protected].

—Magazine writer Uy B. Pham can be reached at [email protected] or on Signal at ubp.88. Follow him on X @uybpham.