“The Emergence of Political Polling Through Kalye Surveys” is a study presented by the authors in a #FactsFirstPH research briefing held on March 25, 2022. The full copy of the research is reposted with permission from the authors.

First of 3 parts

Read the other parts of this series:
Part 2: ‘Kalye surveys’ and deviation from scientific standards of research
Part 3: The ethics of ‘kalye surveys’

Kalye surveys are primarily person-on-the-street interviews that basically ask respondents their presidential vote choice for the May 2022 national elections. The interviews are video-documented and uploaded, and some are even live-streamed, on YouTube. There are also some kalye survey videos that feature the tallying of gathered responses on presidential vote choice and some are commentaries or explainers about kalye surveys.

Kalye surveys surfaced as a popular YouTube material among content creators. Our Philippine Media Monitoring Laboratory team started scanning YouTube using election-related keywords in May 2021, a year prior to the 2022 elections. Kalye surveys started emerging as YouTube election-related content around September 2021, which falls within Quarter 2 of our data collection; and gained traction in terms of content uploads in October when candidates began filing their certificates of candidacy. As of January 2022, which is the end of our Quarter 3 monitoring, over 350 election-related videos have already been uploaded on YouTube.

Using a tool directly linked to the YouTube API, this article covers our review of kalye survey videos from YouTube channels that garnered the highest viewership metric as of the period of our data extraction:

• Quarter 1, which covers May to July 2021; data extracted in August 2021
• Quarter 2, which covers August to October 2021; data extracted in November 2021
• Quarter 3, which covers November 2021 to January 2022; data extracted in February 2022

Through quarterly intensity sampling, we identified the kalye survey content of the five YouTube channels with the most-watched videos when election-related kalye surveys began surfacing as YouTube content in Quarter 2 up to the end of Quarter 3, for a total of 10 channels across these two quarters.

While our review automatically included these 10 channels’ kalye survey content that made it to the top 10 quarterly viewership metric, we expanded our review of videos that these channels posted beyond the videos that made it to the top-viewed metric. We used maximum variation sampling in selecting other videos from the corpus of kalye survey videos that these channels uploaded to enable us to review variations in kalye survey content from these channels. A total of 43 kalye survey videos were reviewed.

‘Gold standard’ of survey practice

Survey is one of the most popular social science research methods for making sense of people’s circumstances, knowledge, beliefs, attitudes, habits, and behavior. While the conduct of surveys may appear as an intuitive and straightforward process of asking questions and acquiring answers, a scientific and ethically-grounded approach to its design, administration, analysis, and dissemination is key for surveys to validly and reliably contribute to sense-making. 

Mainstream political polls, such as those conducted by Pulse Asia and Social Weather Stations (SWS), survey registered voters’ current voting intention, sentiments, and preferences. When political polls adhere to the gold standard of practice, they can provide the best possible and meaningful estimate of the current electoral climate.

Essentially, the gold standard in surveys adheres to the principles of scientific and ethical practice across the entire phase of research – from conceptualization and research design; survey administration; data collection, processing, and storage; data analysis, interpretation, write-up; and up to all forms of research dissemination. As such, results from political polls that adhered to the gold standard of practice can be utilized for further analysis to forecast trends and to calibrate (or recalibrate) political campaigns.

When a survey fails to follow the gold standard of practice, survey results could lead to misinformation. Misinformation is misleading information, and decisions derived from misinformation could have negative consequences. Given that political poll results have been used in strategic campaigning, as reference for voting decisions, and as basis for comparing official election results, political polling groups must ensure that their design, administration, analysis, and dissemination strictly follow scientific and ethical standards in order to validly and reliably contribute to sense-making.

Established public opinion research groups are typically members of public opinion research organizations and social science research associations that prescribe a code of professional and ethical practice and that exact accountability among their members. The rise of ethics review boards in academic institutions is likewise key in upholding ethics and accountability in academe-based and academic-led research. Thus, for public opinion research organizations and academic institutions, mechanisms and processes are in place to safeguard and enforce the gold standard of scientific and ethical practice.

‘Kalye surveys’ and mainstream political polls

Kalye surveys are generally packaged as an “alternative” to the mainstream political polls conducted by established public opinion research organizations and academic institutions. Unlike mainstream political polling wherein the process of data collection is neither video documented nor publicly disseminated, kalye surveys reflect a constructivist approach in chronicling the survey enumerator’s administration of the survey – from providing an opening spiel, approaching prospective respondents, asking the main question and, at times, even follow-up questions, interacting with respondents through arbitrary reactions and side comments, recording of vote choice, tallying of responses, up to delivery of wrap-up statements. Unlike the survey enumerators of mainstream political polling groups who work behind the scenes, YouTube kalye survey content creators also typically perform the role of an on-camera survey enumerator, result aggregator, and all-around content host.

When a survey fails to follow the gold standard of practice, survey results could lead to misinformation.

Kalye surveys attempt to claim authenticity as individuals were filmed while responding to the basic presidential vote choice question, a contrast to the public anonymity maintained for participants of mainstream public opinion surveys. Kalye surveys aim to instantly inform viewers about the respondent’s presidential vote choice as the preference is articulated, unlike in public opinion surveys which reveal results only after full data collection and in aggregated form.

Kalye surveys likewise endeavor to be entertaining to entice viewers to watch the video, as made evident by the framing of the video title as well as the affable and conversational conduct of the survey, which is again a contrast to the rather formal and sedate manner of disseminating technical results characteristic of public opinion studies. There are also kalye surveys that do not disguise their hyperpartisanship, which is different from how mainstream political groups would endeavor to maintain objectivity across the entire phase of the research.

However, even if kalye surveys are packaged differently from mainstream political polling, kalye surveys should still be expected to follow the same gold standard of scientific and ethical practice, especially if the intended outputs of the survey is to serve the purpose of public information. In the second and third parts of this series, we will be presenting instances where kalye surveys tend to deviate from the gold standard of practice. We will also be offering basic prescriptions that address these deviations in order to design, administer, analyze, and disseminate scientifically and ethically-grounded surveys. – Rappler.com

Ma. Rosel S. San Pascual is an associate professor at the Department of Communication Research of the University of the Philippines Diliman College of Mass Communication and a member of the Philippine Media Monitoring Laboratory. She primarily teaches quantitative research methods, measurement and sampling, and quantitative data analysis in both undergraduate and graduate levels. She also actively engages in academic research on communication and new media, online discourse, media effects, mass media, ICT and development, political communication, and transnational migration and family communication.

Anjenica U. Durana is an MA in Communication student at the Department of Communication Research of the University of the Philippines Diliman College of Mass Communication. She previously received her Bachelor of Science, Major in Psychology degree from Ateneo de Manila University, and has been a licensed psychometrician since 2017.

Myrnelle A. Cinco is currently aiding the Department of Communication Research of the University of the Philippines Diliman College of Mass Communication in a number of research projects. She is also a research assistant for an international nonprofit organization working on human rights. She has a background in data analytics, data science, and research. She has a bachelor’s degree in communication research from the University of the Philippines Diliman.