Transcript Content Analysis - University of Wisconsin

Journalism 614:
Content Analysis
Content Analysis
 Study of a recorded human communication
– The coding of communication for the presence
of certain traits, categories, or meanings
– Analysis can relate the occurrence of coded
content with other factors, such as features of
the producer, effects on the receiver, etc.
– Applied to the study of books, magazines,
papers, transcripts, web pages, songs,
speeches, postings,, statements, utterances, etc.
Topics of Content Analysis
 Well suited for communication research
– Critical for answering the classic question…
Who says what, to whom, and with what effect?
– Yet content differences do not equal effects
• Must move beyond simple content studies to relate
coded features to antecedents or consequences
• Relating content features to one another is also
meaningful - i.e., what political issues are featured
in coverage of each presidential candidate?
Sampling in Content Analysis
 Since you can rarely observe all content, must
sample from available content for coding pool
– Units of analysis may differ from units of observation
• Observe story content to analyze newspaper differences
– Sample selection depends largely on unit of analysis
• Example, if studying differences between authors, the unit of
observation may be books, pages, paragraphs, or sentences
• Need to be clear about unit of analysis before planning
sampling strategy to avoid problems later
Questions in Sample Generation
 Must establish the universe to be sampled from
– Ex - Content analysis of television violence by network
• Which TV stations should you observe?
• How many days will you observe them?
• During which hours will you observe television?
– You always make assumptions; be upfront about them
• We will code the six major networks — ABC, CBS, NBC,
FOX, UPN, WB — for a “random week” during the month of
December from 7 PM to 10 PM, each day selected.
– May be Random, Systematic, Stratified, etc.
Example for your exercise/project
 First: Search for news stories containing relevant keywords
during your specified time period from specified sources
– Ex. You find 1816 stories mentioning “global warming” or
“climate change” in the New York Times from 1/1/96 - 12/31/15
 Second: Randomly or Systematically sample a sufficient
number of stories from the full pool of stories
– Ex. Say you want 300 stories, you need approximately every 6th
story (i.e., 1816/300 ≈ 6). So you pick a random number start
number between 1 and 6 and then choose every 6th story.
– Or you can use an online random number generator to pick 300
random numbers for you (www.random.org).
 Third: Content code the 300 stories for relevant features.
Coding in Content Analysis
 Coding is the heart of content analysis
– Process of converting raw data into a standardized form
– Classify content in relation to a conceptual framework
• Ex. Emotionality, Partisan Bias, Source Attribution, etc.
– Must carefully conceptualize coding categories
• Relevant concepts and relevant categories within concepts
– Manifest (visible) / Latent (underlying meaning)
• How big a leap between observation and inference
– The more manifest, the more reliable - ex. word count, date
– The more latent, the more interesting - ex. meaning, frames
Data Management
 End product of coding in usually numerical
 Distinguish units of analysis and observation
 Establish the base for coding (i.e., proportion)
 Understand the limits of your coding system
Additional Considerations
 Problems with coding long periods (100 years)
– Imposing modern standards on the past
 Coding TV content is technically difficult
– Vanderbilt Univ. archive of network news coverage
– Transcripts from Lexis/Nexis and “Closed Captions”
 Emergence of computer-aided techniques
– VBPro, Diction, LIWC, Wordstat, and others
– Dictionary-based, customizable, syntactical
– Latest is machine learning, unsupervised (topic
modeling) or supervised (classify using training set)
Strengths and Weaknesses
 Easy to undertake - no staff, no special equipment
 Easy to correct errors - go back and recode
 Allows for the study of dynamic processes - time
 Unobtrusive - no effects on subject of study
Yet….
 Limited to recorded communication - much is lost
 Limited in terms of claims you can make
Content Coding Exercise
 Must code text with a high degree of agreement
– Reliability between coders is a key criteria
– Can we agree on coding climate change stories?
 Three content features: Frame, Certainty, IPCC
– Frame - Science (emphasis on scientific findings) or
Politics (emphasis on policy debate) - forced choice
– Certainty - How certain is the threat of human-made
global warming? (low, medium, high)
– Does it mention the IPCC - Intergovernmental Panel on
Climate Change? (yes or no)
Content Coding Example #1
“Some of the world's most distinctive and biologically
diverse climate regions – from South America's Andes
Mountains to southern and eastern Africa and the US
Southwest – may be drastically altered by century's end,
endangering plant and animal life there, if there are no
curbs on greenhouse gas emissions. This according to a
new climate-modeling report issued March 26. The
researchers built their forecast on data contained in a
massive study being published in installments this year
by the Intergovernmental Panel on Climate Change.”
Content Coding Example #2
“Most climatologists agree that the earth's temperature
has increased about a degree over the last century. The
debate is how much of it is due to mankind's activity.
Britain's Channel 4 television has just produced "The
Great Global Warming Swindle," a documentary that
devastates most of the claims made by the
environmentalist movement. The scientists interviewed
include top climatologists from MIT and other
prestigious universities around the world. The
documentary hasn't aired in the U.S., but it's available
on the YouTube.”
Content Coding Example #3
“U.S. researchers report that many of the world's current
climates may disappear if current global warming trends
continue, while climates unlike any seen today would be
created, increasing the risk of extinctions and other
ecological events. Professor John Williams, University of
Wisconsin, Madison, and colleagues forecast the risk of
novel or disappearing climates by the year 2100, using
global climate models and greenhouse gas emission
scenarios from the recent assessment by the
Intergovernmental Panel on Climate Change (IPCC).”
Content Coding Example #4
“To opponents of global warming, it must have seemed
like a perfect storm: Al Gore pocketed an Oscar for his
doomsday climate documentary, An Inconvenient Truth.
The International Panel on Climate Change (IPCC) issued
a report with dire warnings about CO2 emissions. Even
George Bush offered conciliatory talk about reducing
carbon pollutants. But then an unwelcome squall appeared
on the horizon: The documentary The Great Global
Warming Swindle. Director Martin Durkin's film
combines interviews with distinguished scientists, a sober
narrative, and damning graphs and statistics to challenge
the advocacy politics of global warming.”
Content Coding Example #5
“Australia will spend $200 million ($161 million U.S) to
reduce the clearing of forests in Asia, Prime Minister John
Howard said. The funds to support sustainable logging
would help curb greenhouse gas emissions, Howard told
reporters in Canberra. His Liberal-National government,
trailing the main opposition Labor Party in opinion polls,
is tackling climate change as it heads to an election this
year.The government has refused to ratify the Kyoto
Protocol that sets a timetable for cutting greenhouse gas
emissions, saying it would hurt economic growth.”
Content Coding Example #6
“While it is incontestable that the atmosphere is warming
with possible dire consequences, it is not agreed on by all
climatologists and scientists that man is the sole cause of
this phenomenon. Many alternatives have been advanced to
the theory that burning fossil fuels is the culprit. These
claims include solar radiation, changing ocean currents, the
effect of clouds, and cosmic rays from outer space. Among
scientists who remain skeptical of the popular CO2 theory
is Professor David Bromich of Ohio University. Bromich
has stated, “The climate models I have seen are
inconsistent with the evidence that we have for the past 50
years from Antarctica.”