Transcript Getting to grips with Altmetrics as a Journal Editor

Is there anybody out there?
Searching the social space for signs of
intelligence
Mike Taylor
Research Specialist
http://orcid.org/0000-0002-8534-5985
[email protected]
For much of the the last century, our only measurement of
the impact of scholarly research has been through the
counting and analysis of citation: one authoring researcher
acknowledging the contribution of another authoring
researcher.
Significant, certainly, given these caveats, but in a wider
social context citation analysis begins to look like an edge
case. Can the measurement of sharing on social networks
provide a wider view of how research is consumed in
society, or is it all chatter and noise – and how do we detect
the conversations of true significance?
The world according to citation
• “I am writing an article and wish to cite
another article”
The world according to alternative
metrics
• “I’m writing an article and might cite this”
• “You should read this article if you’re
interested in #thistopic”
• “My PI wrote this paper”
• “My daughter wrote this and I’m so proud”
• “This article has a titivating title. Anyway, it
made me laugh”
• “These scientists are going to cure cancer”
The phenomenally rich world of
alternative metrics
• Social activity indicators: Twitter, Facebook,
Delicious, Pintrest, Google+
• Scholarly activity indicators: Mendeley,
Citeulike, Zotero
• Scholarly articles: blogs, reviews
• Mass media: news papers, TV
• Re-usage indicators: data, code, graphics
An example from 2013
• Huge potential for social impact
• Press campaign: front page story
on much of the UK press
• Great publisher support from
Nature
• 1000s of tweets
• But what’s missing?
The phenomenally poor world of
alternative metrics
Current alternative metrics don’t count or model:
• Poorly referenced mass media
• Stories about stories
• The flow of the story
• Social media about stories, replies, re-tweets
• Influence on professional bodies
• Representation to Government, Government
policy
Not only are alternative metrics bigger
than citations, they’re also different
•
•
•
•
•
•
Public vs private
Anonymous vs attributable
Persistent vs fleeting
Positive vs negative (counts and sentiment)
Real time vs slower
But article driven, formal links
The different characteristics of
alternative metrics
• Citation: one class of activity, with many subclasses
• Alternative metrics: several types of activity,
with many classes and countless sub-classes
(all vying with each other)
The power of intelligent conversation
• Elevator pitch > monograph
• A word in the ear of a President
versus
• Engaging with millions
• Patient-power
• Lobbying interests
The chatter of (how shall we say
this?) less than intelligent
conversation
•
•
•
•
Not all communication is equal
Not all communication is between equals
Noise is not meritocratic
But is Twitter just meaningless noise?
The myth of social networks
• Often assumed to be trivial, with a focus on
titillating articles
• An analysis of 13.5k papers revealed striking
differences:
• Top 0.5% of social activity – strong emphasis
on policy, funding, areas where science and
government overlap (stem cells, CERN, etc)
• Top 0.5% of scholarly activity – primary
research
The academic networks are building
•
•
•
•
•
•
Orcid / ODIN / THOR
Data DOIs
RDA data citation
Data metrics
Usage APIs / data
Open data, open articles
Mapping academic influence is
becoming easier
• Heading towards a paradigm shift in mapping
academic influence
• Academics probably won’t create negative
links
• This is a matter-of-fact network, flat, a
statement of “what is”
• Insufficient to understand social impact
Science in society
• Open science, citizen science, open access,
open data, cloud infrastructure, open source
code, virtualization
• Social networks, easy access to scholars
• Too hard => too easy?
• Explosion in communication and access
A partial view
• Moving towards a more complete scholarly
network
• Data exists to get an idea of how research is
being consumed in society
• Too much missing to extrapolate
• Almost entirely devoid of political context
Correlations
• Not an even picture, there are threads of
correlations – blogs – tweets – mass media
• We can’t make simple conclusions
• We don’t have enough data to make complex
conclusions
Bigger data
• Deeper: Talking about people, departments,
companies, movements
• More sensitive: Going from “being spoken
about” to “what is being said”
• Wider: “who is speaking”, “to whom are they
speaking”
• Further: “他們在中國說了什麼？”
The role of sociologists and
economists
• Social potential, professional and academic
perspectives
• Is $$$ a good reflection of impact?
• People like to think of the “return on
investment” model, but it’s not that easy, and
the conclusions may be uncomfortable seen in
isolation
Is a social impact index computable?
• Social impact index =
f(social capacity), f(social accessibility), f(social
reach)
• If has no capacity for effecting social change, if
is incomprehensible, if no-one is aware of it…
• We need the data and the maths to identify
the intelligent versus the influential