Transcript here. - ReStore

Expanding your Expertise
as a Social Science
Researcher
Workshop Aims
1. Explore social science research
‘expertise’, including expert thinking,
drawing on the literature
2. Examine how research expertise is
learned, accelerated, and possibilities for
your further development
3. Engage in a collaborative research design
task
4. Plan a strategy for accelerating the further
development of your research expertise
Programme
Timing
10.00 – 11.30
Session Topic
& aim
1
The problematic nature of research
expertise, including expert thinking
break
11.45 – 13.00
2
Learning research expertise and
scope for accelerating development
lunch
13.45 – 15.30
3
Collaborative expert thinking in
practice
break
15.45 – 16.30
4
Implications for personal strategies
How you might benefit from
today’s workshop
• More informed about the problematic nature
of research expertise and how far its
development can be accelerated
• Reviewed and started planning how to
sustain or reinvigorate your research activity
• Extended your network of potential
collaborators and ‘critical friends’
• Informed your thinking about how you can
support others in developing their research
4
Ethical ground-rules
• Sharing work experiences, aspirations and
feelings entails sensitive information
• But candour within a respectful, supportive,
confidential context facilitates learning
So in this workshop, please commit yourself to
following these ‘Chatham House’ rules:
- speak openly
- listen well
- take away ideas…
- but respect others’ intellectual property
- and respect confidentiality
Session 1 (Aim 1)
The problematic nature of
research expertise,
including expert thinking
Starting assumptions
• Today’s academics are affected by
significant work intensification
• It is only worth dwelling on what individuals
have some chance of changing
• Choices carry prices: where do you want to
locate yourself on the expertise continuum?
7
‘Expertise is not an endpoint, it is a continuum…
studies will allow us to frame…meaningful
opportunities for advancing the development of
talent…for ever-expanding numbers of individuals’
(Sosniak 2006:300)
Research & trialling that
informed the workshop
• Draws on research training project (2010-14) within
ESRC Researcher Development Initiative
• Profs Alison Wray, Mike Wallace, Cardiff University
• Reviewed generic expertise literature
• Interviewed 31 very experienced researchers from
different social science disciplines and countries
• Trialled single and multi-disciplinary workshop
designs, content, tasks
• Outcome – resource for supporting development of
mid-career researchers, based on project findings
8
Research interviews
• Successful, very experienced researchers* from Business &
Management, Applied Linguistics, other social sciences
* Editors of top journals, past presidents/chairs of national associations
(e.g. US Academy of Management), plenary speakers at association
conferences, recommended by others, international reputation
• Interviews of up to 90 minutes, or equivalent 11-question
electronic schedule
15
UK
20
Bus &
Man
App Ling
15
10
5
Other
0
Discipline
9
10
US/Canada
5
Other
0
Location
Other:
Mainland Europe, China,
Australia, New Zealand
Interview question areas
• What is social science research expertise,
including expert thinking?
– For you, in your discipline, what constitutes research
expertise and how do you know an expert? What makes
you one?
– How do you approach your research and how has that
changed during your career?
• How did you learn the skills of expert thinking?
– Training? Mentors? Environment? Strategy?
• How do you pass these skills on to others?
– Trainer? Mentor? Advice?
10
45-E
Taking stock: individual review
• On your own, please complete Exercise I
in your handout (pages 1-3)
• Afterwards, reflect on the implications of
your review for your aspirations as a social
science researcher
45-S
11
00-E
Informing your thinking through
evidence from other contexts
• The expertise literature has many limitations, there’s
little research on social science researcher
development, but some ideas might be useful
• You can judge how relevant evidence from expertise
literature and research interviews is to your situation,
and so whether it has applicability
• Allow for the possibility that even seemingly irrelevant
observations could turn out to be informative
• We will be discussing which (if any) of the features
mentioned is unexpected, so note down your
thoughts as we proceed
00-S
12
Some limitations of expertise literature
Expertise is a
social construct…
Most research on expertise
is psychological: controlled
situations and externally
validated measures (e.g.
wins, scores)…
It’s not clear how much
we can generalize
across domains
13
…so who decides who
has expertise?
..but we can’t directly
measure social science
research expertise – it’s
not like winning chess
championships
The research supposes
BOTH generic features of
expertise, AND that expertise
is not transferable across
domains. Being a good
athlete doesn’t mean you’re a
good researcher
What is expertise?
1. A general definition of expertise (Schraw
2006:259), includes expert thinking
• Large, integrated knowledge base
• Sophisticated mental models that guide problem
solving and critical thinking
• Highly automated procedural and monitoring skills
[Expert researchers]
come up with the most
amazing clever ideas.
They see something
out there which once
they see it you think,
‘My gosh, why didn’t I
think of that before’?
14
An expert is ‘up’ on the topic at
hand, familiar with the range and
breadth of the field, knowledgeable
about ancillary fields...
Somebody who can write about [their
research] really well and somebody
who can stand up and talk about this
in a very persuasive and clear way
What is expertise?
2. Identifying expertise (Ericsson 2006)
• Defined in terms of individual capabilities acquired
through experience: assumes no-one born an expert
• Characteristics distinguishing expertise
– Social (reputation, as attributed by others)
– Length of experience in the domain
– Consistent high-level performance of tasks within
domain
Not only has a knowledge of
– this domain
alternatives but can go into
not an
objective as
quality.
– It’s
Social
(reputation,
attributed by
others)
depths
about each
It requires a certain amount
alternative, can articulate the
of recognition from the
theory…of those alternatives.
community for you to be
Has an ability to argue each
defined as an expert
perspective
15
What is expertise?
3. No objective standards for many important
domains of expertise (Shanteau 1988:205-6):
‘…external standards frequently do not exist for many
real-world problems…Indeed, experts are essential in
precisely those domains where there are no right
answers.’
• Social and task-performance criteria are continually
debated in the social sciences, since knowledge of
the social world, its measurement, and authority to
determine criteria are all contested
• So no cross-social science consensus on the nature
of research expertise, or criteria for measuring it
16
Some experts may be stronger in theory and others in methodology,
an ideal expert should blend the knowledge and understanding of both
What is expertise?
4. Expertise is both individual and social
(Hoffman 1996:94):
‘Knowledge—in a sense—must exist inside heads. Where
else could it reside?…when the expert carpenter leaves
the workshop, something does leave with him. Could you
or I use the tools to build, say, a china cabinet?
However, knowledge—in a sense—is an attribution that
resides in social groups. How else could it be developed,
taught, or standardized? How could someone be
regarded as an expert if her judgments are not followed in
the decisions made by other people?’
• Social science researchers come to know a lot and
do creative things with it, through long experience
17
Expert is really not what is in one’s head,
it’s how others’ heads judge this person
What is social science research expertise?
5. Aspects of expertise may differ between
disciplines, fields or specialisms, for example:
Aspects of research
expertise
Polarization of
approaches
Business and
management
Polarization between North
American positivist,
European interpretive &
critical approaches
Journal dominance
Dominance of North
American journals
Intellectual orientation Social sciences and
economics
Goals pursued
Career goals, addressing
knowledge gaps
Communication
Variable importance
capability
Acknowledgement by Made impression on
research community
18others
Applied linguistics
Little geographical
polarization, wide range of
approaches
Little geographical journal
dominance
Social sciences and
humanities
Addressing issues that
could change practices
High importance
Accepted and respected
by research community
How do experts think?
• A style of skilful conceptualizing and arguing
that is appropriate for a knowledge domain.
Ochse studied Nobel Prize winners (1990:259):
‘We must not forget that these subjects [interviewees]
referred specifically to domain relevant thinking styles
taught by masters in that field.’
• In the social sciences expert thinking styles reflect
contestation about social knowledge
• So expert thinking styles are NOT directly
transferable to all knowledge domains
A good question here means one that others in the field would…
immediately recognize as important to be answered…Good means it’s
answerable in principle or even in practice with what we know today
19
How do experts think?
• Pursuing different intellectual purposes is
appropriate in the social sciences
• Expert researchers may pursue depth or breadth
of understanding
• ,
The ‘splitters’ are those who are trying to set up their work as
unique, as different, as separate from all of these other approaches
that are wrong. The ‘lumpers’, on the other hand, are trying to
integrate, they compare and contrast and see how their approach
compares with others and then make…comparative assessments
between these alternatives…
‘Splitters’…are certainly experts. But they tend to be more narrow,
they tend to be deeper and narrower, whereas the ‘lumpers’ are
perhaps shallower and broader. And I think that is a trade-off
20
Who has the authority to label
someone an ‘expert’ researcher?
I know I’m operating most of
the time very much at the
edge of my capabilities
Any decent project that I’ve
done has involved me totally
rethinking where I’m going at
least ten times
I think there is a danger in regarding people as experts. Anyone
who eventually gets that title usually realizes that they have
enormous gaps in their knowledge. One of the problems is that
that unofficial title blinds people to the weaknesses in the
research of ‘experts’. Just because an expert says or does it,
does not mean that it is correct.
Once experts start believing in their own expertness, we are in
deep trouble
21
35-E
Discussion
1. Do any of these features surprise you – if
so, why?
2. Which features give you most pause for
thought about the trajectory towards
greater research expertise that you might
experience or expect?
22
35-S
45-E
Session 2 (Aim 2):
Learning research expertise,
and scope for accelerating
development
How is expertise learned?
1. Self monitoring, building-up metacognition
(awareness of one’s cognitive processes) in
a domain (Feltovich et al 2006:57)
‘The development of expertise is largely a matter of
amassing considerable skills, knowledge, and
mechanisms that monitor and control cognitive processes
to perform a delimited set of tasks efficiently and
effectively. Experts restructure, reorganize, and refine
their representation of knowledge and procedures for
efficient application to their work-a-day
environments…experts certainly know more, but they
also know differently.’
24
How is expertise learned?
• Social scientists learn by monitoring their thinking,
building up a reflexive, self-questioning approach
to enquiry
• Novices may not ‘know what they don’t know’,
gradually building a large stock of knowledge and
awareness of inter-linkages
When I was a PhD student, I tried to think about questions
in…the more conventional way, which is to be more linear.
And to say, ”Well, this research…produced these findings,
and if we extrapolate that, then we should find this…” [But
now] I won’t be excited unless I can see how it fits into a
bigger picture of some sort
25
How is expertise learned?
2. Building-up ‘chunking’. Experts remember
and recall information in larger, more
complex, inter-related ‘chunks’ than novices
Chase & Simon’s chess-playing experiments:
Better players ‘encode the position into larger perceptual
chunks, each consisting of a familiar subconfiguration of
pieces. Pieces within a single chunk are bound by
relations of mutual defense, proximity, attack over small
distances, and common color and type’. (1973:80)
26
• Social scientists learn to group types of theory
(e.g. realist), method (e.g. qualitative), study (e.g.
social class), stock criticism (e.g. of opposing
theories) and responses to them…
How is expertise learned?
3. Building-up flexible restructuring. Expert
knowledge becomes organized in more
abstract ways
Chi et al (1982: XX) studied physics students:
Novices showed an ‘inability to infer further knowledge
from the literal cues in the problem statement. In
contrast, these inferences necessarily are generated in
the context of the relevant knowledge structures that
experts possess’.
• Expert knowledge reflects principles or patterns,
not just the aggregate of concrete ‘facts’ about
social phenomena, theories, methods
27
How is expertise learned?
• Expert knowledge comes to includes dynamic
(repeatedly modified) abstract mental models,
including cause-effect schemata
When I was a graduate student I imagined that all
one needs to be a researcher is...to know the
literature and have some methodological
expertise, and that’s basically it
Roughly every decade I discovered something in terms of, say,
a method[ological] approach that I thought was more
meaningful and helpful than the previous one. This at the
same time increased my uncertainty as to how I understood
what was going on
28
35-E
How is expertise learned?
4. Declarative to procedural knowledge.
Consciously learned knowledge and skills
gradually become tacit with repeated practice
Expertise model (Dreyfus & Dreyfus 2005:787):
‘With enough experience…the brain of the expert
gradually decomposes this class of situations into
subclasses, each of which requires a specific response.
This allows the immediate intuitive situational response
that is characteristic of expertise.’
29
• With increasing experience, social scientists can
draw intuitively on tacit knowledge of social
phenomena, theories, methods, analytical tactics,
research design options…
00-S
How is expertise learned?
5. Developing the ability to sustain motivation
through increasing self-knowledge
• Early setbacks are inevitable due to naivety…
• But more awareness of motivators, strengths and
weaknesses, enables focusing on rewarding work
My first paper I submitted…I can look at that paper now and think that
they were very kind in their reviews. But I was devastated. I thought my
career was over. I sent in this thing that was totally inappropriate…I just
did a small version of my dissertation, and it was totally wrong because I
wasn’t in the conversation
I've learned that I'm better at conceptual theorizing than at empirical
research. In the beginning I was doing lots of empirical work…but I wasn't
creative at it, and it appeared, at least to me, that I enjoyed and was
better at trying to speculate about the big picture
30
00-S
Accelerating expert development?
1. Avoiding the ‘automaticity’ plateau. Actively
self-monitoring habitual thinking and skilful
practice, making continual adjustments to
improve (Ericsson 2006b:691):
…individuals who eventually reach very high levels do
not simply accumulate more routine experience of
domain-related activities, but extend their active skillbuilding period for years or even decades.
• Fast learners consciously avoid continuing to act
within the ‘comfort zone’ of their existing expertise
• They continually push themselves outside their
comfort zone by seeking novel opportunities,
experiences, knowledge, ways of thinking…
Accelerating expert development?
2. Deliberate practice. Regular, short bursts of
concentrated practice with feedback –
Ericsson’s (2006b:694) musicians research:
The expert performers and their teachers identify specific
goals for improving particular aspects of performance
and design training activities that allow the performer to
gradually refine performance with feedback and
opportunities for repetition (deliberate practice). The
performers will gradually acquire mechanisms that
increase their ability to control, self-monitor, and evaluate
their performance…and thus gain independence from the
feedback of their teachers.
• Researchers seek support of expert colleagues
who keep pushing them beyond their comfort zone
Accelerating expert development?
• Most ‘deliberate practice’ in social science
research involves real (not practice) tasks
I see anonymous
peer-review as
central in the
expertise-building
enterprise
One or two colleagues were very useful
in developing my research skills largely
through discussion and through cooperative critiquing of research designs.
Their main help to me came through their
pointing out the weaknesses of my
designs and suggesting other
possibilities.
I have also learned a lot through listening
to others describe and justify their
research designs
Accelerating expert development?
3. Coaching/self-coaching. Practice tasks with
formative feedback (Ericsson 2006b:698-9):
…more-accomplished individuals in the domain, such as
professional coaches and teachers, will always play an
essential role in guiding the sequencing of practice
activities for future experts in a safe and effective manner.
Research on self-regulated learning…has documented
effective study methods that are related to superior
academic performance, especially in high schools.
• Social scientists may actively seek collaboration
with experts in their field so as to learn from them
• Expert researchers may offer practice and feedback
support to less experienced researchers
Accelerating expert development?
4. The research environment. Harnessing
opportunities, supporting other researchers
I learned a great deal from the ‘big cheeses’ from politics and policy
from the UK and other countries [who] used to come in. And
everybody would go down to coffee and tea to the common room
every day, twice a day usually, because you never knew who was
going to be there. There would always be somebody interesting
…hanging out in my office and being able to chat to people when
needed, but they chat to each other and we have meetings where we
talk about shared issues…it’s that kind of community which is
absolutely paramount
See yourself as a citizen in your context where you do your work. You
cannot do your work in isolation, so you’re dependent on the
environment that surrounds you. So the first thing is, you’re not
entitled to anything…you have to make the environment work for you
Accelerating expert development?
5. Dovetailing teaching with research. Making
opportunities to inform research via teaching
and supervision
…a lot of my learning came
through having to write about
what I was reading…
Whenever I teach a new course
my immediate goal is to write a
book that covers the ideas in
that course...I find that through
having to write I need to clarify
my ideas, and that sends me
back to critical reading and
thinking about what I have read
…teaching research
methods… just forces you
to be very clear and
present [information] to
others...You’re selecting
work to illustrate the
methods and you’re
evaluating it with your
students, so that means
you need to work towards
that deeper understanding
where you can evaluate
something
Taking stock: development
opportunities
• On your own, please complete Exercise 2 in
your handout (page 4)
• Afterwards, reflect on how you might:
a) modify your learning pattern to accelerate
the development of your research expertise
b) compensate for any past lack of
opportunities (or ones you didn’t take)
c) maximize opportunities you now have
45-S
37
00-E
Discussion
1. What range of opportunities is there for
mid-career researchers to accelerate the
development of their expertise?
2. What range of opportunities can midcareer researchers create for yourselves,
or initiate in order to get support? t
38
35-S
45-E
Session 3 (Aim 3)
Collaborative expert
thinking in practice
What is interactional expertise?
Interactional expertise means being able to contribute
to collaborative research by ‘talking the talk’ – knowing
just enough about others’ specialist expertise to work
productively with them. Collins (2007:615) claims:
There is an important kind of specialist expertise, called
‘interactional expertise’ that turns on fluency in the
language of the domain rather than hands-on experience;
it is acquired more through immersion in the discourse of
the hands-on experts than through participation in their
characteristic practices.
• Large-scale research to tackle complex social issues requires
experts with complementary specialisms to collaborate
• So research expertise for mid-career social scientists may
extend to interactional expertise needed for collaboration
40
If you just put half a dozen smart people in a room
together you don’t have to have too much of an agenda for
something good to come out of it. As long as people obey
certain social rules, such as respecting each other, being
flexible and letting ideas flow together
You have to be willing to accept ideas of other persons,
back up, and let others take the lead as well as you
You have conversations on things you couldn’t imagine
on your own...at the end of the day when you’re drinking
that bottle of wine and you’re working on the debrief of
the [jointly conducted] interviews, you wish you could
take the white tablecloth of the restaurant with you
because you’ve written on it
41
Group task (handout p5)
1. Create the initial outline for a research
project that harnesses aspects of all group
members’ expertise
2. Prepare your plenary report (<5 minutes)
• Focus, questions, framing, data collection &
analysis, potential answers, extra collaborator?
42
30-S
00-E
Session 4 (Aim 4)
Implications for personal
strategies
Tips from successful social science
researchers interviewed
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
44
Seek training if you need it
Put the hours in and do things properly
Be streetwise about your career needs
Be focused, but alert to new opportunities
Try to work in a strong research environment
Learn to manage the trauma of reviewer feedback
Do something interesting and important
Put yourself out there and disseminate your work
Plan for the long term Which resonate most
Get a good mentor
with you, and why?
Some possibilities for self-help
Working with others:
• Co-author with someone more experienced
• Develop your ‘interactional expertise’ – ability to
think, talk and contribute beyond the boundaries of
your expertise to shared research activities
• Listen to others’ points of view and create new,
jointly constructed knowledge
• Organize a research seminar
• Engage with policymakers and practitioners about
their concerns relating to your area of interest
45
Some possibilities for self-help
Working alone:
• Extend your comfort zone (knowledge, methods)
• Talk ideas through and write to help your thinking
• Target a tougher journal
• Develop a grant proposal
• Address reviewer feedback constructively
Harnessing other responsibilities:
• Use your teaching and supervision to extend your
own research-related learning
• Use your administrative roles to improve the
research environment for everyone
46
Planning your personal strategy
One way is to work incrementally:
1. Clarify your desired trajectory
2. Formulate key goals for moving along it
3. Decide what you can do immediately, over
the next few months, the coming year
4. Review your strategy both regularly and
when opportunities or hindrances arise…
• On your own, please complete Exercise 3
of your handout (page 6)
47
30-S
45-E
Sharing take-away tips
(If you are comfortable doing so)
• Please identify one idea you are thinking
of following up on that you are willing to
share
48
45-S
00-E
Evaluation
49
45-S
00-E
References
Chase, W & Simon, H (1973) Perception in chess, Cognitive Psychology 4: 55-81
Chi , M (2006) Two approaches to the study of experts’ characteristics. In Ericsson et
al, 21-30
Chi, M, Glaser, R & Rees, E (1982) Expertise in problem-solving, in Stemberg, R
(ed) Advances in the Psychology of Human Intelligence, Vol. 1, Hillsdale, NJ:
Erlbaum, 7-75
Ericsson, K A (2006a) An introduction to Cambridge handbook of expertise and
expert performance: its development, organization, and content. In Ericsson et al,
3-19
Ericsson, KA (2006b) The influence of experience and deliberate practice on the
development of superior expert performance. In Ericsson et al, 683-703
Ericsson, KA, Charness, N, Feltovich, P & Hoffman, R (eds) (2006) The Cambridge
Handbook of Expertise and Performance Cambridge: CUP
Ochse, R (1990) Before the Gates of Excellence: The Determinants of Creative
Genius Cambridge: CUP
Schraw, G (2006) Knowledge: structures and process, in Alexander, P and Winne, P
(eds) Handbook of Educational Psychology (2nd edn Mahwah, NJ: Erlbaum 245263
Sosniak, L (2006) Retrospective interviews in the study of expertise and expert
performance. In Ericsson et al, 287-301
50