Transcript Slide 1

Assessing Oral Proficiency and
Grammatical Errors:
Secondary Analysis of DA Interview Data
Sun-Kwang Bae, Curriculum Development Division, DLIFLC
David Moon, Technology Integration Division, DLIFLC
This speech/presentation is authorized by the Defense Language
Institute Foreign Language Center and the Department of Defense.
Contents of this presentation are not necessarily the official views of, or
endorsed by, the U.S. Government, DoD, Department of the Army, or
DLIFLC.
Presentation Outline
•
•
•
•
•
•
The Study Question
Background
The Study
Results
Discussions
Q&A
2
The Study Question
What are the grammar structures/features that
learners of Korean typically control at a given
level of oral proficiency?
[From the perspective of teaching & learning 김지민 (2009); Corder (1981)]
3
Tasks that learners can handle at
each level – ILR L1
Level 1
• Able to ask and answer simple questions related
to everyday survival needs
• Able to satisfy minimum courtesy requirements
• Able to deal with routine everyday situations
• Able to talk simply about self
4
Tasks that learners can handle at
each level – ILR L2
Level 2
• Able to satisfy routine social demands and
limited work requirements
• Able to narrate and describe in all three time
frames
• Able to give detailed instructions/directions
• Able to participate in conversations on current
events as well as give detailed information
about work, family and self
5
Tasks that learners can handle at
each level – ILR L3
Level 3
• Able to participate in most formal and informal
conversations on practical, social and professional
topics
• Able to support opinion in the FLO content areas
• Able to hypothesize
• Able to discuss concrete and abstract topics
• Able to discuss topics of particular interest
• Able to deal with unfamiliar situations
6
Structural control that learners are to
have at each level – ILR L1
Level 1
• Structural accuracy is random or severely
limited
• Almost every utterance has errors in basic
structures
• Time concepts are vague
• Can formulate some questions
7
Structural control that learners are to
have at each level – ILR L2
Level 2
• Discourse is minimally cohesive
• Grammatical structures are usually not very
elaborate and not thoroughly controlled
• Errors are frequent
• Simple structure and basic grammatical
relations are typically controlled
8
Structural control that learners are to
have at each level – ILR L3
Level 3
• Effectively combines structure and vocabulary
to convey meaning
• Discourse is cohesive
• Use of structural devices is flexible and
elaborate
• Errors occur in low frequency and highly
complex structures, but structural inaccuracy
rarely causes misunderstanding
9
The study: data collection
Data Collection
• Face-to-face diagnostic oral assessment
interviews conducted by a pair of specialists
(David Moon, EM, AS, JY, YR, NT)
• During June 1999 – September 2000
• 55 students: L0+ (1), L1 (15), L1+ (18), L2 (14),
L2+ (6), L3 (1)
• The tool: Sample
10
The study: structural features examined
Structural Features
•
•
•
•
•
Sentence patterns: Plain, Honorific, Quotation
Verb conjugations: Irregular, Special
Complex constructions: Passive, Attributive, Negation
Particles: Case, Locative, Instrumental, Auxiliary
Cardinal/Ordinal Number: Native, Sino-Korean
agreement
• Pronouns: Personal, Things, Places
• Conjunctions: Basic, Advanced, Conjunctive Fillers
11
The result: sentence patterns
Note: The figures are percentages “Satisfactory/Need Work/Not Applicable.”
L1
(N=15)
L1+
(N=18)
L2
(N=14)
L2+
(N=6)
Plain: Present
73/26/0
78/22/0
100/0/0
100/0/0
Plain: Past
47/53/0
61/39/0
100/0/0
100/0/0
Honorific: Present
13/83/0
17/83/0
29/71/0
100/0/0
Honorific: Past
0/100/0
6/86/6
21/ 72/7
100/0/0
Negation:
Statement & Question
40/53/7
56/44/0
93/7/0
100/0/0
Negation:
Request & Suggestion
13/20/67
28/43/39
50/0/50
100/0/0
12
The result: verb conjugations &
complex structures
Note: The figures are percentages “Satisfactory/Need Work/Not Applicable.”
L1
(N=15)
L1+
(N=18)
L2
(N=14)
L2+
(N=6)
Irregular verbs
27/66/7
33/67/0
57/43/0
100/0/0
Special verbs
31/69/0
(N=13)
38/62/0
(N=16)
83/17/0
(N=12)
100/0/0
(N=5)
Passive and causative
0/27/73
0/56/44
21/29/50
67/0/33
Attributive
7/93/0
33/67/0
79/21/0
83/17/0
13
The result: quotation
Note: The figures are percentages “Satisfactory/Need Work/Not Applicable.”
L1
(N=15)
L1+
(N=18)
L2
(N=14)
L2+
(N=6)
Quotation: statement and
question
6/38/56
22/67/11
42/44/14
100/0/0
Quotation: request and
suggestion
0/33/67
6/38/56
7/21/72
83/0/17
14
The result: markers/particles
Note: The figures are percentages “Satisfactory/Need Work/Not Applicable.”
L1
(N=15)
L1+
(N=18)
L2
(N=14)
L2+
(N=6)
Subject/Object Case
33/67/0
56/44/0
71/29/0
100/0/0
Dative/Locative
20/80/0
50/50/0
71/29/0
100/0/0
Instrumental
18/73/9
(N=11)
30/60/10
(N=10)
67/33/0
(N=12)
100/0/0
(N=5)
Auxiliary
0/100/0
28/72/0
71/29/0
100/0/0
15
The result: numbers + counters
Note: The figures are percentages “Satisfactory/Need Work/Not Applicable.”
L1
(N=15)
L1+
(N=18)
L2
(N=14)
L2+
(N=6)
Cardinal: Native Korean
33/67/0
44/56/0
71/29/0
100/0/0
Cardinal: Sino-Korean
40/60/0
33/56/11
71/29/0
100/0/0
Ordinal: Native Korean
7/47/46
22/56/22
36/21/43
83/0/17
Ordinal: Sino-Korean
7/40/53
16/34/50
0/14/86
83/0/17
16
The result: pronouns
Note: The figures are percentages “Satisfactory/Need Work/Not Applicable.”
L1
(N=15)
L1+
(N=18)
L2
(N=14)
L2+
(N=6)
Personal
53/34/13
89/11/0
93/7/0
100/0/0
Demonstrative: Things
13/54/33
72/17/11
93/7/0
100/0/0
Demonstrative: Places
26/61/13
67/22/11
100/0/0
100/0/0
17
The result: conjunctions
Note: The figures are percentages “Satisfactory/Need Work/Not Applicable.”
L1
(N=15)
L1+
(N=18)
L2
(N=14)
L2+
(N=6)
Basic conjunctive endings
40/60/0
56/44/0
86/14/0
100/0/0
Advanced conjunctive
endings
0/80/20
6/83/11
0/86/14
17/83/0
Conjunctives/
Conversational Fillers
0/100/0
41/59/0
(N=17)
64/36/0
100/0/0
18
The result: discourse skills
Note: The figures are percentages “Satisfactory/Need Work/Not Applicable.”
L1
(N=15)
L1+
(N=17)
L2
(N=14)
L2+
(N=6)
Repair
7/93/0
35/65/0
79/21/0
100/0/0
Vocabulary choice with
precision
0/100/0
0/100/0
21/79/0
50/50/0
Negotiate meaning
27/63/0
29/71/0
85/15/0
100/0/0
Paraphrase and restate
7/93/0
29/71/0
21/79/0
100/0/0
19
Discussions: grammar features that
learners begin to control at L1 & L1+
Level 1
• Personal pronouns
• Present tense
Level 1+
• Past tense
• Negation (in statements & questions)
• Case markers (subject, object, dative, locative)
• Demonstrative pronouns (things & places)
• Basic conjunctive endings
20
Discussions: grammar features that
learners begin to control at L2
Level 2
• Negation in requests and suggestions
• Irregular verbs and special verbs
• Attributive forms
• Instrumental and auxiliary particles
• Numbers and their agreement with counters
• Conjunctives and conversational fillers
• Repair strategy in discourse
• Negotiate meaning in discourse
21
Discussions: grammar features that
learners begin to control at L2+
Level 2+
• Honorifics
• Passive and causative
• Quotations
• Precise lexical selection
• Paraphrase and restate
The features that learners can’t control by L2+:
• Advanced conjunctive endings
22
Implications & Applications
Q&A
23
Contact Information
Online Diagnostic Assessment
for reading and listening skills:
http://oda.dliflc.edu
Contact: [email protected]
24