Transcript WHAT to test?

LOGO
Dinh Cong Triet
TEST DESIGNING
&
TEST SPECIFICATION
BUILDING
Summer training course- August 2011
10 Questions for thought
Speaking
Reading
Skills
Listening
Knowledge
Writing
WHAT
to test?
Language
lexis
semantics
Grammar
morphology
syntax
Reading
WHAT
Reading aloud
pronunciation
to test?
Silent reading
Subskills
• Identify main facts & details,
Stresses & intonation
• Relate cause & effect
• Identify sequence of events, predicting
outcomes
• inferring meaning from contextual clues
Listening
WHAT
to test?
Subskills
• Identify main facts & details,
• Relate cause & effect
• Identify sequence of events, predicting
outcomes
• inferring meaning from contextual clues
Grammar
WHAT
to test?
Syntax (sentence structure)
Mechanics: e.g Punctuation, capital letter, comma, etc..
pronouns
Rules of grammar
nouns
tenses
verbs
prepositions
subject & verb
agreement
comparatives
adjectives
adverbs
Speaking
quantity
quality
vocabulary
Content / ideas
Sentence complexity
Language
WHAT
clarity
to test?
syntax
grammar
fluency
confidence
Pronunciation
Presentation skills
Organization
accuracy
appropriacy
format
Eye contact
style
Writing
quantity
clarity
Simple
quality
voc
compound
complex
Sentence complexity
Content / ideas
Language
WHAT
to test?
syntax
accuracy
Organization
Paraphrasing
appropriacy
Topic sentence
Supportive
sentences
Speaking
Reading
Skills
Listening
Knowledge
Writing
WHAT
to test?
Language
lexis
semantics
Grammar
morphology
syntax
Reading
WHAT
Reading aloud
pronunciation
to test?
Silent reading
Subskills
• Identify main facts & details,
Stresses & intonation
• Relate cause & effect
• Identify sequence of events, predicting
outcomes
• inferring meaning from contextual clues
Listening
WHAT
to test?
Subskills
• Identify main facts & details,
• Relate cause & effect
• Identify sequence of events, predicting
outcomes
• inferring meaning from contextual clues
Grammar
WHAT
to test?
Syntax (sentence structure)
Mechanics: e.g Punctuation, capital letter, comma, etc..
pronouns
Rules of grammar
nouns
tenses
verbs
prepositions
subject & verb
agreement
comparatives
adjectives
adverbs
Speaking
quantity
quality
vocabulary
Content / ideas
Sentence complexity
Language
WHAT
clarity
to test?
syntax
grammar
fluency
confidence
Pronunciation
Presentation skills
Organization
accuracy
appropriacy
format
Eye contact
style
Writing
quantity
clarity
Simple
quality
voc
compound
complex
Sentence complexity
Content / ideas
Language
WHAT
to test?
syntax
accuracy
Organization
Paraphrasing
appropriacy
Topic sentence
Supportive
sentences
What makes test questions difficult?
THINKING LEVELS
Designing test questions
in accordance with the
levels of thinking
Bloom’s taxonomy
Old version, 1956,
Benjamin Bloom
New version, 1990’s , Lorin Anderson
BLOOM’S TAXONOMY.
Evaluation
Argue, assess, justify, judge, rate,
support, value, evaluate
Synthesis
Analysis
Arrange, collect, compose, plan,
construct, create, design, develop
Analyse, calculate, categorise,
criticise, contrast, distinguish
Application
Apply, choose, demonstrate, illustrate,
solve, use, interpret
Comprehension
Classify, describe, discuss, explain,
express, recognise, identify, report
Knowledge
Arrange, define, list, memorise,
name, relate, recall, repeat, state
Bloom’s taxonomy – Knowledge level
Useful verbs
Tell
List
Describe
Relate
Locate
Write
Find
State
name
Sample questions stems
•
•
•
•
•
•
•
•
•
•
What happened after….?
How many…..?
Who was it that……..?
Can you name the …….?
Describe what happened at…..?
Who spoke to…..?
Can you tell why….?
Find the meaning of …..?
What is …….?
Which is true or false…….?
Bloom’s taxonomy – Comprehension level
Useful verbs
Sample questions stems
Explain
Interpret
Outline
Discuss
Distinguish
Predict
Restate
Compare
Translate
Describe
• Can you write in your own
words….?
• Can you write a brief outline….?
• What do you think could happen
next….?
• Who do you think….?
• What was the main idea…..?
• Who was the key character…..?
• Can you distinguish between….?
• Can you provide an example of …..?
• What do you mean ….?
• Can you provide a definition for….?
Bloom’s taxonomy – Application level
Useful verbs
Solve
Show
Use
illustrate
Construct
Complete
Examine
Classify
Sample questions stems
• Do you know another instance
where….?
• Could this have happened in ….?
• Can you group by charateristics such
as….?
• What factors would you change if…..?
• What questions would you ask of…..?
• From the information given, can you
develop a set of instructions
about……?
• Would this information be useful if you
had….?
Bloom’s taxonomy – Analysis level
Useful verbs
Analyse
Examine
Compare
Contrast
Investigate
Categorise
Identify
Explain
Separate
Sample questions stems
• Which events could have
happened….?
• What do you see as other possible
outcomes…?
• How was this similar to….?
• Can you explain what must have
happened when…?
• What are some of the problems of ….?
• Can you distinguish between….?
• What were some of the motives
behind….?
Bloom’s taxonomy – Synthesis level
Useful verbs
Create
Invent
Compose
Predict
Plan
Construct
Design
Imagine
Propose
devise
Sample questions stems
• Can you design a…..to….?
• Can you see a possible solution to ….?
• If you had access to all resources how
would you deal with….?
• What would happen if …..?
• How many ways can you……?
• Can you write a new recipe for a tasty
dish?
• Why not compose a song about…..?
Bloom’s taxonomy – Evaluation level
Useful verbs
Judge
Select
Choose
Decide
Justify
Debate
Verify
Argue
Recommend
determine
Sample questions stems
• Is there a better solution to ….
• Judge the value of….
• Can you defend your position
about…..?
• Do you think……is a good or bad
thing?
• What changes to ……would you
recommend?
• Do you believe….?
• Are you a …..person?
• How would you feel if….?
• How effective are…..?
• What do you think about…….?
BLOOM’S TAXONOMY.
Skills
Subskills
Question types
Levels
BLOOM’S TAXONOMY.
Skills
Subskills
Question types
Levels
Skills
Reading
Sub-skills






Listening







Question types
Identifying main facts & details
(FD)
Recognizing cause & effects
relationship (CE)
Inferring meaning from
contextual context ( ICC)
Predicting outcomes (PO)
Sequence of events (SOE)
Evaluating writer’s attitude &
view




Identifying facts & details
relate cause & effect
identify sequence of events
Inferring meaning from
contextual clues.
Predicting out comes
Inferring speaker’s attitude &
emotion (sad, happy, worried)
Inferring relationship between
speakers.








True/ false
Re-arrangement
Structured
Open-ended (subjective
response, free writing)
MCQ (wh-question,
matching, completion
with options)
Cloze procedure
MCQ with pictures or
without pictures.
Fill in the blanks/
completion item.
Labelling.
Matching
True / false.
sequencing
Question
levels
BARRETT’S TAXONOMY.
Bloom’s(Know,Comp,
App, Ana, Syn, Eva)
(MCQ, T/F, gap-fill,
matching, sequencing,
open-ended)
Barrett’s (LC, RO,
IC, Eva, Appr)
Question levels
Question types
knowledge
Table of
Specifications
LSRW
Curriculum
specs
Time allocation
Skills & subskills
Mark allocation
Question
Sections/numbers
LOGO
Dinh Cong Triet