Transcript Analysis of Test Formatsx

Authors:
Hector Rojas
Daniel Blanco
Yuly Caraballo
STRENGTHS



Objective
Easy to design
Easy to include a
representative example
WEAKNESSES
Easy to guess
 Only for grammar,
vocabulary, pronunciation
 Not good for testing the
skills.
 Students need to decide on
what answers accepted as
correct.

STRENGTHS
 Objective
 Easy to correct
 Provides clear contexts
 It can be used to correct
different strategies.
WEAKNESSES
 Extremely difficult to
design an item with only
one possible answer
(generally solved by
providing a list of terms)
 Need to think about
possible answers (if a list is
not provided)
STRENGTHS
Easy for the teacher to
create
 Easy for the students to
understand
 Easy to correct and
design .

WEAKNESSES
Unexpected correct answers
are possible
 Mainly for vocabulary
 It can be irritating for
students

STRENGTHS
 It is easy to construct
 Requires simple recognition
of information (eliminatting
the guessing factor)
 It is good to evaluate
listening for specific
information
WEAKNESSES
 Depend on knowledge of
language
 Difficult to grade
depending on evaluation
criteria
 It can be irritating for
students
STRENGTHS
Easy to correct and grade
Scores are less affected by
guessing
 Good for checking
understanding
 Usually it requires less time for
test takers to answer


WEAKNESSES
 Difficult to construct
 It does not allowed test takers
to demonstrate knowledge
beyond the options provided
 Emphasize in recognition and
guessing rather than in
production
STRENGTHS
 Easy to write
 Can cover a wide range of
difficulty levels
 Usually it requires less time
for test takers to answer
 Usually it is easy to grade
and score
WEAKNESSES



Guessing element
The score can not be
reliable
It does not allow test takers
to demonstrate broad
range of knowledge
STRENGTHS




Good for testing some
structures
Easy to write and
administer
Specifically tackle one
grammatical objective
Allow recognition of a
connection between
“grammar” and “meaning”
WEAKNESSES


Very artificial
It can be more than one
possible transformation in
some cases
STRENGTHS


It is good for checking if
students
comprehend texts.
It is useful to test the
student's ability to analyze
a concept.
WEAKNESSES


It is not easy to score
It takes more time to
correct and answer.
FORMAT
RECOMMENDATIONS
Matching
•
•
•
•
•
Give more options than questions.
Each answer should have distracters.
Try meaningful items.
Make sure there’s only one possible selection.
Do not ask to draw lines to match.
Gap filling
•
•
•
•
•
No blank at the beginning.
Give enough context.
Avoid clues.
Only one answer
Give line with the same length
Cloze
Don’t use nth word, for example : 5th , 3rd
Completion
• Avoid the statement becomes ambiguous.
• Use blanks of the same length throughout the test so that the length is
not a clue.
• If possible, put the blank at the end of a statement rather than at the
beginning. Asking for a response before the student understands the
intent of the statement can be confusing and may require more reading
time.
Multiple choice
• Be clear and concise in your word and phrase choices.
• Make sure that there is only one clearly correct answer from the options
given to the student.
• Randomly distribute the correct answer options i.e. A, B, C, D etc so that
there is not a clear pattern that becomes obvious to the student.
True-False
• Make sure that the answer is clear and that it could not be either or.
• Try not to use negative questions such as “this novel was not written
by....”
• Use a random order of true and false responses with your test questions to
avoid creating a pattern.
• Use more false questions than true questions as they have been proven to
cater towards higher cognitive level students.
Questions
• It has to be easy to understand.
• Check how you will judge answers.
Multiple choice items
She had to help the ______ old man up the stairs.
A. weak B. slowly C. try D. wisdom
Adverb Verb
Noun
A. weak B. strong C. energetic D. athletic
All words should belong
to the same grammatical
category.
She needs to get up earlier so she’s buying an
______ clock.
Grammatical clues make
the answer obvius.
A. time B. alarm C. watch D. bell
She needs to get up earlier so she’s buying
______ clock.
A. a time B. an alarm C. a watch D. a bell
Deleting the indefinite
article would make this
item better in terms of
objectivity.
They needed lots of training to operate such
______ equipment.
A. easy B. sophisticated C. blue D. wise
The level of difficulty of
the distractors highlights
the correct alternative.
“Sophisticated” clearly contrasts with the rest of the options, so students
might select it right away.
She sent the _______ yesterday.
A. letter B. gift C. food D. books
Options that collocate with
the verb in the sentence.
(Differentiators)
“Post”, “friend” and “courage” are good replacements for “gift”, “food” and
“books” that are items that can also be sent or mailed.
She wrote a _______ yesterday.
A. letter B. gift C. friend D. books
Completion
He was a very nerv_____ person.
The absence of context will
lead to the acceptance of
unexpected correct answers.
An advanced learner could get to the conclusion of completing with the
letter “y” to form the adjective “nervy”, which means “bold” or “offensive”.
He was always alert. He was a ________ person.
In this case, given the context,
the correct option is “nervous”.
That was a care______ answer.
Yesterday he got on the wrong bus. So today, he
was care_______ to find the right one.
Words that may take
two opposite suffixes
lead to ambiguity.
Enough context will
permit the student know
which alternative is the
most appropiate one.
True-False
Columbus discovered America.
Columbus sailed to the new world in 1942.
The use of trivial information must
be avoided . Given the fact that
they are more likely to be true
than ambiguous information.
If used, they must be presented
in a more objective way.
Since penguins can’t fly, they cannot swim very well either.
Avoiding the use of
negative statements
will prevent confusion
and contradiction.
Since penguins are unable to fly, they swim badly.
Rattlesnakes live in North America, have fangs,
feed on small rodents and crush their prey.
Rattlesnakes crush their preys.
The first three
clauses are true,
however the last
one is not.
The entire sentence
must be either true or
false, otherwise it will
create confusion.
Matching
The instructions do not clarify how
many times items in column 2 may be
used or a basis for matching.
Match column 1 and comlumn 2:
___1. Sit-and-reach
___2. 50-yard dash
___3. Pull-up
___4. Shuttle run
___5. Balke treadmill
___6. Dyer backboard volley
___7. Disch putting
___8. Biopsy
c. Muscle fibers
h. Golf
f.Tennis
a.Vo₂max
e.Agility
g.Arm strength
d.Speed
b.Flexibility
Items are too heterogeneus, making the
answers too obvious.
Contain the same
number of items, so
the last item could
be answered by
elimination.
Transformation
Change the sentences to the future simple:
Doug studied English yesterday.
Doug went to the movies with his family.
He eats a hamburger.
They enjoy the day in the mall.
The activity lacks context
The sentences
should be provided
in the same tense, to
prevent confusion.
This exercise requieres more contextualized instructions, such as:
This is what Doug usually does on the weekend, write about what
he is going to do next the weekened.
Item analysis: measure the performance of each
item in a test and help us to revise and improve
both, items and test as a whole.
Use:
-To build reliablity and validity into a test.
- More diagnosis information on students.

Item difficulty: deals with the number of
students who answer correctly. The best
items are that are neither too difficult or too
hard.
IF = # of students answering the item correctly
Total of students responding that item.

Item discrimination : differenciates those
test takers with high - abilities from those
with low - abilities.

Distractor efficiency: deals with the way a
distractor lures test takers, especially those
with low abilities.
Brown, Douglas (2004) Language assessment:
Principles and Classroom prractices. Longman,
London.
Cunningham, George (1998) Assessment in the
Classroom: Constructing and interpreting Tests.
Routledge, London.
Morrow, James and others (2010)Measurement and
Evaluation in Human Performance. Human kinetics,
Illinois.
Madsen, Harold (1983) Techniques in testing. Oxford
University Press, New York.