Transcript EZilbert CBT Science Tryout

California Computer Based
Testing Tryout in Science
CCSSO Conference on Student Assessment
June 21, 2013
Eric E. Zilbert, Interim Director
Assessment Development and Administration Division
CALIFORNIA DEPARTMENT OF EDUCATION
Tom Torlakson, State Superintendent of Public Instruction
Purpose of the Study
TOM TORLAKSON
State Superintendent
of Public Instruction
• Collect feedback and data useful for state
in transitioning to CBT and CAT.
• Provide students and schools an
opportunity to try CBT.
• Develop and administer new technologyenhanced (TE) items; evaluate the
feasibility of artificial intelligence scoring.
• Conduct special studies of items and tests.
2
Test Design
TOM TORLAKSON
State Superintendent
of Public Instruction
• Science Tests
– Grades 5 & 8 Science
– High School Biology
• Item types
– 40 traditional single select multiplechoice items
– 20 technology-enhanced items
• Drag-and-drop & hotspot functionality
• Simulations
• Artificial intelligence scoring
3
Sample
TOM TORLAKSON
State Superintendent
of Public Instruction
• 21,473 students,
193 schools,
133LEAs
• Represented all
areas of the state
• Included all
demographic
subgroups
4
Key Findings
TOM TORLAKSON
State Superintendent
of Public Instruction
• Of those LEAs and schools that
participated, the majority indicated:
– Their school was “somewhat” or “very well”
prepared for CBT.
– The technology functioned as expected.
– The CBT system was easy to use.
• The majority of students expressed
satisfaction with the experience.
– Instructions, pictures, and item types were
easy to understand and intuitive.
5
Key Findings
TOM TORLAKSON
State Superintendent
of Public Instruction
Reported problems during testing were
primarily related to:
– Software Issues: needed to update
software, difficulty logging on, difficulty
with pausing and reactivating student
testing sessions.
– Server Problems – both local and offsite
• Inhibited dragging and dropping.
• Booted students from the system.
– Lack of resources, including computers,
technical help, and Wi-Fi connections.
6
Key Findings
TOM TORLAKSON
State Superintendent
of Public Instruction
• Multiple choice items performed similarly
in the CBT compared to the previous
paper administration (difficulty,
discrimination).
• Artificial intelligence scoring was
successfully used for:
– Graphs, equations, and numeric-entry
responses, and
– Short constructed-response items.
7
For Further Information
TOM TORLAKSON
State Superintendent
of Public Instruction
Eric E. Zilbert
[email protected]
CBT Tryout Report
www.cde.ca.gov/ta/tg/sr/documents/cbttryoutrpt.pdf
8