Transcript Testovid - an environment for testing almost any aspect of student

Testovid - an environment for
testing almost any aspect of
student assignments
I. Pribela, S. Tošić, M. Ivanović, Z. Budimac
Risan, September 2007
Introduction


Continuous increase of enrolled students
How to avoid straining of teachers:





decrease number of students
increase number of teachers
transfer some work load from teachers to computers
Recent advances in computer technology made
third option possible
One of major conditions is ability to represent
student work on the computer
Automated assessment

Assessment phases






question / assignment selection
question / assessment deployment
answer / solution gathering
answer / solution grading
student / teacher notification
statistical analysis of results
Advantages

Large student groups

faster assessment

moving work load from teachers

leaving more time for more productive aspects of
teaching
Advantages

Small student groups

self-assessment allows students








to monitor their own progress
to point to difficult parts of lectures early
teachers can easily note common problems and difficult
topics
question selection can be better adjusted to individual
needs
assessment results are immediate
same questions (assignments) can be repeated as much
as needed
questions can contain graphics, sound, animations, and
other multimedia content
increase of assessment objectivity
Disadvantages

Type of assessments supported



short multiple choice tests
fill-in-the-blanks tests with fixed correct answers
More appropriate for natural sciences than for
social sciences
Assessment of programming



Programming assignments are perfect example
for automated assessment
Besides generic, special systems for assessing
of computer programs were also developed
There are two distinct types of computer
program assessment:

correctness assessment


does the student program solve given problem?
optimality assessment

how efficient the student program is?
Assessing correctness

Oldest correctness assessment method is



Input data




created by teacher and given to the student
created by the student
hidden from the student
Selecting input data



running student programs against various sets of input data
comparing output form student program with correct one
fixed
chosen randomly every time
Output data


compared with fixed data corresponding to given input
compared with output from correct solution
Assessing optimality

Early systems focused on execution speed
and precision of numeric results

Recently attention has shifted towards
readability and code style
Testovid

Batch system

Students can test solutions by themselves

Teacher can test all tasks at once

Generates execution reports

Uses Apache Ant
Apache Ant

Apache Ant is a tool for automatization on a build
proceses

Looks like Make, but written in Java

Demands Java virtual machine

Most appropriate for Java projects

Uses XML for description of build processes

Open source
Implementation

Client-server architecture


Server
Clients (students, teacher)
Student
Instructor
Student
Student
Batch testing
Interactive testing
Server
Student
Student
Computer lab
Implementation

Directory structure of a server
/
system
test
log
OS
OOP
a) Instructors
temp
OS
OOP
TMP238
students
OS
OOP
John
Mary
TMP189
John
Mary
Steve
John
TMP845
Steve
John
test data
Bob
Bob
c) Temporary directories
b) Log files containing test results
d) Student directories for storage
of latest attempts
Implementation

Two Ant build files



File for single testiranje
File for batch testing
Two configuration files


File with testing modules
File with weights of every module
Testing module
targets.all=test01,test02,test03,test04,test05,t
est06,
test07,test08
test01.name=Kompajliranje
test01.score=1
test02.name=Up
test02.score=1
test03.name=Down
test03.score=1
test04.name=Up - Gornje ogranicenje
test04.score=2
test05.name=Down - Gornje ogranicenje
test05.score=2
test06.name=Sigurnost
test06.score=1
test07.name=Greske u stilu
test07.score=1
test08.name=Greske ili upozorenja u stilu
test08.score=1

List of used modules

Name of every module

Number of points for
each module

All marks are binary
Testing module implementation
<target name="test07">
<checkstyle config="checks.xml"
file="synchronization/primitives/Sema
phore.java"
failureProperty="test07.fail"
failOnViolation="false"
maxErrors="15">
<formatter type="xml"
toFile="style.xml"/>
</checkstyle>
<style in="style.xml" out="style.rep"
style="style.xsl"/>
<condition property="test07.advice"
value="Postoji vise od 15 gresaka u
stilu.">
<equals arg1="${test07.fail}"
arg2="true"/>
</condition>
<property name="test07.advice"
value=""/>
</target>

One module is one Ant
target

Result of execution is
string that contains
message about error
Further Work

Integration of system with Svetovid
environment

Creation of base of usefull modules for
testing

Adding system for plagiarism detection
The end
Thank you for your attention