Transcript slides on Java Auto grader

WeBWorK and JUnit
an automated code grading system
Summary
•
•
•
•
•
Introduction
JUnit basics
JUnit/WeBWorK extension
sample session
Future directions
Introduction
• Want to make a system to
– automatically grade small java assignments
– usable from the very first day of class
• little startup
– provide appropriate feedback at a more
accessible level than compiler errors
Automatic grading
• Want a system to grade
– program fragments
– entire methods or combinations of methods
– entire classes (or multiple classes)
• Provide immediate feedback
– decreases turnaround cycle
– encourages exploration
An accessible system
• Many students find the first several days in
programming to be terrifying
–
–
–
–
new systems using different paradigms
programs that work differently
error messages that are undecipherable
have to write a lot of code just to get the
simplest program
An accessible system
• Our system is designed to
– remove the development environment
– allow program fragments
– provide immediate feedback
• Our system is not designed to replace the
professor
– but it will grant the student greater and more
timely feedback
Provide appropriate feedback
• Compilers check syntax not semantics
• Exceptions catch only gross errors
• Unit testing allows very specific testing to
home in on what is correct and isn’t correct
JUnit basics
• From www.junit.org
– “JUnit is a simple framework to write
repeatable tests.”
JUnit basics
• As you write classes you write tests
– for a LinkedList class
• code to check for normal functioning of linked list
• code to check exceptional circumstances
– delete from empty, removing non-present element
JUnit architecture
• tests are usually contained in separate class
– LinkedList, LinkedListJUnit
• JUnit will automatically run all methods
that begin with test…
• Test methods contain code to notify JUnit of
failures
– Failures may include not raising an exception
• JUnit keeps track of success/failures
JUnit example
• Imagine a Money class for keeping track of
various international moneys
• Constructor, add methods
JUnit test for Money
public class MoneyJUnit extends TestCase {
public void testSimpleAdd() {
Money m12CHF= new Money(12, "CHF");
Money m14CHF= new Money(14, "CHF");
Money expected= new Money(26, "CHF");
Money result= m12CHF.add(m14CHF);
assertTrue(expected.equals(result));
}}
Common JUnit methods
•
•
•
•
•
Assert.assertTrue(bool);
Assert.assertEquals(X , X);
Assert.assertNotNull(Object);
Assert.fail();
All have optional messages to attach.
Handling exceptions
public void testThrowsException() {
try {
Money m = new Money(10,”XYZ”);
} catch (Exception IllegalTenderException ite) {
// do nothing, it was supposed to do this
return;
}
Assert.fail(); // shouldn’t get here
}
• Note uncaught exceptions fail by default.
JUnit and WeBWorK
• Have students type in a
class/method/program fragment
• Insert the code into a template designed by
the problem creator
• Compile the code, reporting errors if
relevant
• Use the standard testing mechanisms to
measure relative correctness of code
Webwork opening screen
JUnit opening screen (empty)
JUnit screen filled well
JUnit correct result
Webwork answer copied back
Webwork says it’s ok
Bad JUnit answer
JUnit agrees
Writing JUnit problems
• Write a template class
• Write a JUnit class to test it
• Write a WeBWorK problem
Sample template class
public class ArraySort{
private int[] numList=
{12, 10, 8, 6, 4};
public void print(){
}
public ArraySort(){
sort();
print();
}
public int[] getArray(){
}
public void sort(){
replaceme
}
for(int i = 0; i<5; i++){
System.out.println( "" + numList[i]);
}
}
return numList;
public static void main( String [] args){
ArraySort ast = new
ArraySort();
}
Sample JUnit test
import junit.framework.*;
public class ArraySortJUnitTest extends
TestCase
{
private int[] startArray = {12, 10, 8, 6, 4};
private ArraySort ast = new ArraySort();
private int[] endArray = ast.getArray();
public void testSlotThree(){
Assert.assertEquals(startArray[2],
endArray[2]);
}
public void testSlotFour(){
Assert.assertEquals(startArray[1],endArray[3]);
}
public void testSlotFive(){
Assert.assertEquals(startArray[0],endArray[4]);
}
public void testSlotOne(){
Assert.assertEquals(startArray[4],
endArray[0]);
}
public void testSlotTwo(){
Assert.assertEquals( startArray[3],
endArray[1]);
}
public static Test suite(){
return new TestSuite(ArraySortJUnitTest.class);
}
public static void main(String[] args){
ArraySortJUnitTest ajst = new ArraySortJUnitTest();
}
}
System Architecture
• WeBWorK displays background info
• User clicks on link which runs traditional
Perl/CGI script
– Displays exact question
– Collects user input
System Architecture (2)
• On user input, X.java template expanded
• Expanded X.java and XJUnit.java copied to
unique temp directory
• Code compiled, errors reported back to user
System Architecture (3)
• JUnit tests run on code
– Code run inside strict Java sandbox
– Not currently CPU Limited
• Number of JUnit tests passed translated into
score [0.0,1.0] (0.1 increments).
• On user click, passed back to WeBWorK
which records score.
Sample JUnitWebWork problem
• Currently involves copying lots of code to
setup transfer between JUnit and CGI-Perl
– Will be greatly simplified
• Elements edited by problem writer
– Problem description
– Name of template class
Next steps
• Writing problems
• Cleaning/testing code
• Improving interface with WeBWorK
– Want to remove the extra clicks
– Use WeBWorK 2.0
– Provide Web interface for editing
• Increasing security
– Infinite loops still problem
– Issues with doing file i/o