Transcript e-status: a Problem-based Learning Web Tool Powered by R

e-status: a Problem-based
Learning Web Tool Powered by R
J. A. González, L. Marco, L. Rodero,
J. A. Sánchez
Statistics and Operations Research Dept.
Technical University of Catalonia, Barcelona
Outline







Introduction
Motivation
Architecture
Example
The feedback issue
Testing e-status
Conclusion







(What)
(Why)
(Where)
(How)
(Who)
(Is it)
()
08/25, COMPSTAT2008, Oporto
2/26
Introduction



e-status is a web-based tool
(http://ka.upc.es/).
It helps the students with exercises
corrected automatically.
R is the engine working behind:
• for computations,
• for graphics,
• for some management.
08/25, COMPSTAT2008, Oporto
3/26
What it is not
e-status is neither:


a static list of problems;
•
Problems can (have to) be very dynamic.
a statistics course or an online textbook;
• Related materials may be linked to the problems,
they are not structural components.

a demonstration tool (like applets collection);
•
Focus is put on the students, so that they play the
active role.
08/25, COMPSTAT2008, Oporto
4/26
Why a problem-oriented tool?

Statistics is conceptually hard to learn:

Hands-on techniques (PBL, CooperativeL)
are effective to reach highest capabilities.
However, lack of numeracy skills is a
matter of concern:

• uncertainty, randomness, …
• Students are not even used to managing data or
making simple computations
08/25, COMPSTAT2008, Oporto
5/26
Why a problem-oriented tool? (2)

Introductory problems are usually solved
in the classroom:
• Content-overloaded courses!
• Great demand of sample problems.
• Lists published; null flexibility.
• Teachers cannot revise all the students exercises,
•
if check requested.
Scarce information about their performance.
08/25, COMPSTAT2008, Oporto
6/26
In the beginning
2003: All-in-one
MS OS +
MS IIS +
MS SQLServer
+ java + PHP
e-status
server
Web
User
08/25, COMPSTAT2008, Oporto
7/26
Architecture 2007
Linux +
Apache +
Cake-PHP
MySQL
any
internet
browser
08/25, COMPSTAT2008, Oporto
8/26
Highlights

Distributed architecture

R runs in the background, responding to
the application requests
• one server for e-status and web,
• one server for DataBase,
• one server for authentication [optional],
• one server for R.
• RServe is used as communication software.
• The PHP client-side implementation was
developed for e-status.
08/25, COMPSTAT2008, Oporto
9/26
An example
Two urns with yellow and blue balls.
One coin to choose the urn.
The mix of colors is random, so
questions like:
You flipped a coin, and a blue
ball was drawn from the selected
urn. Find the probability that
the coin is heads.
have a different solution each time.
08/25, COMPSTAT2008, Oporto
10/26
The example in depth
As data is not fixed in the problem, one has:


to create the data, and
to compute the answers.
R allows to resolve both points.
Deciding the composition of the urns:
h = sample(1:2,1)
n = sample(4:15,2,repl=1)
m1 = sample(2:(n[1]-1),1)
m2 = sample(2:(n[2]-1),1)
#
#
#
#
h=1: heads on left
number of balls [left, right]
yellow on left
yellow on right
08/25, COMPSTAT2008, Oporto
11/26
The example in depth (2)
Finding the probability P(heads | blue), using Bayes’ Theorem:
t = 3-h
n_B.H = n[h] - ifelse(h==1,m1,m2)
P_B.H = n_B.H/n[h]
P_B = P_B.H*0.5 + (n[t] - ifelse(h==1,m2,m1))/n[t]*0.5
P_H.B = P_B.H*0.5/P_B
The exact solution is P_H.B; the student’s answer has to be compared
with this value (within some tolerance, chosen by the teacher).
08/25, COMPSTAT2008, Oporto
12/26
The example in depth (3)
In this case, the problem
includes a descriptive
picture too:
q1 = c(rep(7,m1),rep(4,n[1]-m1))
q2 = c(rep(7,m2),rep(4,n[2]-m2))
q1 = sample(q1)
q2 = sample(q2)
x = c(1,3,5,7,9,2,4,6,8,3,5,7,4,6,5)
T = sqrt(3)
y = c(1,1,1,1,1,rep(1+T,4),
rep(1+2*T,3),rep(1+3*T,2),1+4*T)
X1 = x[1:n[1]]
Y1 = y[1:n[1]]
X2 = 15+x[1:n[2]]
Y2 = y[1:n[2]]
bx=c(-1,-1,11,11)
by=c(y[15]+1,0,0,y[15]+1)
graph=ini_imagen(500,250)
op=par();
par(mar=c(0.5,0.5,0.5,0.2));
plot(c(-5,30),c(0,10),type=“n”,
asp=1, axes=0,xlab=“”,ylab=“”)
symbols(X1,Y1,circ=rep(1,n[1]),bg=q1
,asp=1,inc=0,add=1)
symbols(X2,Y2,circ=rep(1,n[2]),bg=q2
,asp=1,inc=0,add=1)
lines(bx,by)
lines(15+bx,by)
text(5,10,lab=mon[h],cex=2)
text(20,10,lab=mon[t],cex=2)
par(op)
fin_imagen()
08/25, COMPSTAT2008, Oporto
13/26
Typically, an exercise looks like this:
08/25, COMPSTAT2008, Oporto
14/26
The Feedback issue
Feedback allows and improves learning.

Short-term: the student verifies the
correctness of answers
• Sometimes, the system can give a clue.

Long-term: everything is saved. These
records can show strengths and weaknesses
• particularly, for the student,
• in general, for the teacher.
08/25, COMPSTAT2008, Oporto
15/26
BLOCKS
Example: résumé
PROBLEMS
08/25, COMPSTAT2008, Oporto
16/26
Example: marks analysis by question
Gasp!
Very bad results for
7th question…
Rephrase it?
08/25, COMPSTAT2008, Oporto
17/26
R as a controller
R
model
K=seq(n0,n1)
P=runif(3,0.2,0.8)
prob=P[1]
x=0:(n1-n0)
p1=dbinom(x,n1-n0,prob)
lambda=10*P[2]
p2=dpois(x,lambda)
p=sqrt(p1+p2)
p=round(p/sum(p),3)
p[1]=p[1] + 1 - sum(p)
fun=matrix(c(K,p),ncol=2)
colnames(fun)=c("K", "P(X=K)")
...
if (estatus.decimales_correctos(so...
{
resultado_ <- 1
}else{
resultado_ <- 0
}
R
check
Try again?
08/25, COMPSTAT2008, Oporto
18/26
R as a controller (2)


First release compared answers with the
(unique) solution.
At present, the check with R allows richer
assessment of the answers.
• Example: Given a random variable X following the
law N(120, 20), enter an interval [a, b] such that
P(a < X < b) be between 1/4 and 1/3.
• Solution: p
= pnorm(b,120,20)-pnorm(a,120,20)
result = p > 1/4 & p < 1/3
08/25, COMPSTAT2008, Oporto
19/26
Testing
e-status has been experimentally tested:
 2006, Dentistry School (University of
Barcelona);
•
94 participants from 120 enrolled
•
the year, before the final exam
subject contents were split up
• 2 parallel groups; six problems each at the end of
Block A
Block B
08/25, COMPSTAT2008, Oporto
20/26
Testing (2)

2008, Engineering School (Technical
University of Barcelona)
•
210 assigned; 197 made the exam; 145 used e-status
actively
• 2 parallel groups; two common blocks, plus one
•
specific per group, during the year
the assignation involves:
2 samples tests,
independent
Block 1
>2 samples
tests, blocked
Block 2
2 samples
tests, blocked
08/25, COMPSTAT2008, Oporto
>2 samples tests,
independent
21/26
Outcomes
2008:
2006:
•
•
•
•
The final exam included six
questions, three related to
A and three related to B.
Single evaluator.
Compared the difference in
mark within student:
YA – YB
It is expected that the
difference is positive for A
and negative for B (average).
•
•
•
•
Four questions (one for each
item of the 3rd block).
Two evaluators (no confusion:
2 samples tests and >2 samples
tests).
Each mark modelled with a
linear mixed-effects model
(group A or B, and student).
It is expected that A and B
effects are statistically
significant.
08/25, COMPSTAT2008, Oporto
22/26
Results
1442 exercises: the students
performed mainly during the
holiday period (decreases
interaction).
+7500 exercises: the
students performed mainly
before deadlines; less
activity in third block.
1200
Number of exercises
1000
800
600
400
200
0
8
8
8
8
8
8
8
8
8
8
8
00
00
00
00
00
00
00
00
00
00
00
/2
/2
/2
/2
/2
/2
/2
/2
/2
/2
/2
3
3
4
4
4
4
5
5
5
5
6
/0
/0
/0
/0
/0
/0
/0
/0
/0
/0
/0
15
25
01
11
18
27
04
11
18
25
01
08/25, COMPSTAT2008, Oporto
23/26
Results (2)
The measured effect of estatus exercising was 0.96
points (95% CI: 0.20-1.72)
on a ten-point scale.
Among those 94 students
who employed e-status, the
effect size was 1.27 (95%
CI: 0.35-2.19).
We assumed one effect
(not one for A and one for B)
The model failed to find
differences among all the students.
Taking only active participants,
there is some evidence just for
group B (estimation: +0.7 points),
but not for group A.
An explanation: complex subject
─►complex experiment.
Incidental uncertainties could
have increased the noise in the
outcome.
08/25, COMPSTAT2008, Oporto
24/26
Conclusion

e-status has been clearly enriched with R

In accordance with EHEA:

ITs it is not an added value by itself
• powerful and simple.
• active learning method,
• utensil to measure student effort.
• committed to promoting IT tools for getting the
students motivated and to awaken their interest in
statistics.
08/25, COMPSTAT2008, Oporto
25/26