Transcript ALA-Reader - Personal.psu.edu

Free software that scores
biology field reports and classroom essays
T3 International Conference
Denver, CO February 25th, 2006
Dr. Roy B. Clariana – Penn State U.
[email protected]
http://www.personal.psu.edu/rbc4
1
Presentation Goals


This presentation will demonstrate how to
download and use a software tool called ALAReader (analysis of lexical aggregates) to
score students’ biology essays
My goal: To recruit action researchers from
this audience
2 of 24
Presentation flow






Quick ALA-Reader demo – to see it work
Brief look at its genesis and how it works
Step-by-step demo – how you set it up and
use it in your classroom Monday morning
Review the data files produced
One more run
Final questions, Show of hands
3 of 24
Quick ALA-Reader Demo





Place student essays, terms file, expert
referent essay, and ALA-Reader software into
the same folder (click)
Double click ALA-Reader
Click the Mark L button
Type in the file name of a student’s essay
Observe scores
4 of 24
Genesis of ALA-Reader




We are interested in concept maps, and developed
software called ALA-Mapper to score concept maps
Then we noticed that teachers frequently associate
concept maps and essays (i.e., concept maps are a
good organization tool)
For example, Lomask et al. derived concept maps
from student’s essays in order to score the
Connecticut statewide science assessment
So we developed ALA-Reader to score essays based
on ALA-Mapper
5 of 24
Concept map validity
oxygenate pulmonary
CO2 lungs vein
artery
ventricle
blood
atrium
left
atrium
pulmonary
vein
lungs
remove oxygenate
CO2
blood
Concept maps are
generally considered to be
valid and reliable
measures of science
content knowledge (RuizPrimo, Schultz, Li,
Shavelson, CREST in
California, Rye and Rubba
(2002), Jonassen, Beissner,
& Yacci, 1993. . .).
6 of 24
Types of knowing




Declarative knowledge (semantic and episodic)
Procedural knowledge
“Structural knowledge” – knowledge of the
interrelationships of the domain elements, may be
most essential for higher-order learning structural
knowledge seems to be part of both declarative and
procedural knowledge (so it may be a manifestatio of
the other two)
Both concept maps and essays (esp.
compare/contrast prompts) capture aspects of all
three depending on what components of the map or
essay you measure
7 of 24
“Structural knowledge”





Vygotsky (in Luria, 1979); Miller (1969) cardsorting approaches
Deese’s (1965) ideas on the structure of
association in language and thought
Recent neural network representations (e.g.,
Elman, 1995)
Kintsch and Landauer’s ideas on representing text
structure, and latent semantic analysis (Pearson);
ETS; Vantage Learning Corp.
Concept maps (Jonassen…)
8 of 24
measuring it …



One measure of structural knowledge consists
of the “links” in a concept map
A Link can also can be considered to be a
‘proposition’ that includes the two concepts
that are joined by the link
Links are one component of concept maps
that are easy to measure
9 of 24
.. map w/ proposition array
(n2-n)/2 pair-wise comparisons
left atrium
right ventricle
pulmonary vein
left atrium
lungs
oxygenate
pulmonary artery
pulmonary vein
deoxygenate
right ventricle
pulmonary artery
lungs
deoxygenate
Map
LA
0
0
0
1
0
0
L
1
1
1
1
0
OX PA PV DOX RV
0
0
0
0
0
0
1
0
0
0
-
oxygenate
Proposition Array
Most studies use only link (edge)
information, usually called “propositions”.
10 of 24
Convert raw map data into scores



Currently, we use a data reduction and
comparison approach called Pathfinder
network representation (UNM, Schanveldt,
1990) to turn counts into scores
PFNets describe the least weighted path to
connect the terms in the data proximity array
Scores are established by comparing the
participant’s PFNet to a referent (expert)
PFNet, and calculating the number of
common links (the intersection)
11 of 24
Concept map & traditional test scores


Example: During a lesson on the heart,
students created concept maps while reading,
then took a test. The correlations between
the concept map link scores from ALA-Mapper
and terminology multiple-choice test scores
are:
Taricani & Clariana (2006), r = .78
Poindexter & Clariana (2006), r = .77
12 of 24
We use the same approach for
ALA-Reader
Text
PFNet
… an electrical signal starts the
heartbeat, by causing the
atrium to contract. The blood
then flows through the
pulmonary valve into the
pulmonary artery and then into
the lungs. Once inside the
lungs, the blood gives up the
carbon dioxide (cleansed) and
receives oxygen. This
oxygenated blood …
atrium
Proposition array
contract
P valve
P artery
lungs
cleansed
oxygenated
13 of 24
Step-by-step
1.
2.
3.
4.
5.
6.
Create the writing prompt
Students write essays, when done, they save it
in a specific format
Teacher creates key terms with synonyms and
saves as terms.txt
Teacher creates one or two expert referent
essays
Put all into the same folder and run ALA-Reader
Collect scores from the report file
14 of 24
Step 1. Create the writing prompt

For this demo, we used:
In a 300-word essay, describe the structure
and function of the human heart and
circulatory system.
15 of 24
Step 2. Student essay file format
The Dance of Life
Student 5
30 max
The heart is the major component of the circulatory system.
Its function is to circulate blood, which is comprised of both blood cells and plasma, throughout the body.
Two types of blood vessels are used to circulate blood.
Arteries are used to carry oxygen rich blood throughout the body, while veins are used to carry oxygen deficient
blood to the heart.
The heart is one of the most complex organs in the body.
One of its primary jobs is to oxygenate blood for the body.
Here is how it works.
As stated earlier, veins carry oxygen deficient blood to the heart.
The blood flows into the right atrium, then the right ventricle, and then the pulmonary artery pumps blood to
the lungs for oxygen.
Once the blood is oxygenated, pulmonary veins carry blood back to the heart, where it flows from the left
atrium to the left ventricle.
The aorta pumps oxygen rich blood throughout the body, then the veins carry oxygen deficient blood back to
the heart, where the cycle begins again.
Quite literally, the heart is “the heart” of the circulatory system.
It is amazing to think that such a seemingly simple organ is so complex.
16 of 24
Students write essays


Students use Notefolio app* to write their
essay on a TI calculator (TI-83 Plus family,
TI-84 Plus family, TI-89, TI-89 Titanium, TI92 Plus and Voyage™ 200).
Teacher uses NoteFolio™ Creator Software*
on the teachers laptop to organize and view
the students’ essays (TI connectivity kit or TINavigator wireless and TI Connect software*
to import and export)
(*free downloads)
17 of 24
Step 3. create terms.txt file
30 max
aorta aorta
aortic_valve aortic_v
body body
cleanse clean carbon
contract contrac
diastolic diastol
endocardium endocar
epicardium epicar
inferior_vena_cava inferior
left_atrium left_atri
left_ventricle left_vent
lungs lung
membranes membran
mitral_valve mitral bicusp atrioventricular
myocardium myocard
oxygen oxygen
pericardium pericard
pulmonary_artery pulmonary_art
pulmonary_valve pulmonary_val
pulmonary_veins pulmonary_vei
relax relax
right_atrium right_atri
right_ventricle right_vent
superior_vena_cava superior
systolic systol
tricuspid tricusp
• 30 terms maximum
• Space to separate synonyms
• _ to show spaces
mitral_valve mitral bicusp atrioventricular
full term
pattern match
18 of 24
Step 4. create expert1.txt file
Heart Essay
Expert 1
30 max
Blood flows from the extremities towards the inferior vena cava and superior vena cava.
Blood enters the right atrium from the inferior vena cava and superior vena cava.
Flow passes from the right atrium through the tricuspid valve to the right ventricle.
From the right ventricle, blood flows oneway through the pulmonary valve onto the pulmonary artery.
The blood flows through the pulmonary artery to the lungs where it is cleansed and oxygenated.
Blood flows from the lungs back to the heart through the pulmonary veins.
Blood then enters the left atrium from the pulmonary vein.
Flow passes from the left atrium through the mitral valve into the left ventricle.
Blood then flows out of the heart through the aortic valve into the aorta.
The blood in the aorta flows to all parts of the body.
contract and systolic
diastolic and relax
membranes
endocardium and myocardium
epicardium and pericardium
19 of 24
Step 5. Run ALA-Reader





Place student essays, terms file, expert
referent essay, and ALA-Reader software into
the same folder (click)
Double click ALA-Reader
Click the Mark L button
Type in the file name of a student’s essay
Use scores
20 of 24
Step 6. Use scores

See report.txt or L_report.txt
21 of 24
ALA-Reader future
Current beta version forms proposition array based
on sentence-level and also linear associations
Future:
 Add other ways to parse the text, e.g., pure linear,
reverse linear, double-chunk, etc.
 Add a graphing utility to the tool
 Download from my web site at

www.personal.psu.edu/rbc4
22 of 24
Questions




ALA-Reader can mark any essay (you provide
the expert comparison essay and the list of
important terms)
Any Questions
Show of hands…
One-on-one after the session, also email me
with any questions, etc.
23 of 24
Our papers on concept maps and essays



Koul, R., Clariana, R. B., & Salehi, R. (2005). Comparing several
human and computer-based methods for scoring concept maps
and essays. Journal of Educational Computing Research, 32 (3),
261-273. link
Taricani, E. M. & Clariana, R. B. (2006). A technique for
automatically scoring open-ended concept maps. Educational
Technology Research and Development, 54, 61-78.
Clariana, R.B., Koul, R., & Salehi, R. (2006). The criterion-related
validity of a computer-based approach for scoring concept
maps. International Journal of Instructional Media, 33 (3), in
press.

Poindexter, M. T., & Clariana, R.B. (2006). The influence of
relational and proposition-specific processing on structural
knowledge and traditional learning outcomes. International
Journal of Instructional Media, 33 (2), in press.
24 of 24