Transcript Technology for automated assessment: The World-Wide-Mind

Technology for automated
assessment:
The World-Wide-Mind
Ciarán O’Leary
Dublin Institute of Technology
22nd May 2003
Quick Overview
• Requirements
– Large amounts of software online
– Large number of participants in project
– Diversity is key!
• Need to
– Provide architecture and protocol that will
make this software available
– Have a simple entry level – like the WWW
e-Learning Overview
• Reusability of components
• Potential for automated assessment
• Potential for comparisons with other
students
Agenda
• World-Wide-Mind
– Introduction to goal of project
– Definition of a service
– Available software
•
•
•
•
Blocks World example
Usage in learning environment
Comparison with existing models
Possibilities for future development
Part I
World-Wide-Mind
• Dublin City
University
– Dr. Mark Humphrys
• Artificial Intelligence
Project
– Distributed
– Artificial Minds
The Jolly Little Creature
Agent
The Jolly Little Creature’s Mind
• Multiple (massively diverse) modules
• Since, according to faculty psychologists, the
mental causation of behaviour typically involves
the simultaneous activity of a variety of distinct
psychological mechanisms, the best research
strategy would seem to be divide and conquer:
first study the intrinsic characteristics of each of
the presumed faculties, then study the ways in
which they interact
– Jerry Fodor – The Modularity of Mind, p1.
Building JLC’s Mind
• Divide and conquer
• Required diversity through multiple
authors
– The more authors the more diversity
• Make modules available online
– Researchers
– Amateurs
– Students!
Building JLC’s Mind
• Requirements
– Architecture
– Standard/protocol to use to integrate
components
• Requirement for protocol
– No barriers to entry
– Simple
– Not tied to any platform
Summary Part I
• World-Wide-Mind
• Distributed Artificial Intelligent Project
• Success dependent on participation of large
number of very different researchers
– Require diversity
• Need an architecture and protocol for
components to be plugged together
• Major goal is simplicity
Part II
JLC’s Mind
Mind
Body
World
Perceive state
State
State
Select action
Take action
New state
JLC’s Mind Online
Mind
Body
World
Perceive state
State
State
Select action
Take action
New state
Internet
World-Wide-Mind Architecture
Mind
Server
World
Server
Client
World-Wide-Mind Architecture
Mind
Server
World
Server
Mind
Server
World
Server
Mind
Server
Client
Mind
Server
Mind
Server
Mind
Server
Mind
Server
Service Methods
World Service
• newrun
• getstate
• takeaction
• endrun
Mind Service
• newrun
• getaction
• endrun
World-Wide-Mind Protocol
• “The term Web services describes a standardized
way of integrating Web-based applications using
the XML, SOAP, WSDL and UDDI open
standards over an Internet protocol backbone.
XML is used to tag the data, SOAP is used to
transfer the data, WSDL is used for describing the
services available and UDDI is used for listing
what services are available”
– http://www.webopaedia.com
• Society of Mind Markup Language (SOML)
– Lightweight Web Services
SOML Message
1. <soml version="0.9">
2.
<request type="getaction" runid="123456">
3.
<param name="state">
4.
(0, 0, 0, 9, 7, 8, 2, 0, 4)
5.
</param>
6.
<argument name="usemem" value=“true”/>
7.
</request>
8. </soml>
Web Based Client – Java Applet
Web Based Client – Java Applet
Constructing complex minds,
online
Mind
Server
MindM
Server
World
Server
Client
Mind
Server
Who builds the sub-minds?
• Everyone
–
–
–
–
Professional researchers
Students
Amateurs
Anyone interested in AI!
Who builds the parent minds
• Everyone
–
–
–
–
Anyone can put software online
Anyone can put a world online
Anyone can use my mind as part of their mind
Best minds are the ones that will be re-used
How does this make the AI
problem easier?
• Society A solves problem A
• Society B solves problem B
• Problem A + B is solved by writing parent
mind to arbitrate between A and B’s parent
minds
Society of Mind
World A+B
Client
MindM A+B
World A
World B
MindM A
Mind
Mind
MindM B
Mind
Mind
Mind
Mind
Summary Part II
•
•
•
•
•
•
Distribute the mind on the Internet
Use server-centric architecture
Use existing HTTP protocol
Use simple XML-like messages
Simple entry level
Potential for provision of large numbers of
components required for project
Part III
World-Wide-Mind in learning
environment
• 10 Students presented with an online
problem
• Have to interact with the online problem
using client software
• Need to create “mind” or solution
First Implementation
• Blocks World
Goal
Start
A
D
F
B
C
E
F
E
D
C
B
A
Blocks World Service
• State
empty(1)&ontable(A,2)&
clear(A)&ontable(B,3)&
on(F,B)&on(D,F)&
clear(D)&ontable(E,4)&
on(C,E)&clear(C)&
empty(5)&gripper()
• Action
grip(A)
ungrip(3)
Blocks World Run
Mind
Server
Blocks
World
Server
getaction
newrun
endrun
getstate
takeaction
newrun
endrun
Client
Blocks World Service
• Supports
– newrun
• Provides runid
– getstate
• Provides state in form shown
– takeaction
• Receives action sent by client
• Returns updated state following execution of action
– endrun
• Ends run, clears up server
• All messages carry error code information
Building a blocks world “mind”
• Create service that supports
– newrun
• Provides runid
• Can also provide startup-parameters
– getaction
• Receives state from client (which had got it from world
service)
• Provides state in form shown
– endrun
• Ends run, clears up server
• All messages should carry error code information
Scoreboard
• Scores of best performing minds recorded in a list
on the “world” site
– Score is the number of moves needed to stack blocks
• Any mind that interacts with world will have its
score recorded and displayed
• Others can then run that mind in the world
• Performance based assessment
– Performance of algorithm can be judged from score
board
Technical Requirements
• Web Server space
• One of…
–
–
–
–
–
Java Servlets
ASP
JSP
PHP
CGI – C++/Perl/any language
• To write own client (or to integrate another mind),
need to know how to connect to URL
– Simple code in most languages
Connect to URL (Java)
String getResponseFromService(String serviceURL, String request) {
URLConnection connection = new URL(serviceURL).openConnection();
PrintWriter outs = new PrintWriter(connection.getOutputStream());
outs.println(request); // SOML message
outs.close();
StringBuffer returnedText = new StringBuffer();
BufferedReader ins = new BufferedReader(new
InputStreamReader(connection.getInputStream()));
String line = "";
while((line = ins.readLine()) != null) {
returnedText.append(line);
}
return returnedText.toString();
}
Connect to URL (Perl)
sub getResponseFromService() {
$serviceURL = @_[0];
$request = @_[0];
new LWP::UserAgent->request(
POST “$serviceURL”,
Content_Type => 'form-data',
Content => [ q(q) => "$request"])->content;
}
Summary Part III
• Simple blocks world problem online using
protocol/architecture developed for World-Wide-Mind
• Software still available online for others to interact with
• Client software also available
• Students put minds online
– Minimal knowledge of network programming required
– Only really need to read from stdin and write to stdout.
• Scoreboard records best performing algorithms/minds
Part IV
Comments on Architecture
• Lightweight web services
– Low entry level
• Protocol simple
• Technical requirements minimal
– Software continually available
– Problem can be re-used remotely by others
• At the very least it can be tested remotely
Comments on Architecture
• Comparison with World-Wide-Web
– Students put project work (HTML pages/PDF)
online (DCU/DIT)
– Others can review their project work
– Very simple entry level
• Before there were tools (MS Front Page) there was
simple (terrible) HTML
Reusability
• Why don’t we use other peoples software?
– Platform issues
– Programming language isues
• Its in our favour for others to make software
available
• What is their incentive?
– If you’re putting it online anyway…
• Learning objects
– SCORM
• PEARL
• TOIA (IMS QTI compatible tools)
Summary Part IV
•
•
•
•
We need to re-use other peoples software
Local installation has many issues
Complex standards require tools
Low entry level
Part V
Future Development
• Existing standard
– Portal site will provide
•
•
•
•
Sample code
Instructions on creating services
Services
Client software
Future Development
• Possible
– Integration at higher levels with other standards
•
•
•
•
SOAP
XML
WSDL
DAML
– e-Learning standards
• PEARL uses HTTP
Questions?