Transcript Stimulus-Response Agents

Artificial Intelligence
Project 1:
Classification Using Neural Networks
2009. 03. 23
Kim, Kwonill
[email protected]
Biointelligence laboratory
Contents




Project outline
Description on the data set
Description on tools for ANN
Guide to Writing Reports
 Style
 Mandatory contents
 Optional contents


Submission guide / Marking scheme
Demo
(C) 2008, SNU Biointelligence Laboratory
2
Outline

Goal
 Understand MLP & machine learning deeper
 Practice researching and technical writing

Handwritten digits problem (classification)
 To predict the class labels (digits) of handwritten digit data set
 Using Multi Layer Perceptron (MLP)
 Estimating several statistics on the dataset

Data set
 Variation of the ‘Handwritten digit data set’
 http://archive.ics.uci.edu/ml/datasets/Pen-
Based+Recognition+of+Handwritten+Digits
(C) 2008, SNU Biointelligence Laboratory
3
Handwritten Digit Data Set (1/2)

Original Data Set Description
 Digit database of 11,000 samples from every 44 writers
 http://archive.ics.uci.edu/ml/datasets/PenBased+Recognition+of+Handwritten+Digits
 16 attributes
 (xt, yt),
t = 1, … , 8
 0 ~ 100
 Label (Class)
 0,
1, 2, … , 9
(C) 2008, SNU Biointelligence Laboratory
4
Handwritten Digit Data Set (2/2)

Constitution
 Preprocessed data (*.arff, *.csv)
 Total
data
= training data
+ test data
 Data description
 For WEKA
(pendigits_total_set,
(pendigits_training,
(pendigits_test,
(pendigits.names)
(*.arff)
(C) 2008, SNU Biointelligence Laboratory
1099)
749)
350)
5
Tools for Experiments with ANN

Source codes - Choose anything!!
 Free software  Weka (recommended)
 MATLAB tool box (Toolboxes  Neural Network)
 ANN libraries (C, C++, JAVA, …)

Web sites
 http://www.cs.waikato.ac.nz/~ml/weka/
 http://www.faqs.org/faqs/ai-faq/neural-nets/part5/
(C) 2008, SNU Biointelligence Laboratory
6
Reports Style


English only!!
Scientific journal-style
 How to Write A Paper in Scientific Journal Style and Format
 http://abacus.bates.edu/~ganderso/biology/resources/writing/HTWsections.html
Experimental process
Section of Paper
What did I do in a nutshell?
Abstract
What is the problem?
Introduction
How did I solve the problem?
Materials and Methods
What did I find out?
Results
What does it mean?
Discussion
Who helped me out?
Acknowledgments (optional)
Whose work did I refer to?
Literature Cited
Extra Information
Appendices (optional)
(C) 2008, SNU Biointelligence Laboratory
7
Report Contents – Mandatory (1/2)

System description
 Used software and running environments

Result graphs and tables

Analysis & discussion (Very Important!!)
(C) 2008, SNU Biointelligence Laboratory
8
Report Contents – Mandatory (2/2)

Basic experiments
 Changing # of epochs (Draw learning curve)
 Various # of Hidden Units
# Hidden
Train
Units Average
Best Worst
 Std. Dev.
Setting 1
Test
Average
 Std. Dev.
Best Worst
accuracy
Setting 2
Setting 3







(C) 2008, SNU Biointelligence Laboratory
9
Report Contents – Optional

Various experimental settings
 Normalization
 Learning rates
 Structure of MLP
 Feature selection
 Activation functions
 Learning algorithm
…

Evaluation techniques
 ROC curve
 k-fold Crossvalidation
…
(C) 2008, SNU Biointelligence Laboratory
10
Submission Guide

Due date: Apr. 15th (Wed.) 15:00
 Submit both ‘hardcopy’ and ‘email’
 Hardcopy submission to the office (301-419 )
 E-mail submission to [email protected]

Subject : [AI Project1 Report] Student number, Name
 Length: report should be summarized within 12 pages.
 If you build a program by yourself, submit the source code with
comments

We are NOT interested in the accuracy and your programming skill,
but your creativity and research ability.
 If your major is not a C.S, team project with a C.S major student is
possible (Use the class board to find your partner and notice the
information of your team to the 1st project TA(jakramate
@bi.snu.ac.kr) by Mar. 27th)
(C) 2008, SNU Biointelligence Laboratory
11
Marking Scheme

40 points for experiment & analysis
 Extra 4 points for additional expriments



20 points for report
6 points for overall organization
Late work
 - 10% per one day
 Maximum 7 days
* The Maximum Score is Changed
(C) 2008, SNU Biointelligence Laboratory
12
References

Materials about Weka
 Weka GUI guide (PPT)
 Explorer guide (PDF)
 Experimenter guide (PDF)
(C) 2008, SNU Biointelligence Laboratory
13
WEKA Demo
(C) 2008, SNU Biointelligence Laboratory
14
Matlab
(C) 2008, SNU Biointelligence Laboratory
15
QnA

MLP is the simplest form of contemporary neural networks.
(you can see other forms in the ‘ANN’ section of Wikipedia:
http://en.wikipedia.org/wiki/Artificial_neural_network)

Neural network is sometimes called as ANN (artificial neural
network) to stress the difference with the original neural
network in the brain or central nervous system.

Learning in neural networks consists in the optimization of
weights by gradient descent process. To get the global
optimum, we need to try not just several configurations of
parameters, but also various random starting points.
 When you use weka, you need to try several ‘randomSeed’ for this
reason
(C) 2008, SNU Biointelligence Laboratory
16