Transcript Syllabus CS434-534 fall 2015

Neural Network Design and Application
Fall 2015
CPTS 434 & 534
Instructor: John Miller
Office: West 134E WSU Tri-Cities
[email protected]
Class web page can be found at
http://users.tricity.wsu.edu/~jhmiller
Required Text: Learning from Data by Abu-Mostafa, Magdom-Ismail and Lin
Suggested texts:
Building Neural Networks by David M. Skapura,
Introduction to Machine Learning, 2nd ed by Ethem Alpaydin
Neural Networks and Machine Leaning, 3rd ed by Simon Haykin
Nuts and Bolts
Grades:
Tests and Assignments have equal weight
Tests: quizzes and final exam
given in class with open books, lecture notes, and computers
Assignments:
Prior approval is required for late submission. Full credit on
resubmissions until tested on subject matter. 50% credit thereafter.
Graduate Project Reports:
Topic approved by instructor
3 – 5 pages double spaced
Due last class period before dead week
More nuts and bolts
IMPORTANT: Per new WSU policy effective August 24, I will
ONLY be able to respond to emails sent from your WSU email
address. I will NOT be able to respond to emails sent from
your personal email address as of the first day of fall
semester. Effective the 24th, the IT Department will switch the
“preferred” email address in your myWSU to your WSU email
address.
More nuts and bolts
Accommodations for Disabled Students:
Reasonable accommodations are available for students who have a
documented disability. If you have a documented disability, even
temporary, make an appointment as soon as possible with the Disability
Services Coordinator,
Cherish Tijerina, 372-7352, [email protected]
You will need to provide your instructor with the appropriate classroom
accommodation form. The forms should be completed and submitted
during the first week of class. Late notification may delay your
accommodations. All accommodations for disabilities must be approved
through Disability Services. Classroom accommodation forms are
available through the Disability Services Office.
More nuts and bolts
Academic Integrity:
As stated in the WSU Tri-Cities Student Handbook," any member of the
University community who witnesses an apparent act of academic
dishonesty shall report the act either to the instructor responsible for the
course or activity or to the Office of Student Affairs."
The Handbook defines academic dishonesty to include "cheating,
falsification, fabrication, multiple submission [e.g., submitting the same or
slightly revised paper or oral report to different courses as a new piece of
work], plagiarism, abuse of academic material, complicity, or misconduct in
research."
Infractions will be addressed according to procedures specified in the
Handbook.
More nuts and bolts
Safety:
Should there be a need to evacuate the building (e.g., fire alarm or some
other critical event), students should meet the instructor at the Cougar
statue directly outside of the West building. A more comprehensive
explanation of the campus safety plan is available at
http://www.tricity.wsu.edu/safetyplan/
The university emergency management plan is available at
http://oem.wsu.edu/emergencies/
Further, an alert system is available. You can sign up for emergency
alerts (see http://alert.wsu.edu) through the zzusis site
(http://portal.wsu.edu/).
More nuts and bolts
Student Concerns.
If you have any student concerns, you can contact Carol Wilkerson
the Director of Student Affairs in West 269F, (509) 372-7139,
or [email protected].
If you have any concerns about this class, you should contact your
instructor first, if possible.
Attendance Policy.
Absences should be avoided. Students should contact an instructor if an
absence from class is unavoidable.
Students are encouraged to read Section 73 (Absences) of the
Washington State University Academic Regulations, which is found in the
WSU Tri-Cities Student Handbook.
Rise and fall of supervised machine learning techniques,
Jensen and Bateman, Bioinformatics 2011
Predominance of ANN has diminished
Why was ANN so popular?
Can be used as a black box
Makes nonlinear modeling easy
Magic due to biological basis
Applications of ANN by subject
From “Neural Network Design” by Hagan, Demuth and Beale
Applications of ANN by subject
Applications of ANN by subject
Objectives of the class:
1. To learn the general principals of data mining
2. Lean to apply artificial neural networks to classification
and regression problems
3. To compare artificial neural networks to other supervised
machine-learning techniques
Topics:
Basic data mining
fundamentals of machine learning
linear models
high-order polynomial models
overfitting and regularization
dimensionality reduction
clustering
ANN
perceptron
multi-layer perceptron
feed-forward ANN
radial basis function ANN
Other techniques
self organizing maps
support vector machines
Example of a Report-Type homework assignment
Dataset: Golub et al, Molecular Classification of Cancer: Class Discovery
and Class Prediction by Gene Expression Monitoring, Science, 286 (1999)
531-537
Download and become familiar with Weka software.
Open the leukemia gene expression data in Weka.
KNN technique is under the “lazy” menu of classifiers.
Weka refers to KNN as “IBk” for “Instance-Based k”.
After opening IBk, click on the text next to IBk to get a parameter menu.
Set “KNN” to 5 and keep the default value of other parameters.
Under “Test options” choose “Cross-validation” with “Folds” equal to 5.
Include the following in your report:
•Objective and conclusions of the paper
•Nature and Structure of the input data
•Results (include the performance metrics)
•Do your calculations support the authors’ conclusions
Example of a Programming-Type homework assignment
Generate 100 in silico data sets of 2sin(1.5x)+N(0,1)
each with 50 random x-values between 0 and 5
Use 50 data sets for training and 50 data sets for validation
Use the training data sets for polynomial regression of orders 1 – 5
For each order calculate the following:
RMS error for training data sets
RMS error for validation data sets
Bias squared
Variance
Plot your result as error vs order
Interpret your findings in terms of the “bias – variance dilemma”
Example of a Math-Type homework assignment
Derive the result
for Bayesian discriminant points in the 2-class problem with Gaussian class
likelihoods. Assume the mean and variance of C1 are 3 and 1, respectively.
Assume the mean and variance of C2 are 2 and 0.3, respectively.
For a sample size of 100, compare Bayesian discriminant points calculated
from maximum likelihood estimators with those derived from the true means
and variances.
Rise and fall of supervised machine learning techniques,
Jensen and Bateman, Bioinformatics 2011
Availability of sophisticated machine-learning
software packages, like WEKA, facilitates the
application of multiple methods to the same
problem
Tentative Schedule
Tu Aug 21
Th Aug 23
Tu Aug 28
Th Aug 30
Tu Sep 4
Th Sep 6
Tu Sep 11
Th Sep 13
Tu Sep 18
Th Sep 20
Tu Sep 25
Th Sep 27
Tu Oct 2
Th Oct 4
Tu Oct 9
Th Oct 11
Tu Oct 16
Th Oct 18
Tu Oct 23
Th Oct 25
Tu Oct 30
Th Nov 1
Tu Nov 6
Th Nov 8
Tu Nov 13
Th Nov 15
Tu Nov 20
Th Nov 22
Tu Nov 27
Th Nov 29
Dec 3-7
Dec 10-14
Discussion of class syllabus
Introduction to supervised machine learning
Introduction to supervised machine learning
Introduction to Bayesian statistics
Introduction to Bayesian statistics
Parametric methods
Parametric methods
Multivariate Data
Multivariate Data
Test #1
Artificial Neural Networks
Artificial Neural Networks
Artificial Neural Networks
Artificial Neural Networks
Artificial Neural Networks
Genetic Algorithm
Genetic Algorithm
Radial Basis Functions
Radial Basis Functions
Self-Organizing Maps
Self-Organizing Maps
Test #2.
Advanced network designs
Advanced network designs
Advanced network designs
Thanksgiving break
Thanksgiving break
Support Vector Machines
Support Vector Machines
Support Vector Machines
Review
Finals week Test #3