Transcript Automatic Web Application Failure Detection from User Behavior

Automated Failure Detection in
Internet Services via Changes in
User Behavior
Lukas Biewald
Gregory Friedman
Hypothesis: We can detect web
app bugs through user behavior
Session 1
Session 2
Session n
index.html
index.html
index.html
filter.html
filter.html
news.html
news.html
news.html
OK
filter.html
OK
…
filter.html
BUG?
Classification/Anomaly Detection
One Class Support Vector Machine (Burges 1998)
Features: Strings of session data, request data. (Try a string kernel…)
Index,News,Filter,Index…
Advantages: SVM can handle the
large feature space, easy to
implement and known to work well for
a large range of problems.
Index,Filter,Filter,News…
…
Hidden Markov Model (HMM)
Bug
Bug
Bug
Bug
Data
Data
Data
Data
Here Data could be webpage bigrams
and trigram frequencies over sessions.
Advantages: HMM can easily handle
partially observed training data.
Features aren’t as rich. (Could be
good or bad)
Difficulties
Can we find web logs with sessions and matching
bug reports?
Talking to Amazon, Ofoto, Ebates…
How reliable is the bug report data?
Can use unsupervised/partially supervised
training…
Do the bugs really change user behavior in
interesting ways?
Unknown, but it seems like some of them
should…