Transcript What is a Process?

Neo4j for Process Mining
Big Data and Hadoop Users Group of West Michigan
Gordon Food Service 3/9/2016
Agenda
•
•
•
•
•
•
Introduction
What is a Process
What is a Process Model
What is Process Mining
Data Prospecting with Neo4j
Tool Demo
Introduction
In the domain of information systems it is very
difficult to see a product let alone determine how
efficiently it was developed. In most cases we are left
with textual information to understand and make
assessments of a process.
Introduction
The purpose of this proof of concept is to explore the
application of graph database techniques in
conjunction with process mining to improve the
assessment of software development processes.
*Better ways to capture, visualize, and analyze processes*
What is a Process?
What is a Process?
• A series of actions or steps taken in order to achieve a
particular end.
• A collection of linked tasks which find their end in the
delivery of a service or product to a client.
• A set of activities and tasks that, once completed, will
accomplish an organizational goal.
*There are as many definitions of process as there are people*
Definition for Presentation
• A process comprises an identified body of work that can be
characterized as a set of related activities or a defined
sequence of steps and decisions
• The process name is in action verb - noun form (usually)
• The name is singular
• The result must be discrete and countable
• The result is what the customer of the process wants
• The process is initiated by a triggering event that could be
an action, time, or a condition
Workflow Modeling: Tools for Process Improvement and Application Development, Sharp & McDermott
Process Examples
Suggested Process
What We Call It
Why Not a Process
Customer Relationship
Management
Process Area
Doesn’t deliver a single, specific result; a set of
related business processes meeting an overall
objective
Acquire New Customer
Business Process
Delivers a single, specific result, and meets all other
criteria. An “end-to-end business process
Assess Prospect Financial Status
or Setup Customer
Sub process
Too small - both deliver specific results, but are
intermediate results in an E2E process
Calculate Customer Credit Limit
or Create Customer Account
Activity, Step, Task
Much too small - a part of a sub process. Possibly
described in a procedure or use case.
Workflow Modeling: Tools for Process Improvement and Application Development, Sharp & McDermott
What is a Process Model?
BPMN Basic Elements
Activities
Collapsed
Sub-Process
Task
Gateways
Events
Exclusive
Start
End
Inclusive
Timer
Parallel
Start
Intermediate Throwing Receiving
Message
Throwing
Signal
Signal
Message
Data
Data Object
Data Store
White
Box
Black Box Pools
Team B
Throw 1
Team C
Pool with
Swimlanes
Process 1
Team A
White Box Pools
Black
Box
BPMN Pools
Receive 1
Process A
External
Entity
BPMN Pool Communication
Data Store
What is Process Mining?
Process Mining
Process Mining Discovery, Conformance, and Enhancement of Business Processes, Wil M. P. Van Der Aalst
Three types of Process Models
Event Log
Event Log
Model
Discovery
Model
Conformance
Checking
Diagnostics
Enhancement
New Model
Event Log
Model
Event Logs
• A process consists of
cases
• A case consists of events
such that each event
relates to one case
• Events within a case are
ordered
• Events can have attributes
Play-Out
Process Mining Discovery, Conformance, and Enhancement of Business Processes, Wil M. P. Van Der Aalst
Play-In
Process Mining Discovery, Conformance, and Enhancement of Business Processes, Wil M. P. Van Der Aalst
Play-In
L3 = [<a,c,d>45, <b,c,d>42, <a,c,e>38, <b,c,e>22]
a
d
c
b
e
a b
c
d e
-
#
>
#
#
b #
-
>
#
#
c
<
<
-
>
>
d #
#
<
-
#
e
#
<
#
-
a
#
XOR-split pattern:
c>d, c>e, and d#e
Process Mining Discovery, Conformance, and Enhancement of Business Processes, Wil M. P. Van Der Aalst
Replay
Process Mining Discovery, Conformance, and Enhancement of Business Processes, Wil M. P. Van Der Aalst
Replay
Process Mining Discovery, Conformance, and Enhancement of Business Processes, Wil M. P. Van Der Aalst
Data Prospecting with Neo4j
General Approach
•
•
•
•
•
Given that the artifacts from the SLDC are recorded and
linked in JIRA (i.e. issue tracker) they can be viewed as a
graph
Find a promising project in this case Wholesale Order
Capture (WOC) and load the issues with their associate
links into a labeled graph (i.e. Neo4J)
Use Cypher queries to assess the “Prospect” for developing
event logs
Remediate good prospects (i.e. if possible)
Extract the event log from the graph database and load it
into Prom
Traceability Network
Wiki
(Confluence)
CA Service Desk
Use Cases, Epics
High Level Reqs
Prototypes
Issue Tracker
(JIRA)
Release Notes
Requirement
Change
Management
Release
Deploy
Task
Test Case Spec.
Test Run
Incident
Management
Defect
Automated
Code Analysis
(SonarQube)
Standards
Quality Profile
Problem
Management
Issue
Repository
Management
(Fisheye)
Manual
Code Analysis
(Crucible)
Source Code
Source Code
Repository
Code
Review
Paired
Programming
Configuration
Management
Graph Model
is in Version
10062
{key}
{issue type}
{priority}
{create date}
{resolution name}
{resolution date}
Version
Epic
16
Fix
Version
is in Fix Version
10062
Is Component of
10062
Comp
ENH
8
Is related to
10030
Is Validated By
10040
Is Implemented By
10062
Task
3
Is Implemented By
10062
REQ
6
Is child of
10012
Is Validated By
10040
TCS
12
Is Recorded By
10070
Is Impacted By
10071
Has a sub-requirement of
10012
SubTask
7
Sub-REQ
20
TBD
Test
13
Bug
1
Commit
TBD
TBD
Code
Review
File
TBD
TBD
TBD
TBD
Scan
Defect
Comment
TBD
Issue
TBD
Rule
Metric
Challenges Compiling Event Logs
• Picking events and activities for the model
• Finding good the timestamps
• Assessing the relevancy and accuracy of the
timestamps
• Relating and ordering events and activities across
systems
Neo4j Demo
ProM Demo
Conclusions
•
•
•
•
While the WOC project was very successful from a project
context, it was not as successful in a process improvement
context
Controls for workflow and traceability between issues
needs to be improved
Neo4j is a very effective tool for prospecting and
processing event logs
Process mining of the SDLC is feasible but will take
significant effort on the data side (no surprise here !)