Transcript Slides - Lucentia

OBTAINING KEY PERFORMANCE
INDICATORS BY USING DATA MINING
TECHNIQUES
Roberto Tardío & Jesús Peral
Lucentia Research Group
Department of Software and Computing Systems
UNIVERSITY OF ALICANTE
OUTLINE
1. Introduction
2. Background
3. Methodology
4. Case study
5. Discussion
1. INTRODUCTION
1. Introduction
2. Background
3. Methodology
4. Case study
5. Discussion


Dashboards and Scorecards (Kaplan et al., 1996)
decision makers to quickly assess the status of an
organization.
Dashboards  the preferred tool across organizations to monitor
business performance.

Key Performance Indicators (KPIs) (Parmenter, 2015) play a crucial
role, since they facilitate quick and precise information by comparing
current performance against a target required to fulfill business
objectives.
1. INTRODUCTION
1. Introduction
2. Background
3. Methodology
4. Case study
5. Discussion

KPIs are not always well known

sometimes it is difficult to find an adequate KPI to associate
with each business objective (Angoss, 2011).
Organizations use existing lists of KPIs
 An organization performs an innovative activity




KPIs may be redundant (Rodríguez et al., 2009), misdirecting the
effort and resources of the organization.
people responsible for (wrong) KPIs develop a resistance to change
once they have found how to maximize their value (Parmenter, 2015) .
there is a tendency to focus on results themselves (Parmenter, 2015;
Angoss, 2011 ) (e.g. Sales) rather than on the actual indicators that
can be worked on (e.g. Successful deliveries/Total deliveries) and lead
to the results obtained.
1. INTRODUCTION
1. Introduction
2. Background
3. Methodology
4. Case study
5. Discussion

There is a need for techniques and methods that
improve the KPI elicitation process, providing
decision makers with information about relationships
between KPIs and their characteristics.
Big Data

the implications of the data for the company are unknown, and,
thus, eliciting their relationships with internal KPIs can make
these data actionable, adding value to them.
1. INTRODUCTION
1. Introduction
2. Background
3. Methodology
Big Data  huge volume, complex and
heterogeneous sources
4. Case study
5. Discussion
KPIs elicitation
Visualization.
What You See Is What You Get. Only when
the analytical results are friendly displayed, it
may be effectively utilized by users
1. INTRODUCTION
1. Introduction
2. Background
3. Methodology
4. Case study
5. Discussion

Our approach combines these two aspects:
to drive data mining techniques.
 obtaining specific KPIs for business objectives in a semiautomatic way.

The main benefit of our approach



organizations do not need to rely on existing KPI lists.
In order to show the applicability of our approach

we apply our proposal to the novel field of MOOC (Massive
Open Online Course) courses in order to identify additional KPIs
to the ones being currently used.
2. BACKGROUND
1. Introduction
2. Background
3. Methodology
4. Case study
5. Discussion

(Kaplan et al., 1996)  Balanced Scorecard, a tool
that consists on a balanced list of KPIs associated with
objectives covering different business perspectives.

(Kaplan et al., 2004)  Strategy Map, describes the way that
the organization intends to achieve its objectives, by capturing
the relationships between them in an informal way.

(Horkoff et al., 2014)  Business strategy models, combine
KPIs, objectives, and their relationships all together in a
single formal view.

Are the KPIs adequate??
2. BACKGROUND
1. Introduction
2. Background
3. Methodology
4. Case study
5. Discussion


(Parmenter, 2015)  the design and implementation of
KPIs within Dashboards. The author differentiates
between Key Result Indicators (KRIs) and KPIs
(Rodríguez et al., 2009)  the QRPMS method to select KPIs
and elicit relationships between them. The method starts
from a pre-existing set of candidate KPIs, and performs a
series of analysis steps.

using data mining techniques
2. BACKGROUND
1. Introduction
2. Background
3. Methodology
4. Case study
5. Discussion

Big Data

datasets that we can not manage with current
methodologies or data mining software tools principally
due to their huge size and complexity.
Big Data mining is the capability of extracting useful
information from these large datasets or streams of data.



New mining techniques are necessary due to the volume,
variability, and velocity, of such data.
The Big Data challenge is becoming one of the most exciting
opportunities for the years to come.
2. BACKGROUND
1. Introduction
2. Background
3. Methodology
4. Case study
5. Discussion


There are a number of works focused on monitoring
performance by means of KPIs

However, most of the works that tackle the problem of KPI
selection require a pre-existing set of KPIs.
Obtaining this set of KPIs
can be a tough task in already established organizations (Angoss,
2011),
 becomes a challenge when the business activity is developed in an
innovative environment.

3. METHODOLOGY
1. Introduction
2. Background
3. Methodology
4. Case study
5. Discussion

STAGES 1 & 2
 First of all, we start by focusing on modeling the
business strategy and known KPIs (if any) to guide the
process.
Business strategy model includes:
The relationships between the different business objectives to be
achieved
 (optionally) The processes that support them (the objectives).


The dependencies are modeled in a semiautomatic mode.
1. Introduction
2. Background
3. Methodology
4. Case study
5. Discussion
3. METHODOLOGY
1. Introduction
2. Background
3. Methodology
4. Case study
5. Discussion

STAGES 3 & 4
 Decision makers  provide the required
information to fulfill the objectives.
 By interviewing these decision makers we can create
new user requirement views  in order to
implement the DW.
The aim of this step is to relate business objectives with
entities and measures that are related to their performance.
a set of candidate KPIs for each objective is defined.
Analysis to merge the information
multidimensional model for analysis
3. METHODOLOGY
1. Introduction
2. Background
3. Methodology
4. Case study
5. Discussion
STAGES 5 & 6
 The multidimensional model allows the mapping from
the indicators to DW elements  DW schema is
generated automatically.

The following step is to analyze the candidate KPIs through
data mining techniques to ensure that they reflect the
relationships identified during business strategy modeling.

Finally, we define or update the analysis views for different
roles, materialized in dashboards that will allow decision
makers to access and monitor the new KPIs.
4. CASE STUDY
1. Introduction
2. Background
3. Methodology
4. Case study

The effect of the globalization along with the
proliferation of open online courses has radically
changed the traditional sectors of education.
5. Discussion

New technologies
symbolise a big opportunity
 it is also required to face significant challenges to take full
advantages of them.


Massive Open Online Course (MOOC)
an online course with the objective of interacting and promoting
participation and open access via the web.
 slides, video lectures (off-line and on-line), user forums…

gain popularity: number of students has increased exponentially
during the last years.
4. CASE STUDY
1. Introduction
2. Background
3. Methodology
4. Case study
5. Discussion


We present the process followed to elicit and model the
critical information from the MOOC named
UniMOOC (Platform Courses for Entrepreneurs of the
University of Alicante).
UniMOOC is a MOOC
currently has over unique 20,000 students registered and focuses on
entrepreneurship.
 the course includes several units and modules as well as links to
social networks for students to interchange opinions.

4. CASE STUDY
1. Introduction
2. Background
3. Methodology
4. Case study
5. Discussion
STAGES 1 & 2
4. CASE STUDY
1. Introduction
2. Background
3. Methodology
4. Case study
STAGES 3 & 4
 Interviews with the organizers of this course.
 A first set of indicators were obtained in a generic way:
5. Discussion


increment in number of students, dropout ratio, recovery ratio of
students, % Of active students, % Of students who fail the course,
etc.
An initial version of the multidimensional model for analysis.


two analysis cubes: Enrollment and Activity.
Enrollment, allows us to analyze if the characteristics of the
students, such as country, interests and expectations present certain
patterns.
1. Introduction
2. Background
3. Methodology
4. Case study
5. Discussion
4. CASE STUDY
1. Introduction
2. Background
3. Methodology
4. Case study
STAGES 5 & 6
 We have started by applying the classical data mining
techniques to the database of the course.
5. Discussion

Due to the big amount of data of this course  these
techniques are not very suitable because they are difficult to
interpret:
they produce a lot of rules in association rules and decision trees.
 they also produce many hidden neural connections in the artificial
neural networks.


The best way to analyse these data is by using visualization
methods.

the visualization techniques allow to see how the graphical grow
dynamically.
1. Introduction
2. Background
3. Methodology
4. Case study
5. Discussion
1. Introduction
2. Background
3. Methodology
4. Case study
5. Discussion
5. DISCUSSION
1. Introduction
2. Background
3. Methodology
4. Case study

Dashboards are the preferred tool across organizations to
monitor business performance.
5. Discussion

Different data visualization techniques

Key Performance Indicators (KPIs) play a crucial role in facilitating
quick and precise information by comparing current performance
against a target required to fulfill business objectives.
5. DISCUSSION
1. Introduction
2. Background
3. Methodology
4. Case study

Very often it is difficult to find an adequate KPI to
associate with each business objective.
5. Discussion
The main objective is to obtain specific KPIs for business
objectives in a semi-automatic way.
 This approach is illustrated with a case study, a MOOC course,
which is a very novel area and therefore very suitable for their
purpose.

5. FUTURE WORK
1. Introduction
2. Background
3. Methodology
4. Case study

Automatic extraction of KPIs from
Business strategy model.
 Student interviews and feedbacks.

5. Discussion

Data Mining techniques (supervised, unsupervised, hybrid) to
check the correlation.

Big Data environments
extract KPIs from data.
 visualization methods.

QUERIES?