A personality mining system for automated applicant ranking in

Download Report

Transcript A personality mining system for automated applicant ranking in

Evanthia Faliagka, Maria Rigou, Spiros Sirmakessis
Qualities of good teachers
Teachers are distinguished as liked or disliked based on
three criteria: academic qualifications, relationship
with students and personality traits
Traits that yield positive educational results
Conscientiousness (efficient, not lazy, thorough)
 Agreeableness (warm, forgiving, sympathetic)
 Openness to experience (curious, imaginative, excitable)
 Extroverted (sociable, enthusiastic, forceful, positive)
 Emotionally stable (calm, self-confident, not shy)
Big-Five personality model (NEO PI-R)
Previous work
Academic qualifications of teachers come from:
Teacher personality is typically evaluated through
CVs and accompanying documents
special purpose questionnaires/tests
Recruiting teachers could be performed online provided
that we can have some unbiased feedback on teacher
Social web data (from Facebook, Twitter, LinkedIn, etc.) can be
the source of such feedback especially in the case of active users
if we can interpret social web activities in terms of personality
traits demonstrated.
Proposed system
A system that automates candidate teacher prescreening process providing an overall candidate
based on supervised learning
 and automatically extracting applicant personality
measures from tweets and Fb posts
This approach has been implemented as a web
based teacher evaluation system
CV data
Login to
Login to
Personality traits of the
candidate are assessed
by analyzing posts to
Twitter and Fb
Applicant education,
work experience and
loyalty are directly
extracted from LinkedIn
candidate’s skills
Data mining
Ranked list
of teachers
Academic qualifications
Education (in years of formal academic training)
Work experience (in years of working at relevant
job positions
Loyalty (average number of years spent per job)
Personality mining
Judging a teacher’s personality (or ANY personality) is a hard
problem for automated e-recruitment systems
We focus on the extroversion personality trait
It is reflected through language use in written speech
It is discriminated through text analysis
It is a crucial characteristic in teacher personality
The emotional positivity and social orientation of a person,
both directly extracted from LIWC frequencies, can act as
predictors of the extroversion trait
Linguistic Inquiry and Word Count system
Calculation of teacher extroversion
To find which words are mentioned most frequently by the
candidate we analyze the raw text of tweets and Facebook
The words identified are input to the TreeTagger tool for
lexical analysis and lemmatization
Then using the LIWC dictionary the system classifies the
canonical form of word output by the TreeTagger
A dictionary of word stems classified in certain psycholinguistic
We calculate the LIWC extroversion score E
E is estimated directly from LIWC scores, by summing the
emotional positivity score and the social orientation score
Calculation of teacher extroversion
Finally, we use the regression model which was trained
in a previous work of ours that predicts the candidates’
extroversion from their LIWC scores in the {posemo,
negemo, social} categories
E = S +1.335*P - 2.25*N
 E is the extroversion score
 S the frequency of social words
 P the frequency of positive emotion works
 N the frequency of negative emotion words
Login page
Experience mining
Our system uses machine learning algorithms
 It
requires a training set as an input
 It automatically builds the ranking model
 It calculates the final scoring function h(x)
It returns the final ranked list of teachers by
applying the learned function to sort them
Pilot Scenario
42 teachers logged in to our system as candidates for a job
position in a private elementary school
The job position was also announced through the system
Teachers were also evaluated manually on their academic
qualifications and interviewed for assessing their personality
by the school director
Automated extroversion scores were compared to the
interview results referring to each teacher’s extroversion
The data collected are to be used as the training set for the ranking
Pilot Scenario
Grading scale for the personality extroversion score: 0-5
We used Weka to test the correlation of the scores output from the
system (i.e. model predictions) with the actual scores assigned by the
Comparison metrics (system vs director):
Overlap size of the top-k list
Correlation coefficient of the top-k candidates
Mean absolute difference of top-k candidates’ ranks
Ranking error
6 (75%)
The proposed system could be of practical value in
speeding up the teacher recruitment process
Automating qualifications and personality assessment
Future work:
Use larger training sets
Instead of manual character assessment, use special
questionnaires and train the system with their results
Use additional social network metrics (LinkedIn endorsements
and recommendations, no of re-tweets, Fb likes and shares, etc)
Incorporate automated assessment of teacher scores in more
personality traits (agreeableness, openness to experience, etc)
Thank you!