Transcript A personality mining system for automated applicant ranking in

IDENTIFYING GREAT TEACHERS
THROUGH THEIR ONLINE PRESENCE
Evanthia Faliagka, Maria Rigou, Spiros Sirmakessis
Qualities of good teachers


Teachers are distinguished as liked or disliked based on
three criteria: academic qualifications, relationship
with students and personality traits
Traits that yield positive educational results
Conscientiousness (efficient, not lazy, thorough)
 Agreeableness (warm, forgiving, sympathetic)
 Openness to experience (curious, imaginative, excitable)
 Extroverted (sociable, enthusiastic, forceful, positive)
 Emotionally stable (calm, self-confident, not shy)


Big-Five personality model (NEO PI-R)
Previous work

Academic qualifications of teachers come from:



Teacher personality is typically evaluated through



CVs and accompanying documents
LinkedIn
special purpose questionnaires/tests
Interviews
Recruiting teachers could be performed online provided
that we can have some unbiased feedback on teacher
personality


Social web data (from Facebook, Twitter, LinkedIn, etc.) can be
the source of such feedback especially in the case of active users
if we can interpret social web activities in terms of personality
traits demonstrated.
Proposed system

A system that automates candidate teacher prescreening process providing an overall candidate
ranking
based on supervised learning
 and automatically extracting applicant personality
measures from tweets and Fb posts


This approach has been implemented as a web
based teacher evaluation system
Position
requirements
Architecture
School
director
CV data
Login to
Candidate
teachers
Login to
Personality traits of the
candidate are assessed
by analyzing posts to
Twitter and Fb
Applicant education,
work experience and
loyalty are directly
extracted from LinkedIn
Extracting
candidate’s skills
Data mining
algorithms
Calculating
personality
traits
Ranked list
of teachers
Academic qualifications



Education (in years of formal academic training)
Work experience (in years of working at relevant
job positions
Loyalty (average number of years spent per job)
Personality mining


Judging a teacher’s personality (or ANY personality) is a hard
problem for automated e-recruitment systems
We focus on the extroversion personality trait




It is reflected through language use in written speech
It is discriminated through text analysis
It is a crucial characteristic in teacher personality
The emotional positivity and social orientation of a person,
both directly extracted from LIWC frequencies, can act as
predictors of the extroversion trait
Linguistic Inquiry and Word Count system
Calculation of teacher extroversion



To find which words are mentioned most frequently by the
candidate we analyze the raw text of tweets and Facebook
posts
The words identified are input to the TreeTagger tool for
lexical analysis and lemmatization
Then using the LIWC dictionary the system classifies the
canonical form of word output by the TreeTagger


A dictionary of word stems classified in certain psycholinguistic
categories
We calculate the LIWC extroversion score E

E is estimated directly from LIWC scores, by summing the
emotional positivity score and the social orientation score
Calculation of teacher extroversion

Finally, we use the regression model which was trained
in a previous work of ours that predicts the candidates’
extroversion from their LIWC scores in the {posemo,
negemo, social} categories
E = S +1.335*P - 2.25*N
Where:
 E is the extroversion score
 S the frequency of social words
 P the frequency of positive emotion works
 N the frequency of negative emotion words
Login page
Experience mining
Ranking

Our system uses machine learning algorithms
 It
requires a training set as an input
 It automatically builds the ranking model
 It calculates the final scoring function h(x)

It returns the final ranked list of teachers by
applying the learned function to sort them
Learning-to-rank
Pilot Scenario

42 teachers logged in to our system as candidates for a job
position in a private elementary school



The job position was also announced through the system
Teachers were also evaluated manually on their academic
qualifications and interviewed for assessing their personality
by the school director
Automated extroversion scores were compared to the
interview results referring to each teacher’s extroversion

The data collected are to be used as the training set for the ranking
algorithm
Pilot Scenario

Grading scale for the personality extroversion score: 0-5

We used Weka to test the correlation of the scores output from the
system (i.e. model predictions) with the actual scores assigned by the
director

Comparison metrics (system vs director):



Overlap size of the top-k list
Correlation coefficient of the top-k candidates
Mean absolute difference of top-k candidates’ ranks
k=8
Top-k
Correlation
Ranking error
Candidates
6 (75%)
0.72
2,6
Conclusions

The proposed system could be of practical value in
speeding up the teacher recruitment process


Automating qualifications and personality assessment
Future work:




Use larger training sets
Instead of manual character assessment, use special
questionnaires and train the system with their results
Use additional social network metrics (LinkedIn endorsements
and recommendations, no of re-tweets, Fb likes and shares, etc)
Incorporate automated assessment of teacher scores in more
personality traits (agreeableness, openness to experience, etc)
Thank you!