Transcript Day1_parallel2_2.218_Helen_Higsonx

Student Engagement and Learning: The Pedagogical
and Practical Value of Formal Work Placements
Andrews, J*, Green, J.P**, Higson, H.E*, and Jones, C.M*
* Aston University
**University of Ulster
Background
► In the White paper (2011) Higher Education: Students at the Heart of the System
the Government affirms that students should expect to receive excellent teaching.
► One of the dimensions identified by Gibbs (2010) of a high quality learning
experience is the levels of student effort and engagement.
► The White Paper (2011) also highlights the decline in students undertaking a
“sandwich” degree, despite the views of educationalists and employers on the
benefits of a placement year (Confederation of British Industry, 2009), which is in
part attributed to students feeling that “the extra year of study was not producing
enough added benefit.” (White Paper, page 41).
► The Wilson Review (2012) A Review of Business – University Collaboration,
recommends that: “Ideally, every full‐time undergraduate student should have
the opportunity to experience a structured, university‐approved
undergraduate internship during their period of study.” (Wilson Review, page
40).
Specifically in Northern Ireland
“The Department expects the institutions to ensure that all
learners have the opportunity to undertake a period of
work placement whilst undertaking a higher education
course. This includes work experience, voluntary and
community work and work outside term time, all of which
may be either directly or indirectly linked to their course.”
(Graduating to Success, 2012)
So what are the benefits to students?
Placements and degree performance:
Do placements lead to better marks?
Do more “engaged” students complete a placement year?
Do “weaker” students benefit more from placement?
Evidence
Evidence
Is the completion of
placement associated
with better final year
performance?
Are “better students”
completing a placement
year?
Do “better students”
gain more from a
placement year in terms
of improved final year
performance?
Duignan
(2002)
Gomez et
al (2004)
Mandilaras
(2004)
Surridge
(2008)
Mansfield
(2011)
Green
(2011)
Foster et
al (2011)
Duignan
(2003)
Gomez et
al (2004)
Rawlings
et al (2005)
Surridge
(2008)
HEFCE
(2009)
Mansfield
(2011)
Green
(2011)
Duignan
(2003)
Gomez et
al (2004)
Mendez
(2008)
Surridge
(2008)
Mendez and
Rona (2010)
Driffield, Foster
and Higson
(2011)
Driffield, Foster
and Higson
(2011)
Driffield, Foster and Higson
(2011)
What is a better student?
► From the literature a “better student” is one who performs better
in either pre-university or prior university assessment.
► Performance is a function of ability, effort and reward.
► Is a “better student” not an “engaged student”?
► Newman (1992) argues that student engagement occurs when
“...students make a psychological investment in learning. They
try hard to learn what school offers. They take pride not simply
in earning the formal indications of success (grades), but in
understanding the material and incorporating or internalizing it
in their lives”.
What is a “better student”?
► The use of pre-university and prior university performance as
measures of engagement is problematic if it is accepted that the
extrinsic rewards are different.
► If extrinsic reward is perceived as degree classification (which
evidence suggests is linked to graduate level employment,
Moores and Reddy, 2011), effort on pre-university education a
priori is likely to be high to gain university entry. In first year,
which for most UK universities (including Aston and Ulster) does
not contribute towards degree classification, a priori it is likely
that the extrinsic reward is lower, but for engaged students
making a “psychological investment in learning” there would
be no change in effort.
The Key Question?
► With regard to the placement year, if only engaged students
complete this then it could be argued that the finding of previous
studies that students who complete placements perform better
in final year is really that students who are engaged
throughout their course of study perform better in final year. In
other words the completion of placement, “the placement effect”
is a surrogate for student engagement, “the engagement effect”.
► In final year which contributes the most to degree classification
for the vast majority of UK degrees, it may be argued that the
effort of all students and the level of engagement will be at a
maximum.
Statistical issues
► In two random samples that are drawn from identical
populations, where the only differentiating factor is whether a
student has completed a placement or not, then providing the
sample sizes are large enough for the central limit theorem to
apply, ANOVA, standard OLS regression, or indeed a simple
comparison of means will provide a test of whether the effect of
a placement is significant.
► If the two samples are drawn from different populations,
however (more engaged and less engaged students through out
the entire course of study), which is the issue under
investigation in this study, then neither ANOVA or OLS
regression will allow for sample selection.
Statistical issues
► This issue is then a classical example of a sample selection
problem, as considered by Heckman (1979) in the context of
investigating the determinants of wage offers (in this study final
year degree performance) but only having access to data with
regard to those employed (students in final year, irrespective if a
placement year has been completed, who are engaged to
achieve a high degree classification).
► Heckman’s (1979) two stage procedure involves estimating a
probit model to account for the probability of an individual
undertaking a placement. The second stage estimates the
performance model including the selection parameter (Mills
ratio).
Methodology
► To avoid bias
 employed sample selection model developed by Heckman (1979).
 aimed to find a characteristic correlated with likelihood of students
choosing to take placements, but uncorrelated with their ability
 used the background of the student, measured by the National
Statistics Socio-economic Classification (NS-SEC) of the parents
► Approach
 employed a general to specific modelling strategy; started with
variables that conceptually or theoretically may be significant.
 the model was then tested systematically and insignificant variables
were removed
 tested for subsequent loss of predictive power, until arrival at the
final specifications.
Sample
► For Aston approximately 5,700 student observations are
included in the final sample, over the graduation years 20012010, covering courses across the university’s five Schools: (1)
Aston Business School (ABS); (2) Inter-disciplinary Studies
(IDS); (3) Engineering & Applied Sciences (E&A); (4) Languages
and Social Sciences (L&S); and (5) Life & Health Sciences
(LHS).
► For Ulster 737 student observations are included in the final
sample covering graduation in 2009 and 2010 most of which
derive from the Ulster Business School.
Descriptive statistics Aston (Table 1)
► A total of 39% of students from Aston included in the analysis engaged
in a work placement year.
► Out of the base population of students included in the study, 10% of
them from Aston are classified as mature students (above the age of 21
when their course begins), 50% of them are female, 79% of them are
British and 94% of them have no disability.
► In terms of parental class, 7% of students come from the Higher
Managerial Class, 11% from the Intermediate class and nearly 30%
from the Lower Managerial Class.
► Finally, in terms of school background, 32% of them came from a
comprehensive school, 6% from Grammar School and 10% came from
an Independent School.
Descriptive statistics
 A total of 66 per cent of students went on work placement.
 There are almost no mature students at Ulster (the University of
Ulster offers many of its degrees on a part-time basis, which
tends to attract mature students) for this reason to investigate the
impact of “relative maturity” a variable that identifies a student’s
age at the start of their course is investigated. The average age is
19.5 years. There are more females - 68% - and almost all of the
students are from Northern Ireland.
 The pattern of class distribution reflects the Aston data with a
significant number of students coming from the Intermediate and
Lower Managerial classes.
 In terms of schooling, 36% of students come from Grammar
schools, reflecting Northern Ireland’s option to retain such schools
in the 1970’s and 1980’s, and 54% of students studied at
comprehensive school.
How well can the probability of a student
taking a placement be predicted? (Table 2)
► For Aston, mature students are 7% less likely to take a placement
compared with students under the age of 21.
► For continuous variables, the marginal effect shows the increase
(decrease) in probability of a student, of a given type, undertaking a
placement resulting from a one unit increase in the variable concerned
whilst holding all other variables constant at their means. For example
at Ulster, as a student becomes older by 1 year they are 15% more
likely to go on placement. Interestingly these results somewhat
contradict each other but it is important to note that the variable Mature
in the Aston model is very different to the Age variable in the Ulster
model.
► Typically there were almost no ‘mature’ students included in the Ulster
data. Thus the positive coefficient suggests that older students, who are
not over the age of 21, and are therefore not defined as mature, are
more likely to undertake a placement.
►
How well can the probability of a student
taking a placement be predicted? (Table 2)
► Gender plays no role in determining the choice as to whether to go on
placement – the coefficients are statistically insignificant. In terms of
parental class, students with parents that work in the Higher Managerial
class appear to be more likely to undertake a placement at both Aston
and Ulster.
► The level of engagement is measured by a student’s average score in
their 1st year which does not contribute to final classification at Aston or
Ulster. As can be seen, for both Aston and Ulster the coefficient
estimate for the 1st year average is highly significant. This suggests that
“more engaged” students do choose to go on placement.
Do “engaged students” choose to go on
placement? (Table 3)
► The conclusions that can be drawn are relatively robust across both Aston and
Ulster. The selection term (Mills Ratio) is insignificant suggesting that there is
no self-selection present. The coefficient on the placement dummy is positive
and highly statistically significant at 0.053 for Aston and 0.060 for Ulster. This
suggests that undertaking a placement improves the performance in finals by
5.2% for Aston and 6% for Ulster. For many students this might amount to a
difference in degree classification. One observable difference between Aston
and Ulster concerns the impact of gender, which is statistically significant for
Ulster. Although it should be noted that the Ulster sample has a much higher
proportion of female students compared to Aston which is roughly 50:50.
► The overall conclusion which can be reached is that “more engaged students”
as measured by first year performance do choose to go on placement (as
shown by the probit model) but regardless of the level of engagement the
completion of a work placement year improves the students performance
in the final year.
Is there heterogeneity in the Aston results?
(Tables 4 & 5)
► Performance in the 1st year (the measure of student engagement) has a strong
and statistically significant impact on a student’s decision to do a placement. This
effect is stronger in L&S and IDS compared with the other Schools. These
results show quite clearly that engaged students are more likely to go on
placement.
► The Mills ratio is insignificant for each School. This suggests that there is no
selection effect at the School level. However, the coefficient for the placement
dummy variable does vary across schools.
► Nevertheless, the conclusion is the same as the general results above:
Engaged students do choose to do placements but this does not matter, a
placement year improves final year performance.
Is the benefit of placement the same for all
students? (Table 6)
► This question is easily analysed by splitting the data in to separate samples –
one sample for students who obtain an average of 60% or higher in their second
year, and one sample for students who obtain an average of less than 60% in
their second year. In other words we are assessing which students benefit more
from the placement.
► As no sample selection effect has been identified standard OLS regression is
employed. The dependent variable is simply the average final year mark and the
primary interest is the coefficient for the placement dummy variable. For each
sub-sample of students it is clear that going on placement improves
performance, the variable is positive and statistically significant for both Aston
and Ulster. It can therefore be concluded that students achieving a 2:2 or less
in the year prior to placement gain more from a placement compared to 2:1 or
higher students.
Does the “quality of placement” matter?
(Table 7)
► Placement quality is measured as the actual mark achieved in the placement
year. This means that the sample is now restricted to placement only students.
► Again self-election is ignored by excluding the Mills ratio from the analysis. In
addition, some students are omitted from the analysis for Aston because their
placement mark is either “pass” or “fail”.
► The authors accept that placement mark may not be an ideal way of measuring
the quality of a placement. However, it is argued that the higher the placement
score the more engaged the student has been on placement.
► The results suggest that increased placement performance has a positive impact
on the performance in final year. The coefficient estimates for both Aston and
Ulster are positive and significant however, they are quite small in magnitude.
CONCLUSIONS:
► The results of this study indicate that improved final year
performance is observed for students undertaking a work placement
year.
► Further, “more engaged students” as measured by first year degree
performance do complete a placement year. However there is no
evidence of sample selection bias, “the student engagement effect”.
► The evidence presented suggests that irrespective of the level of
engagement, the completion of a placement year “per se” is
associated with better final year performance. I
► In addition those students performing less well in prior university
assessment benefit more from taking a placement year in terms of
improved final year performance and the “quality of the placement”
is also related to final year performance.
Further work
► Follow up qualitative studies have been conducted at Ulster and
Aston.
Further information
Professor Helen Higson
[email protected]