Transcript Assessment
Advancing Competency-based Assessment in Entrepreneurship Michael H. Morris, Ph.D. Professor and N. Malone Mitchell Chair Head, School of Entrepreneurship Oklahoma State University Assessment of What? (for us---student level, course level and program level) Undergraduate Curriculum Scholarly Activity/Research Graduate Curriculum Campus-based Outreach Experiential Learning Integrated ‘E’ Programs Community-based Outreach Cross-Campus Initiatives Competency : defined Competence is a fuzzy concept useful in bridging the gap between education and job requirements (Boon and van der Klink (2002) To have competencies is to possess the necessary attributes to perform competently (Burgoyne, 1988) A characteristic of an individual that has been shown to drive superior job performance (Hartle, 1995) Observable behaviors that superior performers exhibit more consistently than average performers (Klein, 1996) IMAGINE > BELIEVE > CREATE OKLAHOMA STATE UNIVERSITY Competency : defined Competencies include knowledge, skills, attitudes, values, behaviors and characteristics that people need to do a job successfully (Bryant & Poustie, 2001) They correlate with job performance, can be measured against standards, and can be improved with training (Bryant & Poustie, 2001) One can contrast areas of competence (aspects of the job which an individual can perform) with competency (a person’s behavior underpinning competent performance) Competencies are connected to activities & tasks, subject to learning and developmental processes, and tend to be interrelated (Bergevoet, Mulder and Van Woerkum, 2005) IMAGINE > BELIEVE > CREATE OKLAHOMA STATE UNIVERSITY Why teaching competencies matters… ‘competency’ is a term that allows for flexibility in adapting to diverse and changing organizational demands (Garman and Johnson, 2006) From an hrm perspective, a competency is something that can be developed (Klarus, et al., 1999) Management vs. Entrepreneurship Management: Getting people together to accomplish desired goals and objectives. It comprises PODSCORB Entrepreneurship: Recognizing and exploiting opportunity through new combinations. But what does it comprise? IMAGINE > BELIEVE > CREATE OKLAHOMA STATE UNIVERSITY The entrepreneurial experience… Limited Sense of Control Ambiguity Loneliness Dejection Stress Exhilaration Freedom Uncertainty Responsibility Self-reliance Adaptation Discipline Learning Change IMAGINE> BELIEVE > CREATE OKLAHOMA STATE UNIVERSITY Does the context matter? Start-up Small or family business Buying a business Franchising Corporate entrepreneurship Social entrepreneurship Public sector entrepreneurship Entrepreneurship in a discipline (art, engineering) IMAGINE > BELIEVE > CREATE OKLAHOMA STATE UNIVERSITY Or is the behavior universal? Innovative, risk-taking, proactive behaviors Acting upon opportunity Unique combinations Creating something from nothing (or from something very different) A mindset that is both attitudinal and behavioral …regardless of the context IMAGINE> BELIEVE > CREATE OKLAHOMA STATE UNIVERSITY And so… Perhaps the managerial requirements or competencies needed to succeed in social entrepreneurship or corporate entrepreneurship or new venture creation differ But do the entrepreneurial competencies differ? Perhaps not! IMAGINE> BELIEVE > CREATE OKLAHOMA STATE UNIVERSITY Stage 1 Methodology: Delineating Competencies Two expert panels Three waves Survey Monkey Produced total of 167 competencies Split into two major groups: managerial and entrepreneurial Eventually arrived at 13 core entrepreneurial competencies The key competencies (note their interdependencies) Recognizing Opportunity Assessing Opportunity Vision/Seeing the Future Creative Problem-solving Resource Leveraging/Bootstrapping Mitigating and Managing Risk Planning/Modeling When Nothing Exists Innovation---Value-driven New Product and Concept Development Building and Managing Networks The Ability to Maintain Focus Yet Adapt Action Orientation/Implementation Tenacity/Perseverance Ability to Learn from Experiences IMAGINE > BELIEVE > CREATE OKLAHOMA STATE UNIVERSITY Distinguishing Sample Competencies managerial Organizing Team building & Staffing Communicating Budgeting Controlling Motivating Planning Directing Operating Assessing entrepreneurial Recognizing Opportunity Assessing Opportunity Creative Problem-solving Resource Leveraging Guerrilla Skills Mitigating and Managing Risk Planning When Nothing Exists Innovation---Products, Services, Processes Building & Managing Social Networks Adaptation while Focusing Implementation of Something Novel or New So we are doing both Developing managerial competencies in the business school Developing entrepreneurial competencies in the entrepreneurship program Both are needed for success in an entrepreneurial context, although the relative importance of a given competency will vary IMAGINE> BELIEVE > CREATE OKLAHOMA STATE UNIVERSITY How do we teach these competencies? Lectures define each competency illustrate each strategies for managing each examples relate to other competencies and learning points repetition Experiential learning in the classroom Experiential learning outside the classroom Teaching: experiential learning Cases Student incubators Tech commercialization teams Small business consulting projects Entrepreneurial audits Marketing inventions Creativity field experiences (e.g., the Lowe’s experience) Simulations Entrepreneurs in the classroom Interviews of E’s Unique internships Mentorships and job shadowing Role plays (VC’s, family firms, etc.) Business models Business plans and competitions Social entrepreneurship projects in the community Linking experiential learning to outreach Native American Entrepreneurship Academy Community Microcredit Fund Entrepreneurship Empowerment in South Africa Inner City Engagement Disabled Veterans Bootcamp OSU Entrepreneur’s Bootcamp Riata Business Plan Competition Entrepreneurial Mentors Program OSU Technology Commercialization Initiative Dilemmas and Debates Enterprise Creation Competition, a national business plan competition Women Igniting the Spirit of Entrepreneurship The Experiential Classroom Riata Entrepreneurial Internships Distinguished Lectures & Workshops Commercial Test Kitchen Connectivity is key Classroom Experiential Learning Outreach Integrate across the E curriculum Contexts Start-up Ventures Early growth firms Entrepreneurship within Professions & Disciplines Non-Profit & Social Family Businesses Entrepreneurship Rapid Growth Public Sector Ventures Cultural Entrepreneurship Corporate Entrepreneurship Academic Entrepreneurship Competencies Opportunity Identification Risk Management Opportunity Assessment Creativity Resource Leveraging Adaptation Guerrilla Techniques Social Networking New Product- ServiceProcess Development Implementation Skills Novel planning Measuring competencies—some caveats Luken 2004; Le Diest and Winterton, 2005 the definition of a given competence is not a homogeneous definition competencies are not stable competence assessments are always subjective competence assessments are based on individuals, whereas the definition of the competence concept tries to include the context as well Competence or skill is tied to an individual, but the individual’s competence may vary depending on context the competence concept includes capacity, whereas it is also important to look at actual performance Sub-dimensions in mastering a competency Knowledge and Understanding: what do you need to know about resource leveraging Attitude/Affect and Self-Awareness: what do you need to think, believe and feel about resource leveraging? Skills and Behaviors: what do you need to be able to do in terms of resource leveraging? These are all learning outcomes We can do more not just in terms of conveying knowledge, but in all three areas, especially to the extent that we stress experiential learning IMAGINE> BELIEVE > CREATE OKLAHOMA STATE UNIVERSITY Each competency requires a definition and level of proficiency Definition: What do we mean by the competency? Level of Proficiency: What must the student be able to demonstrate in terms of knowledge, skills, capabilities and attitudes he competency to indicate mastery of the competency? An illustration Example: Risk management Definition: The ability to identify relevant risks surrounding an entrepreneurial action and systematically mitigate those risks. Level of Proficiency: 1. Understands key types of risks 2. Can identify principle risks surrounding a given entrepreneurial action 3. Can prioritize risks based on magnitude and probability of loss 4. Is able to develop specific actions to -stage the risk -share the risk -reduce the risk IMAGINE> BELIEVE > CREATE OKLAHOMA STATE UNIVERSITY Sample ways to break it down Knowledge Attitudes/values Nature of risk versus uncertainty Dimensions of risk Categories of risk General techniques for mitigating risk Willingness to assume moderate levels of risk Belief that risk is manageable Sense of association between risk level and potential return Behaviors/Skills Ability to estimate risk Ability to isolate risk Ability to moderate level of risk A second illustration Example: Opportunity identification Definition: The ability to specify unrecognized or unfilled gaps in the external environment creating an opening for a new product, service or process. Level of Proficiency: 1. Understands general sources of opportunity 2. Is capable of scanning the environment to identify emerging patterns & trends, competitor shortcomings, unutilized resources & unmet needs 3. Can connect an opening in the environment to a specific target audience with a need IMAGINE> BELIEVE > CREATE OKLAHOMA STATE UNIVERSITY Sample ways to it down… Knowledge Attitudes/values Key sources of opportunity Major types of opportunity The nature of opportunities Four ways in which opportunities are identified Understanding of specific opportunity generation techniques Curiosity about why things work a certain way Value one places on being alert to opportunity Openness to being exposed to diverse and changing situations Behaviors/Skills Ability to draw associations Ability to grasp and hold onto ideas as they occur to us Ability to assess customer needs Response to a failure (e.g., elevator pitch loss) # of opportunities generated Novelty of ideas generated Measurement approaches Pre- and post- measures using rating scales Judging experiential project portfolio Behavioral event interviews In class assessments tied to exercises Student diaries or registers Peer assessments Self-assessments at end of program Behavioral assessments after graduation (see also Bird, 1995) IMAGINE> BELIEVE > CREATE OKLAHOMA STATE UNIVERSITY Examples of existing scales that tie in Innovativeness: Kirton Adaption Innovation Inventory Networking: Hatala Network Accessibility Scale Ambiguity Tolerance: Budner’s Tolerance for Ambiguity Scale Adaptability: Haynie and Shepherd’s Cognitive Adaptability Scale Setting the standard for a rubric Criterion-referenced evaluation: student performance is assessed relative to standards set by the discipline or entrepreneurship faculty Norm-referenced evaluation: students are evaluated on the basis of comparisons to other students Establishing norms We have no norms Benchmarks must be established Suggest we initially evaluate students relative to one another Over time we might create benchmarks using successful entrepreneurs But----are certain competencies more critical for success in certain types of contexts? Is our focus less on achieving some absolute level on a competency ---- or more on showing improvement relative to where a student started? Competency is a process of continual development through one’s life---not riding a bike---fades without practice Sample template for scales (Mertler 2001) Beginning 1 Developing 2 Accomplished 3 Exemplary 4 Score Criteria description description #1 description description Criteria description description #1 description description Criteria description description #1 description description Criteria description description #1 description description Total Score = _______ Score at Program Outset _______ Average Student Score ________ Some other tips Match measurement devices to instructional goals Do not rely on a single measure of a competency Teacher judgment is the primary means for assessing competency Certify competency in progressive stages Get the students on your side Allow for easy adjustment of competency measures as content changes Allow a feedback loop where measurement of competencies is used to modify course content and curriculum design -SeeTyo (1980) “Be the change you wish to see in the world.” -M. Gandhi