Transcript Slide 1
Where are we? Where to next? Jim Macnamara PhD, FPRIA, FAMI, CPM, FAMEC Professor of Public Communication Professor Jim Macnamara PhD, FAMI, CPM, FPRIA, FAMEC University of Technology Sydney University of Technology Sydney This discussion A quick update on international standards Continuing gaps and inconsistencies Some further barriers to overcome Professor Jim Macnamara PhD, FAMI, CPM, FPRIA, FAMEC University of Technology Sydney Barcelona Declaration 2010 Goal setting and measurement are fundamental aspects of any PR program Measuring outcomes is preferred to measuring outputs The effect on business [organisation] results should be measured where possible Media measurement requires quantity and quality Advertising value equivalents (AVEs) are not the value of PR Social media can and should be measured Transparency and replicability are paramount Professor Jim Macnamara PhD, FAMI, CPM, FPRIA, FAMEC University of Technology Sydney International standards push Coalition for Public Relations Research Standards • Established 2011 by AMEC, IPR and CPRF Social Media Standards Conclave (#SMMStandards) • Established 2012 by Coalition + 8 other PR/communication organizations – Global Alliance, PRSA, CIPR, IABC, SNCR, FIBEP, WOMMA, DAA • Working with eight companies – Dell, Ford, General Motors, McDonalds, Proctor & Gamble, Thompson Reuters, South-West Airlines, SAS • Also consulting with – Media Ratings Council, AAAA, ANA, Web Analytics Association Measurement Summit discussions 2011–2013 Academic input? Professor Jim Macnamara PhD, FAMI, CPM, FPRIA, FAMEC University of Technology Sydney Professor Jim Macnamara PhD, FAMI, CPM, FPRIA, FAMEC University of Technology Sydney M&E outputs and guides Proposed Interim Standards for Metrics in Traditional Media Analysis Social Media Standard Definitions for Reach and Impressions Sources and Methods Transparency Table Institute for Public Relations (IPR) Association for Measurement and Evaluation of Communication (AMEC) Professor Jim Macnamara PhD, FAMI, CPM, FPRIA, FAMEC University of Technology Sydney PRIA Evaluation Model Organisation/Business Objectives Communication Objectives Evaluated against objectives Formative Research PR Activity Undertaken PR Outputs Analysed For example: • Audience reach • Tone/sentiment of communication • Key messages • Share of voice • Competitor benchmarking • Media relations • Publications sent • Events held • Sponsorships undertaken • Communication engaged in Measurement Monitoring Q: Did activities generate outputs that reached publics effectively and efficiently? Professor Jim Macnamara PhD, FAMI, CPM, FPRIA, FAMEC University of Technology Sydney PR Outtakes Analysed PR Outcomes Analysed Organisation/ Business Results • Message recall/ retention • Consideration • Responses • Feedback • Engagement • Awareness • Understanding • Consideration • Attitudes/perceptions • Action/behaviour • Relationships • Reputation Evaluative Research Evaluative Research What outtakes and outcomes were achieved among key publics and to what extent? • Brand positioning • Sales/profits • Market share • Share price • Donations/grants • Customer/staff retention Reporting Did organisation/business results shift as a consequence? How far we’ve come … Professor Jim Macnamara PhD, FAMI, CPM, FPRIA, FAMEC University of Technology Sydney Definitions – e.g., ‘reach’ and ‘impressions’ “… REACH represents the total number of unique people who had an opportunity to see an item” #SMMStandards, 2012, para. 15. IMPRESSIONS– the number of people having the opportunity for exposure to a media story; also known as ‘opportunity to see’ (OTS) … usually refers to audited circulation” Proposed Interim Standards for Metrics in Traditional Media Analysis (Eisenmann et al., 2012, p. 3) Impressions – “the number of people who might have had the opportunity to be exposed to a story” Dictionary of Public Relations Measurement and Research (Stacks & Bowen, 2013, p. 14) “Impressions represent the number of times an item was displayed” Social Media Standards Definitions: Reach and Impressions (Digital Analytics Association, 2013) “Impressions represent the gross number of items that could have been seen by all people, including repeats” Impressions represents “the number of possible exposures of a media item to a defined set of Professor Jim Macnamara PhD, FAMI, CPM, FPRIA, FAMEC stakeholders” … Sydney University of Technology Dictionary of Public Relations Measurement and Research (Stacks & Bowen, 2013, p. 14) Return on Investment (ROI) PR ROI Return on Impressions (ROI) Return on Media Investment (ROMI) Return on Target Influence (ROTI) Return on Earned Media (ROEM) Return on Expectations (ROE) Return on Communication Investment (ROCI) Return on Brand Investment (ROBI) Social media ROI (SROI) Professor Jim Macnamara PhD, FAMI, CPM, FPRIA, FAMEC University of Technology Sydney Metrics Basic outputs Outputs Outtakes Outtakes Outcomes Counting press clippings Unique visitors Engagement Audience Views Influence Reach Likes Impact Target audience reach Followers Awareness Impressions Fans Attitudes Opportunities to see (OTS) Clickthroughs Trust Share of voice Downloads Loyalty Cost per thousand (CPM) Comments Reputation Hits Tone Relationships Visits Sentiment ROI? Professor Jim Macnamara PhD, FAMI, CPM, FPRIA, FAMEC University of Technology Sydney PRIA Evaluation Model Organisation/Business Objectives Communication Objectives Evaluated against objectives Formative Research PR Activity Undertaken PR Outputs Analysed For example: • Audience reach • Tone/sentiment of communication • Key messages • Share of voice • Competitor benchmarking • Media relations • Publications sent • Events held • Sponsorships undertaken • Communication engaged in Measurement Monitoring Q: Did activities generate outputs that reached publics effectively and efficiently? Professor Jim Macnamara PhD, FAMI, CPM, FPRIA, FAMEC University of Technology Sydney PR Outtakes Analysed PR Outcomes Analysed Organisation/ Business Results • Message recall/ retention • Consideration • Responses • Feedback • Engagement • Awareness • Understanding • Consideration • Attitudes/perceptions • Action/behaviour • Relationships • Reputation Evaluative Research Evaluative Research What outtakes and outcomes were achieved among key publics and to what extent? • Brand positioning • Sales/profits • Market share • Share price • Donations/grants • Customer/staff retention Reporting Did organisation/business results shift as a consequence? The so-called barriers Cost Time Too hard Management don’t want it Professor Jim Macnamara PhD, FAMI, CPM, FPRIA, FAMEC University of Technology Sydney Major barriers to address Conflation of measurement and evaluation Findings primarily used for reporting Measurement Evaluation Data collection Data analysis Early findings used to fine-tune campaign Measurement is taking measures – metrics Evaluation is determining VALUE Value = a perception from a perspective Professor Jim Macnamara PhD, FAMI, CPM, FPRIA, FAMEC University of Technology Sydney Major barriers to address Most measurement and evaluation look backwards • “Tells us what we know intuitively” • “What’s done is done … can’t change the past” • “Post-rationalisation and self-justification” Look forwards • What insights can improve PR/communication/relationships in future? • What insights can contribute to the organisation’s strategy and outcomes? – – – – – Gaps Trends Opportunities Competitor strategy Emergent issues Professor Jim Macnamara PhD, FAMI, CPM, FPRIA, FAMEC University of Technology Sydney Obsession with numbers Professor Jim Macnamara PhD, FAMI, CPM, FPRIA, FAMEC University of Technology Sydney Conclusions Recognise measurement and evaluation as different and separate • One is about metrics and analytics, the other is about VALUE Collect qualitative research data as well as quantitative • Noting that many outtakes, outcomes and outflows are perceptions and attitudes (e.g., satisfaction, trust, reputation) and relationships Look forward, not simply reporting retrospectively • Contribute to organisational strategy and outcomes Professor Jim Macnamara PhD, FAMI, CPM, FPRIA, FAMEC University of Technology Sydney Professor Jim Macnamara PhD, FAMI, CPM, FPRIA, FAMEC University of Technology Sydney References/resources Interim Standards for Metrics in Traditional Media Analysis http://www.instituteforpr.org/topics/proposed-interim-standards-for-metrics-in-traditionalmedia-analysis Social Media Standards ‘Sources and Methods Transparency Table’ http://www.smmstandards.com/category/content-sourcing-methods/ ‘Social Media Standard Definitions for Reach and Impressions’, http://www.smmstandards.com/2013/03/proposed-social-media-standard-definitions-forreach-and-impressions-from-the-digital-analytics-association/. Also at http://www.smmstandards.com/wp-content/uploads/2013/03/SMM-StandardDefinitions_DAA_v4.pdf Jim Macnamara, (2014), ‘The development of international standards for measurement and evaluation of public relations and corporate communication: A review’, available at http://www.uts.edu.au/research-and-teaching/our-research/australian-centre-publiccommunication/research/research-reports Tom Watson & Anzgar Zerfass, (2011), ‘Return on investment in public relations: A critique of concepts used by practitioners from communication and management sciences perspectives’, PRism, vol. 8, no. 1, available at http://www.prismjournal.org/vol8_1.html Professor Jim Macnamara PhD, FAMI, CPM, FPRIA, FAMEC University of Technology Sydney