Setting the Standards—Changing the Way We Measure Public Relations Success

Setting standards for public relations measurement may not sound sexy, but I would argue that it’s one of the hottest new developments in public relations this year.

When it comes to determining the success of public relations campaigns, practitioners have long measured a number of variables, including output metrics (such as “reach,” “impressions,” “web site visitors,” and tone), outcome metrics (such as awareness, understanding, perceptions, and intent), and business outcome metrics (ROI, customer loyalty, etc.).

Because there is a market-driven imperative to measure campaign success, our profession needs to create a toolkit of consistent, reliable, and comparable metrics that would allow practitioners to work more efficiently, to work more intelligently, to build organizational knowledge, and to document how public relations contributes to building organizational value. In the specific case of social media, a lack of consensus about measurement standards has led to client trepidation.

The time has come when we, as public relations practitioners and researchers, must develop industry-wide measurement standards. As an executive at a global automotive company told us recently, “We can’t afford the ‘dueling banjos’ of measurement anymore. We need to develop a position on standard definitions and metrics now or be left behind.” An executive at a major food and beverage company remarked that, “We must “create a standard measurement system across all programs and campaigns to deliver consistent, comparable data.”

Why do standards matter? Suppose you’re the Chief Communications Officer of a corporation with four business units, each working with its own internal public relations group, agency partner, and measurement firm. You have directed your staff to compare program tactics and results, using measurement, as the basis to understand what works and what does not work so well. You assign your chief of staff the task of reviewing program measurement results for the past year in order to develop a measurement knowledge center. Two weeks later, the chief of staff returns with her hands in the air; it seems that each business units, each agency, has been using its own metrics, making comparison quite difficult, if not impossible.

Help is on the way. The Coalition for Public Relations Research Standards has brought together leading professional organizations to address standardization of the most important measurements: Council of Public Relations Firms, Institute for Public Relations, PRSA, Global Alliance, and AMEC. We will be adding European, Latin American, and APAC organizations in the near future. Because standards are market-driven—voluntary, not imposed—client organizations will be part of the review and adoption process. The customer panel, composed of companies such as General Electric, General Motors, Southwest Airlines, and McDonald’s, will make the final determination that “this looks like a standard we can use.”

Under the Coalition’s umbrella, working groups composed of subject matter experts are developing proposed standards in the areas of social media measurement, traditional media measurement, communications lifecycle, ROI, and ethics. The Coalition has recently launched a microsite for the publication of proposed standards. This web site is the hub for review and comment by practitioners.

The Coalition took shape about a year ago when we realized that groups within the Council of Public Relations firms and other industry organizations were all working to address this challenge. Leaders within our industry decided to convene a steering committee and fold these initiatives together into one. The result is an unprecedented example of industry collaboration that will serve our collective interests as well as the interests of clients.

Key Benefits of Standards for Firms

  • Ability to learn what works and what doesn’t work
  • More credible measures to show to other functional areas within client firms
  • Better ability to sell firm services to clients.
  • Measurement becomes more accessible to account teams and firms.
  • Better ability to compete based on smart, creative application of the standards.

Public relations firms stand to benefit a great deal from the development and adoption of measurement standards. At present, PR firms spend large amounts of time educating  clients about why they use specific measures. With standards in place, firms can go beyond the basics and focus on delivering real insights. Firms will compete based on how they use and apply the standard measurements.

So how can you help? First, take a look at our standards website. As more standards are agreed upon and rolled out, please review our work, tell us if you like it or not, and make suggestions. Within firms, I’d ask leaders to encourage their staff—account staff as well as research staff—to learn about these measurement standards, adopt these standards, and let your clients know that you are adopting industry standards.

If you have your own thoughts about where and how standards might best be used, we’d appreciate hearing about it!

We are interested in hearing about where are the macro and micro areas where we do need standards.

This year, we’ve gotten to the point of getting to some agreed upon standards. Five years ago, people might have been skeptical we could have achieved this. Collectively we need to say we’re listening to the marketplace who say they want standards. We’re getting our act together and are going to roll out more within the next year. This is the most organized, thoughtful, logical initiative around measurement that’s happened. That’s because of market pressures and the support of the organizations. Standards give a common language for people to talk to each other. The industry is ready for it.

Benefits to firm leaders:

  1. They’ll have the ability to have comparable measurement data so they can work more smartly and efficiently because they can learn what works and doesn’t work.
  2. More consistent and credible measures to show to other functional areas within the company and to clients.
  3. We can sell programs and ideas better to clients. It will help us to grow if we have this figured out.
  4. Account teams and firms that do their own measurement will have resources to use that will save them a lot of time and money.
  5. Adopting these standards will allow you to become more competitive and innovative. You can compete based on how to USE and apply these standards to deliver insights.

20 Responses to “Setting the Standards—Changing the Way We Measure Public Relations Success”

  1. Deb Brown said on January 15, 2013 at 10:25 am

    We agree that standards in measurement are critical in demonstrating the important ROI that strategic communications programs provide. However, it’s important to build in standards that don’t restrict but, rather, allow for enough flexibility and customization so that each organization can measure exactly what it wants. For example, we have a proprietary measurement program at Peppercomm that provides a rigorous framework consisting of both the latest technology and the all-important human touch, but it has so much flexibility built in that each client can determine what metrics are important. The framework is the same from client to client, but the vast room for customization is what makes our measurement system extremely attractive to clients. The danger in standardization is when the metrics are standardized and the flexibility is limited. Standardization should be viewed in terms of categories (e.g. how we define audience reach) but allow enough room for the clients to tell us what’s important, such as how they want to weigh an aspect of a program. If done right, new standards will become a win-win for both the agency and client.

  2. David Geddes said on January 15, 2013 at 4:36 pm

    Deb-

    We appreciate the thoughtful comment. As chair of the Coalition for Public Relations Research Standards, I would like to comment upon your specific points. Our initiative was born out of frustration expressed by organizations (clients), agencies, and measurement firms. As my friend Katie Paine expressed it in a blog post on the Institute for Public Relations blog on August 22, 2012:

    “When anyone asks why standards are important, I cite a recent conversation with a client. It started with what seemed a simple question: ‘Are we able to see how many people saw the stories, and how do you calculate reach?’ A dozen emails later, we finally sorted out that what the client called reach, we call opportunities to see. What she was calling placement, we call items. And what we referred to as “placement” didn’t matter to the client at all.”

    Standards will deal with the definitions, terminology, and specifications at the foundation of measurement structures. Standards enable practitioners to talk with each other in a common language. By coming to agreement on definitions and specifications, agencies and research firms will save time educating and re-educating clients (and each other, for that matter).

    The Coalition has no intention of moving too far up the metrics hierarchy. For example, teams are working on measurement specifications for (i) audience reach for tradition print; (ii) reach, impressions, engagement, influence, relevance, impact, and value for social media; (iii) awareness, understanding, relevance, and advocacy in survey research; and (iv) ROI.

    Using these foundational metrics, agencies can then innovate, as your firm has done, by developing its rigorous frameworks for their clients. As I write in the blog, “with standards in place, firms can go beyond the basics and focus on delivering real insights. Firms will compete based on how they use and apply the standard measurements.”

    We’re glad that you see this as a win-win situation. Please keep in touch with this initiative,

  3. […] recent Council of Public Relations Firms blog post by vice president of research and development David Geddes proposes the creation and adoption of […]

  4. Philip Sheldrake said on January 18, 2013 at 8:34 am

    Hi David,

    You know we share this common ambition, but unless it’s just semantics I think we might differ in our estimation of what’s possible, perhaps indeed of what’s desirable. From our past conversations I’m not so sure, so would appreciate your clarification.

    To get to the crux…

    If your organization is unique, with a unique position in a unique market, with unique mission and vision, and uniquely tailored strategies demanding a unique mix of tactics, is it too great a leap of faith to expect the (optimal) blend of communications to be unique and for communications performance measurement to therefore require a unique and tailored blend of metrics?

    This is the underlying assumption for the Influence Scorecard methodology, an augmentation of the Balanced Scorecard addressing our purpose here. The Balanced Scorecard is, as you know, the dominant business performance management framework amongst the Global 2000, and I believe its practitioners accept the assertion I make above.

    Most pertinently, the Balanced Scorecard Institute defines a strategic business unit (SBU) as a unit of the company that has a separate mission and objectives, and can be planned and evaluated independently from the other parts of the company. In my book, I extend this definition by concluding that an SBU is an entity with one universal set of influence objectives.

    I know we agree that we won’t replace AVE with a universal metric. We won’t even replace it with a ‘standard’ handful. The only way forward in my opinion is to establish the framework to allow comms / influence to be measured in the context of the unique organisation or SBU, despite this making inter-SBU benchmarking more challenging than we might like.

    Like the Balanced Scorecard, the Influence Scorecard then requires the practitioner to select from perhaps many hundreds of metric possibilities to design the appropriate balance for his/her needs, to guide performance and for single- and double-loop learning (‘how are we doing?’ and ‘does our strategy remain the right one?’)

    I expand on this theme here: http://eulr.co/PDVArm

    Best regards.

  5. David Geddes said on January 18, 2013 at 1:12 pm

    Philip – We are in 100% agreement. Let me give an example. There are many makers of innovative mobile communications devices. Why is this possible? This diversity of distinctive products is only possible because the ANSI, the IEEE, and the ISO have developed standards so that mobile devices can communicate with each other.

    In public relations measurement, we are developing the nuts and bolts, so to speak, so that organizations can develop their own balanced scorecards, measurement dashboards, or whatever you want to call them. For example, we have proposed a specific way to ask “awareness” questions and are coming out with a specific way to measure social media reach. This will enable practitioners to compare program results. Let me return to my original hypothetical case. A CCO may face the situation that programs are measuring awareness using different questions, hence the results are not comparable. Our goal is to standardize the building block. Organizations and agencies can then develop innovative scorecards and dashboards suitable to their needs.

    What you are outlining falls into the domain of “best practices.” Best practices are about implementing standards.

    Thanks for the comment.

    • Philip Sheldrake said on January 20, 2013 at 4:22 pm

      Thanks again David. I think your use of examples is a great way to get to the nub of the matter. So here’s another. Company X has three SBUs…

      SBU1 makes software for statistical process control of a particular kind of machine tool. Approximately 3000 companies use such machines.

      SBU2 specialises in high value software components to analyse wing vibration in large commercial aircraft, a market with no more than five potential customers.

      SBU3, Company X’s third and most recent unit, was formed to develop and market a $1.99 smartphone app. The app employs the phone’s sensors to ascertain and categorise your car driving style. Its marketing communications team has found an entertaining sweetspot comparing ‘his’ and ‘her’ driving styles! Awareness is considered critical to “viral growth” amongst the many millions of drivers.

      SBU1 CCO has determined that the balanced suite of metrics that best guides performance and quantifies results includes a measure of awareness. SBU2 CCO considers any concept of awareness moot. SBU3 CCO considers awareness central to low cost customer acquisition.

      Would you agree that no-one can provide the Chief Exec of Company X with inter-SBU comparative measures of awareness? Is it possible that, while an universal definition of “awareness” is recognised, the exact nature of its quantification will be bespoke to the situational context? And that such tailoring advantages execution but at the expense of the universality of comparison?

  6. Kate Craig-Wood said on January 20, 2013 at 8:15 am

    This is really interesting – thanks to Philip to pointing me here via twitter.

    I must admit that for us PR of all flavours (inc social) is something of a leap of faith. I have set a modest budget of a few £k plus acknowledge that for it to really work I have to personally contribute – I have built up good relationship with some journalists.

    Our measurement methodologies are fairly poor, however. As part of our KPIs we track now many times we get cited in sets of publications (trade, local, national news, online/offline etc) which gives us a good idea of whether it is working or not.

    Fundamentally, however, the “ROI” on our PR is not actually related to how much we spend on it, but rather is related to how much interesting stuff we are doing.

    That said, I’d love to be able to measure it more precisely. SEO is a great example; we know exactly what our rankings are on our key terms and draw charts again it. Our SEO consultant does various complex analysis that I dont ned to understand as long as it shows her metrics and my own metrics to be a move forwards.

    With PR, as Philip says, you are often talking about something new or unique in which case performance of those stories cannot be readily assessed against other stories. We are still not even sure which ones wil do well half the time (we get some perverse reactions) making it all the harder to track.

    Having standardised building blocks would help us a lot.

    • David Geddes
      David Geddes said on January 20, 2013 at 10:11 am

      Kate, thanks for the view from a small agency. To address your overall comment, the following steps are essential in selecting metrics and KPIs. (1) determine your client’s business objectives, (2) delineate your target audiences and audience-specific communications objectives. These must support business objectives. (3) specify your measurable PR program objectives. Try to expand beyond output metics. Think about what you would like the target audiences to know, to think, and to do. (4) set baseline levels and targets. How much can you reasonably achieve? What constitutes minimal success? What goes beyond minimal success? What would be a home run, tomuse an American baseball analogy? (5) measure and apply your metrics to improvement.

      In reality, the process of setting measurable objectives is the hardest part.

      We will be coming out with an important standard on ROI in Q1 2013.

      The article below is a useful short summary of setting measurable objectives.

      http://www.instituteforpr.org/topics/setting-measurable-objectives/
      http://www.instituteforpr.org/iprwp/wp-content/uploads/Master-PR-Tactics-document-08-10-16.pdf

      Thanks for writing.

  7. David Geddes said on January 21, 2013 at 3:18 pm

    Philip, standards are all about HOW to measure. Best practices are about WHAT to measure and how to apply these measurement to business decisions.

    Standards address a specific question. If we give a group of independent observers (i) a standard set of definitions and instructions and (ii) a specific set of data to collect, will they come up with results that are, for statistical purposes, in agreement? In the case of your Company X, the WAY to measure awareness should use standardized measures of awareness. The Coalition has indeed proposed a specific set of questions to measure awareness, knowledge, interest, preference, and advocacy.

    What you discuss with Company X and the three business units is a question of best practices, not standards. “How” companies assemble metrics into dashboards, scorecards (balanced or otherwise), indices, and the like must be customized to each business.

    This is the stuff of setting up a robust communications measurement program. The steps have been laid out many times. See my response to Kate Craig-Woods where I cite several articles about how to set measurable objectives.

    At the end of the day, developing standards is not and should not be exciting. Developing useful scorecards to guide decision-making should be exciting. Companies and agencies that develop the most useful scorecards and apply the metrics to business decision-making best will build a sustainable competitive advantage. This is not easy.

  8. David Michaelson said on January 21, 2013 at 3:43 pm

    Before we discuss standards, it is critical to understand the benefits standards afford. This is discussed in depth in a peer reviewed paper I co-authored with Don Stacks (University of Miami). The paper is available from the following link.

    http://www.prsa.org/SearchResults/download/6D-050201/0/Standardization_in_Public_Relations_Measurement_an

  9. Angela Jeffrey said on January 23, 2013 at 5:17 pm

    Friends – I have worked with several PR agencies this year to help them develop measurement methodologies and select appropriate service providers. In this work, I’ve found the “quasi-standards” developed by the IPR and AMEC (in the Valid Metrics Framework) to be invaluable. As the Coalition standards are gradually ratified and rolled-out, I believe we’ll see a much larger proportion of PR agencies and in-house professionals take firm steps toward creating measurement programs that otherwise might have been too daunting. To Phillip’s point, the artistry with which one designs an actual measurement plan for a given client will always require a bespoke approach; but the individual pieces in the plan will be understood and recognized. Thank goodness – is all I have to say.

  10. Kristin Jones said on January 31, 2013 at 8:36 am

    What an interesting read. PR measurement is something that everyone in our field agrees must be done, but at this point not everyone is ready to devote the time and resources to effective measurement (meaning analyzing and applying data rather than just monitoring coverage results). Could setting standards across the board – be it terminology or expectations – help? I believe to an extent, yes. They will allow us to more effectively communicate with our clients and executives the results of data. And, they will help us know the exact information we need to find, the amount, how to report it and what terminology will best suit the needs of others to make reports clear and valueable. However, as Deb mentioned above, a benefit to flexible PR measurement is that it allows PR to pick the results that are most meaningful and customize measurement plans. I believe maintaining this flexibility is vital as the industry works to develop measurement standards.
    Thank you for posting. Kristin.

  11. David Geddes
    David Geddes said on January 31, 2013 at 11:41 am

    Kristin — Excellent comment. Let me return to my comment that standards provide the definitions, specifications, and methods for measurement. Let me use an example. A standard cell phone relies upon hundreds of standards. These allow the phone manufacturers to invest time in creating innovative devices, without worrying about whether the components, purchased from many suppliers will work together in the final product. In similar fashion, a company or agency can take standards being developed for social media (see http://bit.ly/11aDVC4 for a status report) and develop their own measurement platform, while using pre-determined standards. In other words, we are looking fort (i) standards plus (ii) innovation.

  12. Lucas Coe said on February 1, 2013 at 10:54 am

    Great post and a really interesting read.

    I guess the problem I have with standards is that it loses credibility when you consider the different needs of smaller and larger clients.

    Take social media for example – we all know that 10 lead conversions via Twitter will have much more value to a start-up business than an established brand in the FTSE 100.

    So my question is: how can something have the same value, when in real terms, it has entirely different implications for different people?

    • David Geddes
      David Geddes said on February 1, 2013 at 1:32 pm

      Lucas — Let me return to an earlier response: standards are about HOW to measure. Best practices are about WHAT to measure and how to set up a communications measurement program. A standard will only tell you HOW to measuring Twitter conversations. This kind of standard is being developed by the social media measurement standards group (see http://www.smmstandards.org/ for their work and roadmap). You are correct that it is up to every organization and agency to determine what the measurement means.

      The following example may help. I go to my physician for an annual checkup. He reports on my blood pressure, total cholesterol (120/70), HDL (40), LDL (80), total cholesterol (120) and triglycerides. These are standard metrics in the sense that I could go to any lab around the world, ask for these tests, and they will be measured in exactly the same way. This is all that standards are.

      But what do these numbers mean? On their own, nothing. They gain meaning when my physician ASSESSES or EVALUATES my health given my age, family history, and lifestyle, and then PRESCRIBES something to do (or not do … there go the fries).

      To get back to your comment, standards will not answer your question about the value of, say, a set of Twitter conversations. That is something that the company PR team needs to do using BEST PRACTICES. What does this involve? (1) Set goals and objectives. (2) Develop appropriate metrics. (3) Run program. (4) Measure results. (5) Assess results in light of goals and objectives. (6) Use the results of measurement and evaluation to make changes in communications strategy and tactics. (7) Repeat. In previous responses, I have listed a number of resources to help.

      I hope this is useful. one theme in the comments above is the misunderstanding of the differences between standards and best practices.

  13. The Measurement Minute said on February 8, 2013 at 1:42 pm

    […] David Geddes recently explored the possibilities and benefits behind industry-wide standards in his blog post, highlighting the ability to learn what works and doesn’t work, the ability to sell firm services […]

  14. Neelima Khanna said on February 27, 2013 at 12:06 am

    Thanks David for sharing this excellent piece. As you rightly pointed out, creating standards are essential for development of any process, product, innovation and an industry at large. Customized matrices for PR measurement can be created within the boundaries of standards that would be easily understood by all – agencies and clients. Here in India, in several of our client interactions, we are sometimes surprised when a client comes up with a distinct term for a data point that they may have created as part of any internal measurement process. We encourage them to adopt matrices which are very much based on AMEC & IPR’s proposed standards; matrices that help them to gain intelligence, understand trends, incorporate insights into business planning. Alongside, we also encourage them to give up matrices as AVEs or those which offer research delight but little business value e.g volume of mentions with little reference to tonality. Through this collective work, clients are able to migrate to research and insights which is in best of their interest and commonly understood by PR, marketing and top management.
    -Neelima Khanna, twtr: @khannaneelima

  15. David Geddes said on February 27, 2013 at 12:56 pm

    Neeli, you are a model for our industry! Let me use one example. Today, there is mass confusion about the definitions of, and how to measure,
    (i) reach and impressions for social media, (ii) engagement, (iii) influence, (iv) tone & advocacy, and (v) value. Fortunately, the Social Media Measurement Conclave is working on clarifying what these terms mean and will be providing specifications fort how to best measure them. For a recent summary, see http://bit.ly/11358Xt. The standards are just the foundation. Corporations, agencies, and research firms should be encouraged to innovate, but we do need to be sure we are all talking about the same thing.

    Second, as you note in your comments on the use of matrices, we need to move beyond the easy measures out outputs — such as story volume and AVEs — and measure with respect to communications objectives.

  16. […] that would allow practitioners to work more efficiently” as stated by David Geddes in his blog post about setting standards. Many people believed there was one “right” way to evaluate PR, and everyone in the […]

Leave a Reply