Teleconference:
Supporting Archived And Embracing Customers With Social Technologies Original air date: Thursday, June 05, 2008
FOR INTERACTIVE MARKETING PROFESSIONALS October 17, 2008
Metrics For Social Applications In A Downturn
Measure What Matters, Not Just What's Easy To Measure by Josh Bernoff with Christine Spivey Overby, Jennifer Joseph, Emily Murphy Length: 14 pages EXECUTIVE SUMMARY Given the skepticism that comes with social applications, you'll need metrics to prove their worth, especially with economic conditions deteriorating. Unfortunately, many interactive marketers we surveyed were still using metrics like unique visitors and page views rather than the metrics that matter, like sales, leads, or increases in awareness. But we also found innovative measurement techniques, like the Facebook campaign that counted how many women were willing to make a statement online about cervical cancer and the marketers at Dell seeing if searches on their products were more likely to land on their own blogs than on negative reviews. To succeed with measuring social applications, flex metrics to match evolving applications, build in hooks and tracking, and tap your vendors' and agencies' experience to avoid pitfalls. TABLE OF CONTENTS A Strategic Approach To Measuring Social Marketing Applications The Three Principles Of Social Application Measurement Marketers Find It Hard To Measure The Value Of Social Applications Interviews With Successful Marketers Show That You Can Measure Value RECOMMENDATIONS Measurement Pays Off As The ROI Of Social NOTES & RESOURCES Forrester interviewed 10 companies, including Carnival Cruises, Constant Contact, Dell, Emerson Process Management, HP, Intuit, Lego, and Sony. RELATED RESEARCH DOCUMENTS Measuring Engagement
June 10, 2008
The Interactive Marketing Metrics You Need
Applications Improves Supplemental Material
May 29, 2008
Interactive Marketers Are Bullish In a Recession
April 30, 2008
How To Choose The Right Social Technologies
March 4, 2008
Objectives: The Key To Creating A Social Strategy
October 9, 2007
A STRATEGIC APPROACH TO MEASURING SOCIAL MARKETING APPLICATIONS Even with a recession looming, enthusiasm among marketers for social applications has never been higher. But measuring the value of social applications is confusing. In the short run, it's a lot easier to just jump in, try things out, and figure out what to measure later, especially if you buy into the "conversation is good no matter what" school of social applications. The only problem is that this dangerous, shortsighted thinking could not only ruin your project but potentially hurt your career. Marketers use a variety of types of interactive marketing metrics. Now you must also measure the value of your blog, community, Facebook page, YouTube video program, or whatever you undertake.(see endnote 1) Here's why: Your social application should be aimed at a specific objective. Unless you measure progress toward that objective, you have no idea whether you're accomplishing anything valuable.(see endnote 2) Course corrections will be necessary. Once social applications get underway, you'll inevitably face decisions about how to evolve them. If you're measuring something that matters, you'll be able to tell if you've made the right decisions, or the wrong ones. Otherwise you're flying blind. You'll need proof that it's paying off. In a recessionary climate, your application will be fighting for resources with other marketing applications.(see endnote 3) The Three Principles Of Social Application Measurement But what to measure and how? There's no simple prescription. What you measure will depend heavily on what type of application you're building, what industry you're in, and what objective you're pursuing.(see endnote 4) In this document, we propose a strategic approach to measurement that will help you choose the metrics most likely to help you. This strategic approach is based on three principles: 1. Measure what matters. Given a choice, select metrics that have a proven ability to move your business like sales or conversions over broader, simpler metrics like traffic. 2. Think in terms of assets, not campaigns. In the end, whatever you are creating a community, a blog, an ambassador program will, if successful, be growing in value and utility
over time. Eschew the campaign-based metrics that come from advertising in favor of metrics that measure the long-term value of the asset you are building. 3. Evolve metrics as you evolve your application. Always be on the lookout for ways to improve your metrics as online technology changes and as your application evolves. MARKETERS FIND IT HARD TO MEASURE THE VALUE OF SOCIAL APPLICATIONS In Q2 of this year, we surveyed 205 interactive marketers on the social technology applications they deployed and how they measure them. This research revealed that optimism had outpaced hard metrics: Few measure direct returns. Less than half of respondents said they measured the ROI of their social applications (see Figure 1). Profitability remains in question. Their average 2007 application budgets ranged from a low of $19,000, for Twitter, to a high of $240,000, for advertising in social networks (see Figure 2). Not a single application was rated as profitable by even half of the marketers using it (see Figure 3). Easy metrics were popular but fail to make the case. The most popular metrics were based on traffic which is easy to measure but hard to translate into results rather than metrics closer to actual business goals (see Figure 4). Application by application, here's what marketers told us: For social networks, participation was more popular but less effective than advertising. When it comes to social networks like Facebook and MySpace, nearly twice as many of our respondents said they participated in the networks than advertised on them.(see endnote 5) Advertising on social networks was the most expensive program among our respondents, with an average 2007 budget of $240,000, but 36% of those advertising in this venue reported a profit, the highest number in our survey. By comparison, even though participating in a network like this cost only $76,000 on average, only 27% of those who participated rated it as profitable. For those advertising, the most popular metric was clickthroughs; for those participating, it was the number of friend connections. Ratings and reviews were among the most measurable applications. Of 61 respondents who had deployed ratings and reviews on their sites, 34% said they were profitable, and another 51% said they were valuable even if they couldn't measure that value exactly. Compared with other applications, ratings and reviews were relatively inexpensive with a budget of only $69,000 on average. The most popular metrics for ratings and reviews were volume of participation and unique visitors. They saw all types of communities as effective. We asked about three types of communities: private communities open by invitation only, communities of customers, and public communities open to anyone. Marketers rated all three similarly, with about one-quarter saying they were profitable and one-half calling them valuable. Again, the most popular metrics were traffic measurements like unique visitors, volume of participation, and number of participants. But more than half of the respondents in customer communities did report using customercentric metrics volume of user-generated content and number of customer contacts. Blogs were popular, cheap, and moderately measurable. More than half of our respondents had deployed corporate or employee blogs. At $43,000 a year, they were cheaper
than many other social applications. But only 24% of respondents using blogs said they were profitable. As with communities, the most popular metrics were unique visitors, volume of user participation, and page views. Videos and podcasts were hard to prove profitable. With an average budget of $93,000, videos intended for sites like YouTube were somewhat more expensive and more popular than podcasts, where budgets averaged $63,000. Twenty percent said the videos were profitable; 23% said the podcasts were. Both proved hard to measure, with unique visitors the most popular metric. Widgets and Twitter were too new to justify easily. The newest technologies in our survey were widgets, with an average budget of $78,000, and Twitter, which was the cheapest of all applications at only $19,000 per respondent participating. But as you might expect with new technologies, less than one in five respondents using these technologies said they were profitable. Marketers using either were most likely to justify it by measuring volume of user participation.
Figure 1: Less Than Half Of Interactive Marketers Surveyed Measure ROI
DOWNLOAD THIS FIGURE:
PowerPoint slides
Excel spreadsheet
Figure 2: Marketer Participation And Spending On Social Technology Applications
DOWNLOAD THIS FIGURE:
PowerPoint slides
Excel spreadsheet
Figure 3: Most Marketers Can't Prove Applications Are Profitable
DOWNLOAD THIS FIGURE:
PowerPoint slides
Excel spreadsheet
Figure 4: Marketers Favor Metrics That Are Easier To Measure But Relate Less To Business Goals
DOWNLOAD THIS FIGURE:
PowerPoint slides
Excel spreadsheet
Interviews With Successful Marketers Show That You Can Measure Value With the marketers in our survey focused on such simple metrics and so often unable to prove ROI, we wondered are there best practices other marketers can learn from? We went back to
some people with successful social applications many of them from our book, Groundswell and found some companies that were in a better position to prove the value of their applications.(see endnote 6) Some best practices: The value of social networks is in how messages are passed along. Don't just count your Facebook and MySpace friends. Instead, examine how they spread your messages. For example, to measure the value of the Facebook application for Merck's Cervical Cancer vaccine Gardasil, its agency Rapp counted not just the brands' 55,000 fans but the 5,000 who were engaged enough to post graphical messages on its wall. Disposable contact lens brand Acuvue wanted to spread awareness in Australia through a Facebook application that called for people to send "winks" to each other not only did 500,000 people receive those winks, but sales volume went up 17% in the following quarter. If you're advertising, rather than participating, consider Facebook's new engagement ads.(see endnote 7) Only sales matters for ratings and reviews. Ratings and reviews work best on retail sites. To see if they're working, track sales. Last year's Forrester Groundswell award winner Netshops used PowerReviews ratings to boost sales by 26%.(see endnote 8) Marketers interested in pursuing ratings applications should be patient; it takes a while for customers to generate enough ratings to benefit other visitors to the site. Measure increases in overall sales, sales of top-rated products, and sales that follow viewing a ratings/reviews page. The value of communities comes from their objectives. Communities are one of the only social technologies that can be deployed for any of the five main social objectives: listening, talking, energizing, supporting, and embracing. Metrics should with align these objectives.(see endnote 9) For example, Intuit sets up discussion forums so that customers can support each other and surveys those customers to determine how many got their questions answered. Email marketing services provider Constant Contact measures the engagement of customers in its community, scoring them on retention, referrals, and the size of the mailing lists they manage all of which directly drive the company's business. Dell measures the success of its Ideastorm community, an embracing application, based on the number and value of the ideas it produces. Bloggers can track leads and monitor search engine rankings. Traffic is one way to measure the value of a blog, but it's not the most important. To blogger Jim Cahill at Emerson Process Management, a factory automation systems vendor, success means 10 to 12 leads per week, including requests for quotes (RFQs) that can turn into $500,000 sales. One key value of blogs is how highly they rate on search engines. At HP, Alison Waterson tracks what the cost of those click-throughs would be through paid placements. Dell's head of community, Bob Pearson, examines search results for product names, seeing if links to his company's blogs fill slots that might otherwise be taken up by negative reviews. For videos and podcasts, build in tracking measures. Every marketing YouTube video should have a URL at the end to determine whether viewers are just perusing the video or following up; marketers should also track where the video is embedded in other popular Web spots to identify how it's spreading. Podcasts have become less popular, partly because it's hard to track their value. We recommend that marketers follow up with podcast subscribers to determine how many went on to consideration or purchase. Widgets need to tell marketers how they're being used. Widgets need a marketing push of their own to get established we saw billboards for UPS' cute package-tracking-and-news widget (the character's name is actually "Widget") all over Italy. Is it worth it? To find out, widgets should report back to the home office on how often they're used and when they're spread to others.(see endnote 10)
Twitter's value depends on how it fits into your strategy. Comcast has used Twitter and other online interventions to solve 6,000 customer support problems a good measurement for this strategy would be a decrease in negative online sentiment. Dell uses Twitter to let people know about discounts and has already sold more than half a million dollars worth of hardware through this channel. Your best measurement for Twitter will vary based on your objective.(see endnote 11)
RECOMMENDATIONS
MEASUREMENT PAYS OFF AS THE ROI OF SOCIAL APPLICATIONS IMPROVES Social applications defy traditional ROI logic. As they scale and community members take on tasks like helping with moderation that were once your job, their costs can come down and their value can go up. But you'll never make it to that point if you haven't positioned your measurement strategy to prove they're working and adding to the engagement of your customers with your brand.(see endnote 12) Be flexible and creative with measurement. Social applications often need adjustment in the early stages as the unpredictable behavior of consumers goes in directions designers don't anticipate. Marketers using social applications often find they need to adjust their metrics to flex with their evolving applications. Build in hooks and tracking. If you don't build measurement that matters into your original application, you'll be kicking yourself as it takes off. What good is traffic to your brand? Build in Web visits, opt-ins, and other tracking mechanisms so you can tell what's working. The good news: Since social applications are typically custom-built, you'll have the chance to make measurement part of what you're creating from the start. Pick vendors that make metrics part of the plan. Technology vendors and agencies tout their experience. Ask them how they measure success, and look for creativity and experience in the answers. Vendors like Bazaarvoice have experience with ways to measure progress; they'll help you go right to measuring what matters. Don't pick partners who focus on technology over business goals.(see endnote 13)
SUPPLEMENTAL MATERIAL Methodology Forrester's May 2008 Global Social Application Measurement Online Survey was fielded to 205 interactive marketing professionals and social application vendors on behalf of their clients from our ongoing Marketing & Strategy research panel. The panel consists of volunteers who join on the basis of interest and familiarity with specific marketing topics. For quality assurance, panelists are required to provide contact information and answer basic questions about their firms' revenue and budgets. Forrester fielded the survey during May 2008. Respondent incentives included a summary of the survey results and a copy of reports published from the data. Exact sample sizes are provided in this report on a question-by-question basis. Panels are not guaranteed to be representative of the population. Unless otherwise noted, statistical data is
intended to be used for descriptive and not inferential purposes. If you're interested in joining one of Forrester's Research Panels, you may visit us at http://Forrester.com/Panel.
Companies Interviewed For This Document
Carnival Cruises Constant Contact Dell Emerson Process Management HP Intuit Lego Sony
ENDNOTES 1. With increased interactive marketing spend comes increased expectations; tracking results through basic measures such as click-throughs won't be sufficient. Interactive marketers need to progress from using these simple operational measures to looking at overall customer engagement with their messaging. See the May 29, 2008, "The Interactive Marketing Metrics You Need" report. Back to Text 2. To succeed with a social application, start by choosing your objective: listening to, talking with, energizing, supporting, or embracing your customers and their ideas. See the October 9, 2007, "Objectives: The Key To Creating A Social Strategy" report. Back to Text 3. Our survey of 333 interactive marketers revealed strong support for maintaining or increasing budgets in categories including social networking, email, blogging, and search optimization. In a recession like this, marketers should focus on the measurability of their online and social applications and think in terms of building long-term assets, not one-off campaigns to boost quarterly sales. See the April 30, 2008, "Interactive Marketers Are Bullish In A Recession" report. Back to Text 4. We described which technologies work best for listening, talking, energizing, supporting, and embracing objective...