CHATTER MATTERS - social media research is reaching its tipping point

SummaryThis report is an overview of social media research as it appears in mid-2011, so, by the time you read it, things may have changed. Where possible, it reports the consensus view while recognizing that there are passionate disagreements about definitions, procedures and deliverables. While social media research holds great promise, most of its successful implementations are still unable to meet the requirements of anonymity, scientific criteria of reliability or any clear separation from marketing.

Copyright 2011 American Marketing Association All Rights Reserved Marketing Research

Fall 2011

FEATURES; Pg. 22 Vol. 23 No. 3

3942 words

 CHATTER MATTERS

Social media research is reaching its tipping point

From a commercial point of view, social media research can be divided into three categories:  social media monitoring; purposed communities (e.g., MROCs and community panels);   other forms of social media research (e.g., research into social media, netnography and social media as a sample source). The first two categories account for more research spend than the third, a trend that is unlikely to change before 2013

 SOCIAL MEDIA MONITORING

Social media monitoring (also referred to as buzz monitoring, blog mining or simply social media research) refers to research based on listening to the discourse of the Web, especially social media, and usually refers to the use of automated tools to process that discourse, looking at thousands or millions of naturally occurring conversations. 
An example of a passive or category type of social media monitoring was the CREEN project (2005). CREEN monitored 100,000 blogs for three years and recorded dual instances of science-related words with fear/anxiety-related words. The project tracked the volume of hits over time and when peaks were observed, researchers reviewed the terms that were driving the increases. Examples of spikes in the data were "Schiavo" (relating to the Terri Schiavo life support case) and "stem" (relating to stem cell research).
An example of brand-focused monitoring is the hotel satisfaction research conducted by Accor with the agency Synthesio. In this study, Synthesio tracked 4,000 specific Accor hotels, along with 8,000 competitors (in eight languages), to produce a global dashboard, 40 regional dashboards and 4,000 hotel-specific dashboards. Each dashboard displays key competitors and is updated weekly. The dashboard combines analysis of open-ended comments in social media, scores from evaluation sites such as TripAdvisor and Booking.com, and more traditional measures. The system allows Accor to quickly identify underperforming hotels and locate and act on individual negative comments. As a result, Accor has reported a rise in brand equity, satisfaction and bookings.

COMMERCIAL SOCIAL MEDIA MONITORING
Although there are some free social media tools, most of them only skim the discourse of the Web. In order to conduct serious research, it is necessary to either develop tools or use one of the many commercial services, such as Radian6, Lithium or Synthesio.  
Social media monitoring starts by collecting a body of text to analyze (a corpus) and uses spiders/bots to crawl relevant parts of the Web and social media. Although the tools are increasingly powerful, they do have limitations. For example, not all of the Web is accessible (e.g., large parts of Facebook). Once obtained, the data then needs to be cleaned. Posts that originate from the client, from its various agencies and from bots need to be identified. Erroneous matches also need to be removed. For example, when looking at soft drinks, Coke the drink is good, but coke the drug is not.

Analysis includes one or more of the following: counts, trends, sentiment analysis and influence.  

Counts. Counting how often a word or phrase occurs is fairly superficial, but sometimes word counts turn out to be interesting. Several studies have reported success in correlating the frequency of politicians being mentioned and their success in elections. (This was true with Facebook during the 2010 U.S. elections and Tweetminster during the 2010 U.K. general election.) Google has shown success in mapping diseases by analyzing the words that people type into search engines to identify incidences of flu and, more recently, Dengue fever.  

Trends. Trends are an extension of counting, looking at whether a term is becoming more used or less used over time and checking to see if it can be linked to other phenomena. Looking at whether a term is trending on Twitter has become a key component in reputation management and brands look to see if they can identify social media trends associated with their campaigns. 

Sentiment analysis. Sentiment analysis is either the core benefit of social media monitoring or the snake oil of 2011, depending on who is talking. Sentiment analysis can code posts as positive, negative or neutral. The number of posts collected by automated social media monitoring is too large to code manually, so one of the following approaches is followed: 

• automated techniques, applying a variety of algorithms; 
• coding some of the data manually, allowing software to "learn" how to code the rest; 
• or manually coding a sample of the database.

Key disputes about sentiment analysis relate to how accurate the automated coding is and how accurate it needs to be. A study conducted by Freshminds in 2010 concluded that, when comparing posts classed as positive and negative, the accuracy was often below 50 percent.
The analysis of automated sentiment analysis needs to take into account the differences between manual coders and their costs. As Conversition's Annie Pettit has said, if manual coders are 80 percent accurate and automated systems are 70 percent accurate, we'll need to use automated systems some of the time because of the volumes they can process.
At the moment, most experienced users of sentiment analysis do some or all of it manually. Buyers need to be aware of excessive claims from some suppliers, with some even claiming 98 percent accuracy or better! 



Influence and identification. Most of the tools used for social media monitoring were not designed specifically for market research. The tools are equally suitable for marketing. As well as identifying important pieces of information, the platforms can find out who is saying what, allowing them to be targeted for marketing actions.
This power to find out who is saying what, who is listening to whom and who appears to have influence is a twofold challenge for market research. First, it risks removing the anonymity of the people being researched. Second, if market researchers are not prepared to be involved in response marketing, they may become marginalized.

THE BUSINESS OF SOCIAL MEDIA MONITORING 
Despite its earlier promise, social media monitoring has only made modest inroads into market research so far. Growth has been limited in the following ways: 

1. Social media monitoring only reports what people are talking about. If they are not talking about you or your issues, it can't help.
2.  Social media monitoring is much more expensive than was expected. Even monitoring a few brands and terms across a few markets is likely to cost several thousand dollars a month.
3. The perceived need to use manual coding has held back the credibility of social media monitoring, making it slower and more expensive.
4. Clients have been conservative about changing their brand and customer satisfaction tracking studies.
5. Different social media produce different results. Subtle search term changes can affect outcomes, resulting in worries about the validity of the findings. 

Social media monitoring is establishing a base within market research, but an even bigger one is outside. Most brands recognize that they need to monitor what people are saying about them, even if they can't use that process to replace traditional research.
It is likely that the cost of social media monitoring will fall as software improves and as competition between the providers increases. This will increase the use of social media monitoring tools for both research and non-research purposes.

PURPOSED COMMUNITIES 
Purposed communities solve the limitations of observing naturally occurring conversations by letting researchers create conversations. Purposed communities can be specific to market research or they can be broader brand communities, such as MyStarbucksIdea. Market research communities deliver large levels of insight but are only marginally related to marketing. Brand communities tend to focus on marketing with insight coming second.  

MROCs. Market research online communities, dubbed MROCs by Forrester Research in 2008, have been growing in impact for several years. MROCs were initially pioneered by Communispace and became mainstream by 2010. MROCs tend to be branded, are composed of customers and are either short-term or long-term.  

Long-term MROCs. Long-term MROCs tend to last more than six months, usually much longer. The size of MROCs is highly variable, with some providers favoring smaller communities of 100 to 200, while others prefer larger numbers, sometimes more than 2,000, blurring the boundaries between MROCs and community panels. Advocates of MROCs claim they deliver more research for the same budget and allow brands to get closer to customers.
European low-cost airline EasyJet and its agency Join the Dots have been running an MROC since 2008 with 2,000 customers. The MROC is highly productive. The agency and EasyJet have weekly teleconferences to review the learnings from the previous week in order to fine-tune the current week and to plan the research for the next few weeks. The community is used for a wide range of research, from concept screening, to ideation, to customer experience. Sophie Dekkers, the customer research manager at EasyJet, said of the community, "We were able to conduct more research, for more areas of the business, in a faster time frame but within the same budgetary constraints." 

 Short-term MROCs. Short-term communities typically run from about two weeks to three months, with the number of members ranging from about 25 to about 300. But there are examples that are shorter, longer, smaller and larger. However, some authorities, such as Communispace CEO Diane Hessan, argue that a short-term community is an oxymoron. One of the commercial benefits is that they tend to fit a specific client need and budget, making them direct competitors with traditional qualitative and bulletin board groups--albeit with a sense of being up to date and more in tune with the voice of the customer.  

MROCs in a business context. MROCs are qualitative in nature, even when surveys and polls are used. The special nature of the recruitment of MROCs and the sensitized nature of the participants' involvement means that the data should be interpreted in a qualitative way, rather than assuming that it directly represents the marketplace.
MROCs make greater demands on clients than traditional research, especially long-term MROCs. Although some clients run their own MROCs, most find it is best to outsource the management and engagement of their community to third parties, usually research agencies.
The use of MROCs is continuing to evolve as they become more widespread. Ethnographically inspired techniques have been in use for the past three years, in particular using smartphones to enrich the data collected. Other research companies have been developing crowd-sourced creativity and insight. 

Community panels. Private panels have existed for years, but community panels are much more than an e-version of traditional in-house panels. Community panels employ the latest techniques in engagement and communities to optimize relationships with brands.
Community panels fit between MROCs and traditional private/in-house panels. Typical community panels are large enough to support quantitative research (ranging from 5,000 to 50,000), but their sophistication and interaction separates them from traditional panels. Community panels use communities, online discussions, interactivity, engaging surveys and longitudinal analysis to create a cost-effective "voice of the customer" resource.
Community panels offer several benefits:

1. Cost: Companies report that they are achieving cost savings by using community panels, especially those with larger research spending.
2. Speed: Studies are quicker because they are not being designed and recruited from scratch.
3. Longitudinal analysis: Community panels facilitate longitudinal analysis of changes in views, behavior and preferences.
4. Consistency: Using the same sample source, research tool and design increases the consistency and comparability of the results. 

Community panels also have the following limitations:

1. They are usually branded and the members of the community know who they are dealing with.
2. Most community panels only include customers.
3. The members become more sensitized to the research and typically more likely to be brand advocates.

Community panels are tapping into several strong trends, including "voice of the customer," quicker/faster research and the idea of working with customers as opposed to just researching them. 

Brand and other communities. Just a couple of years ago, it looked as though brand communities were going to be one of the biggest things in marketing and an essential component of market research. Most conference presentations on communities talked about MyStarbucksIdea, Dell's IdeaStorm and, of course, Lego's communities. However, the growth in major brand communities seems to have stalled, reminding marketers that brands have to go where the people are; the people do not go where the brands are. Where brand communities exist, they are used for both marketing and market research, but they are not a current major factor in marketing or research.
By contrast, one area that has shown rapid development is that of branded Facebook pages. In April, Bright Edge estimated that 70 percent of the Fortune 200 brands did not have a Facebook page. Although most brands have a fairly weak presence on Facebook, some leading brands have attracted millions of followers. According to a list compiled by Socialbakers, at June 1, Coca-Cola had 29 million Facebook fans, followed by Disney with 25 million and Starbucks with 23 million.
Some brands are using Facebook as a place to recruit respondents for conventional research. Other brands engage with fans in Facebook itself. For example, in May 2011 Oreo asked, "How would you describe Oreo cookies to someone who never tasted them?" In six days they received 3,660 replies.
At the moment, natural communities such as Facebook fans and Twitter followers are much less significant in market research than MROCs and community panels, but this could change quickly.

OTHER FORMS OF SOCIAL MEDIA RESEARCH 
This third category briefly reviews the breadth and diversity that constitutes social media research. Although this group includes some innovative and interesting manifestations of social media research, it should be borne in mind that these are currently of less financial importance to the market research industry than communities and social media monitoring.  

Netnography. Robert Kozinets' highly readable book "Netnography: Doing Ethnographic Research Online" (Sage Publications, 2009) describes netnography as human-to-human investigation, conducted via social media. Kozinets excludes social media monitoring from the term netnography and makes the case for a participant-observer model, with the researcher engaging with the participants.
Netnography can be a very powerful technique in terms of obtaining insight, but it is not a scalable approach as it requires time and skill. Netnography is likely to remain a specialized niche, applied to a subset of research situations--for example, research conducted prior to creating a survey or even before submitting a research proposal.  

Smartphone-augmented research. The combination of social media and the increasingly ubiquitous smartphone is opening up a range of research approaches. Communities are tasking members to capture slices of their lives as images and videos and to upload them. Systems such as Revelation Project allow respondents to use their smartphones to engage in auto-ethnography, while other researchers have started using smartphones as links to surveys, blogs and research tasks. 
Like most mobile research, the use of smartphones in social media research has added color and definition but has not been a major player yet. The material gathered from smartphone-augmented research tends to be unstructured text, images and video, all of which can make the analysis expensive, time-consuming and hard to scale.  

Network analysis. Network analysis is based on processing large amounts of behavioral data, including website analytics, social graphs, the flow of memes in social media, geolocation data (e.g., from Foursquare) and media behavior within social media (e.g., ads viewed, videos watched and virtual sites visited). Most of the providers of network analysis come from outside the market research industry.
Research companies are linking network analysis with conventional data. For example, Gemius provide methods of tracking the online behavior of panel members, and several panel companies are tracking their members' social media interactions. But the growth of network analysis by non-research companies dwarfs the contribution of market research organizations.

Social media as a sample source. There are several approaches to using social media as a sample source. Some organizations, such as Peanut Labs (now part of Research Now), have created systems based on a wide variety of social networks, with the sample being directed toward conventional surveys. Some brands are using their social media presence to attract sample for their qualitative and quantitative studies. A number of software vendors have apps or options that allow surveys directly from their social media presences--for example, Global Park's Social Insight Connect works directly from a brand's Facebook page.  

Research into social media. Commercial research into social media tends to be focused on understanding how to use it for marketing (e.g., to generate viral campaigns, to market services in social media and to understand the nature of influence). 
One phenomenon that holds back the commissioning of more research into social media is the widespread practice of developing social media products or services by trial and error rather than by prior research.

ETHICS AND SOCIAL MEDIA RESEARCH  
Social media research has created a new range of problems for market researchers to consider. Some of these problems are traditional issues in new contexts, while others are new.  

Informed consent. In terms of social media monitoring, researchers have to consider whether they have the authors' consent to read and use posts and comments, and if they do have consent, are the authors necessarily informed about what they have consented to? The issue of consent hit the headlines in 2010 when The Wall Street Journal reported that Nielsen had been caught scraping conversations from the PatientsLikeMe community, quoting a community member who said that he/she "felt totally violated."
Communities and panels also challenge the concept of consent. In January 2011 TNS Gallup in Denmark withdrew Webprofil, a product that tracked community members; members had initially consented, but had not understood the implications.  

Anonymity. Traditional market research is based on an assumption that respondents remain anonymous. However, social media research challenges that principle. In communities there is a risk that the anonymity of participants will be compromised, in terms of researchers, clients and each other; and the risk grows over time.  
Social media monitoring tools are often capable of reporting who is saying what and identifying those people across different strands of social media. This is really useful to marketers but is in direct contravention of market research traditions. Another problem arises with the use of literal quotes mined from social media monitoring. If a literal quote is reported to the client, it will often be possible for the client to identify the respondent who made the comment by using search engines. 

The conflation of market research and marketing. Social media research tends to blur the distinction between marketing and market research. One of the by-products of people being members of well-run branded, private communities is that they tend to become brand advocates, leading to increased consumption of the brand's services or products and to positive word of mouth, which breaks market research codes.
In terms of social media monitoring, many of the tools offer the client the opportunity to drill down to specific comments and individuals. This is part of what these tools were designed to do, but it conflicts with the traditional rules of market research.  

The trustworthiness of social media findings. Traditional quantitative research has well-established methods of assessing its reliability and validity. The methods of assessing the trustworthiness of qualitative research are less precise but are well-established. By contrast, the views about social media research vary widely.
Although views about social media research exist on a continuum, it is useful to think of three positions that typify the debate:

1. Social media research is simply traditional research with a new sample source. All the traditional rules about sampling, measurement, reliability and validity apply.
2. Social media research is different because it has not established what the population is, has not created a sampling frame and has not established that measurements from it follow a normal distribution. Therefore, traditional methods of establishing reliability and validity are unproven in social media.
3. The third view attacks not the methods of research, but the subject of the research. This view holds that the comments people make online reflect constructed personas, with comments made for purposes other than the expression of "true" views. This third camp is doubtful that social media research can directly inform clients about the "real" world.

WHAT'S NEXT? 
The major trade bodies are all working on new guidelines, or have recently published new guidelines, in an attempt to bring social media research into the broader structure of market research. However, it is unlikely that most of the successful implementations of social media research will be able to meet the requirements of anonymity, scientific criteria of reliability or any clear separation from marketing. This will either result in the rules being changed or a large part of social media research being conducted outside the realm of traditional market research.
In the last 12 months, the larger agencies have started buying social media specialists, showing that social media research is reaching its tipping point. Clients are demanding cheaper and faster research, and are demanding tangible insights. Those aspects of social media research that can deliver these in scalable ways are going to expand rapidly.
Over the next 18 months, expect to see:

1. community panels becoming the default for a large part of quantitative research;
2.  social media monitoring and network analysis being dominated by non-research companies, with market research picking up the leftovers;
3. long-term MROCs becoming the default method of getting close to customers; 
4. short-term MROCs becoming a key tool for qualitative and qualitative/quantitative research projects.

Most of the other forms of social media research require too much time, skill or investment to make a major change to research, even though some companies and agencies will find them really useful and profitable.

Author: Ray Poynter 

Ray Poynter is executive vice president at Vision Critical and author of “The Handbook of Online and Social Media Research: Tools and Techniques for Market Researchers” (Wiley, 2010). He may be reached at Poynter.Ray@gmail.com.

read it and download this article here