Secrets of a Master Moderator, by Naomi Hendersen

Naomi Hendersen brings experience from moderating over 6,000 focus groups in the United States over more than thirty years starting 1978. Secrets of a Master Moderator is a collection of articles she wrote over the period to share her extensive experience.

The articles cover specifically the art of focus groups. Meaning the art of having unknown people share their perceptions, opinions, beliefs and attitudes toward a topic, while respecting their stance whatever it is.

The result is a very smart treatise about listening, a very thoughful thesis on the art of organizing focus groups, elements of what makes good moderating, and a wonderful list of tips for anyone in this business.

The below synthesis tries to reorganize the many article in the flow of a « how to » manual. As I took notes with my Amazon Kindle it is 95% copy pastes from the book.

Ready to jump in a word where what counts cannot be counted ?

Introduction

A definition of qualitative research

Qualitative research refers to a set of tools for understanding the grounds, conditions, and linkages behind people’s beliefs and behavior. Qualitative research gains from sensitive application of a certain amount of indirect questioning.

Five main categories comprise the bulk of qualitative research studies: Focus Groups, Mini-Groups, Triads, Dyads, and In-Depth Interviews (IDIs). Survey research relies heavily on closed-ended questions, while IDIs rely on open-ended questions.

Qualitative research has evolved from related disciplines of cognitive anthropology, sociology, human ethology, ecological psychology, holistic ethnography, and sociolinguistics, now follows a “standard” framework for interviewing selected target populations. Those elements include the following:

• Set timeline (e.g. two hours, or thirty minutes, or six hours)
• Trained interviewer, moderator, or researcher
• Open frame format—loose structure that allows for respondent “gold mines” to emerge
• Fixed number of respondents across a series of interview events
• Respondents share common traits In group settings, respondents typically do not know each other
• Respondents have generic study context, not specific context Respondents generally receive stipends for participation
• Research setting created in facilities or in the field to accurately document activities and, where possible, provide for observers
• Discussion has a clear purpose and stated desired outcomes
• Four stages for each research interview event
• Allowance for bias of group influence
• Allowance for bias of researcher
• Knowledge that reported behavior may differ from actual behavior

Qualitative research addresses the nature of structure, attitudes and motivations, rather than their frequency and distribution… the underlying goal is to explore, In-depth, the feelings and beliefs people hold, and to learn how those feelings shape overt behavior.

Five items need to be in place to create that environment:

• Trust between moderator and respondents
• Respect for what respondents have to say
• Steady pace to keep discussion moving along
• Variety of simple activities that keep interest level up
• Methods of asking questions that do not “lead” respondents

Qualitative research is a science

The things in life that count are things that cannot be counted. There is no way to measure the love of parent for a child or the depth of emotion when a friend stands by you in tough times. There is no way to measure the peace of standing in a field of wildflowers on a sunny day or the soothing quality of waves on a deserted beach. There is no measure of the magic of getting the perfect gift you wanted or replicating the exact taste of your grandmother’s peach pie twenty-six years after she died. How do you quantitively measure something like job satisfaction? You can ask about all those things in qualitative research and discover key threads that make up the fabric of society. The science of the heart (emotion) has just as much value as the science of the head (rational thought).

Group influence within research sessions is a valuable resource for understanding how real-world patterns of persuasion are likely to play out. There seems to be an excessive concern with so-called “dominators” and, consequently, the results of many group sessions are unfairly dismissed or discussions are too rigidly controlled to have much value. True dominators are egotistical bullies, and they differ from influencers, who, by virtue of enhanced knowledge or personal qualities, are capable of explaining their point of view to others. We learn a great deal from these influencers about arguments that may have to be advanced or defeated during product rollouts.

The way to validate qualitative research findings is by triangulating, which in a research context refers to examining the same question in a variety of ways and through a succession of techniques.

Moderating is a learnt skills

Veteran moderators have often heard respondents, back room observers, and others make this comment: “You have an easy job; all you have to do is ask some questions and sit back and let participants talk—sure wish I had a job like that!” Experienced moderators know that the easier we make it look, the more comments like that will surface and the more the truth about what moderators do will be obscured. Like the duck we see gliding across the pond with seeming ease, no one sees those feet paddling madly under the water.

Casting a spell is clearly part of the magic of moderating. A complete stranger, often from out of town, shows up in a mirrored room to talk to a group of people who do not know each other. Within two hours that stranger needs to become a “newest best friend” and is expected to create rapport, ask short questions to get detailed answers, hear from two-thirds of the room on most questions, attend to non-verbal as well as verbal communications, and stay on time. In addition, these “spell-binders” have to invite less verbose participants to chime in and suppress dominators without causing anyone to shut down. They have to handle thought leaders and “me too” types; they have to expose stimuli in a neutral way; they have to refrain from presenting their own opinions either by word (e.g. “I do that too”) or deed (expressing shock at comments made).

One of the challenges of being a moderator is that the better you are, the easier it looks to others. You would not take your car to an unqualified and untrained repair person for a brake job. You would not let a child ride in a school bus driven by an unskilled and unbonded driver. Do not give a qualitative research project to someone who is not trained in the tools, techniques, and procedures for collecting qualitative research from consumers.

You cannot learn moderating from reading a book. Moderating is an experiential event—watching is not leading, analyzing QRE research is not leading, listening to recordings is not leading. A person must actually lead QRE to understand the myriad variables present.

There is a set of skills that frames the key qualities of effective in-depth interviewers, and they include the following:

• Good interviewing skills (e.g. listening without judging, asking clear questions, etc.)
• Right mix of intelligence and good common sense
• Good voice tone, pacing, pitch, and volume
• Appropriate combination of critical reasoning and imaginative thinking
• Eye for detail and ability to hold big picture at the same time
• Able to appear genuinely interested (as a person) and truly detached (as a researcher)
• Appropriate blend of empathy and neutrality in word and deed
• Able to think analytically and live without a sense of closure

Regardless of the basis of the training, it should teach interviewers to incorporate the following factors: How to:

• Write effective questions Probe for clarity Establish and maintain authentic rapport with respondent
• Pace the interviewing session
• Use interventions and projective techniques appropriately
• Analyze data to support client objectives

Perception, Opinion, Beliefs and Attitudes.

The foundation for uncovering consumer opinions in a qualitative environment rests on four cornerstones: perceptions, opinions, beliefs, and attitudes (POBAs). Most qualitative research events (QREs) such as focus groups, focus panels, in-depth interviews, dyads, triads, minigroups, extended groups, mock juries, children’s play sessions, and more rest on these four elements.

Perception can be defined as a mental image or concept of something filtered through the experiences of a consumer. Questions that fit into this category should all address what consumers see in the physical arena and how they filter what they see through previous experiences.

Opinion is a subjective judgment or appraisal formed in the mind of the consumer and on which he or she bases his or her reasons for feeling one way or another about a product, a service, or an idea. It can be seen as more than an impression, but less than positive knowledge that can be defended. What is useful for clients is the depth of consumers’ opinions, including how long they talk about them, the words they use to defend their position, and the degree to which they work to sway the thoughts of others in the group.

Belief is a state of mind in which trust or confidence is placed in a person or a thing based on evidence. Beliefs stand at the core of one’s personality and are hard to shake since they are formed early and reinforced on a regular basis.

An attitude is the feeling or emotional or mental position that one holds about a fact or a statement. Really to delve into attitudes, moderators must engage respondents in lively conversations about the key purpose of a qualitative research study. When someone feels strongly about something, either positively or negatively, the words that he or she uses to talk about it, the pitch, speed, and pacing of their voices all give clues to their attitude toward it.

Run focus groups in four steps

Introduction. This typically includes a generic purpose statement for respondents and disclosures about the special research room (mirrors, microphones, observers, taping). During this period, respondents provide self-introductions. The moderator uses part of this time to begin creating a safe space for respondent participation by outlining guidelines for group process.

Rapport building and general questions. This section usually includes easy questions that anyone in the group can answer. Typical questions focus on category usage and knowledge. During this period, the moderator is building trust through eye contact and both verbal and nonverbal feedback. Group norms are established.

Specific questions and interventions. The lion’s share of the QRE is spent in this section. The moderator employs a variety of techniques to keep the questions on target to key issues. During this section, the moderator uses interventions (any activity that breaks up the two-way dialogue between the respondents and moderator) to understand the subtle thinking of target market respondents. Other activities can also occur including use of projective techniques, presentation of audiovisual materials, paper and pencil tasks, and/ or hands-on team tasks. Throughout this section, deep probing of comments, nominalizations, and consumer statements is conducted to further understand respondent thinking.

Closure. The summary and linking of key insights brought to light in the QRE, along with additional questions requested by the end-user.

Understand the participants

Why respondents accept to talk

It is important to realize that respondents do not share the same reason for participating as clients! Respondents do not care about line extensions, more shelf space, or the true benefits of a royal blue vs. a teal blue box for the product. They do not really care about the pricing difference between the regular and the plus-size product, and they do not really care how the new logo looks for that famous bank.

What they do care about includes making sure they have been heard, that their opinions are respected, that it is safe to say what they really think, and that they are valued as customers or potential customers. They also care about having a say in what products or services are being sold in their country.

Here are some sample responses I have gotten over the years:

• “I was curious and wanted to know what other people thought about ___________.”
• “I wanted my voice to be heard. I feel strongly about ___________ and wanted my viewpoint to be on record.”
• “I have been having problems with ___________, and I thought by coming to this group discussion, I might learn something that would help me.”
• “I thought it would be fun.”
• “I wanted to make a difference in the future of ___________. By coming to this session, I felt I might be able to shape the outcome.” “
• I heard about focus groups, and I thought it would be interesting to see research in action.”
• “I thought I might get some advance information before ___________ is seen by the general public.”

When watching focus groups, do not expect to be entertained. In fact, researchers can expect to encounter boredom, frustration, disappointment, enlightenment, confirmation, distraction, and confusion along with some really stellar insights. Finally, keep in mind that qualitative research is not rocket science. It is real people providing their points of view about products, services, ideas, concepts, or advertising. Sometimes it is funny. Sometimes it is sad. It can be tiresome, enervating, exciting, or enlivening. However the process goes, it is always about people. You have a window into the lives of respondents. Enjoy the view!

Don’t ask people what they want

People are notoriously poor at self-reflection. We have too many defense mechanisms that keep us ignorant of our own motivations and out of touch with our true feelings. We often seek the solace of an authority figure—“ the cool kids” or “the thought leader.” We tend toward defensiveness and project our own negative thoughts onto others.

Most of our decision making is emotionally based. Consumers are primarily driven by fear, passion, vanity, shame, and similar feeling states rather than rational arguments about product superiority. Qualitative research is a strong resource for sensing how people’s emotional drivers align with assertions about product benefits and characteristics.

Consequently, asking consumers direct questions about such issues as what products they need is a notoriously unreliable business that often prompts responses that are trivial and unproductive.

Similarly, asking respondents for an instant evaluation of how much they like or dislike something and counting the “yeas” and “nays” is unlikely to yield very useful or valid results.

Consumers are inherently conservative and skeptical about new ideas. Upon first hearing they tend to be disdainful and unconvinced by persuasive communications. Most new product concepts leave them cold. Special qualitative research procedures are necessary to go beyond this initial hesitation.

Respect respondents

Both moderators and clients should respect respondents and the information they bring along with their perceptions, opinions, beliefs, and attitudes (POBAs). And both should honor respondents who have come alone to a research facility to meet with a group of strangers for a stipend to discuss a topic only vaguely outlined on the phone. Unfortunately, disrespect is a common mindset, a kind of gallows humor found among recruiters, observer/ clients, and sometimes even among some moderators.

If the group is a “dud”, look to yourself first! Did you get the right people in the room? Was the screener correct? Did you monitor recruitment? Was the purpose clear? Were the questions clear and answerable? Did you establish and maintain rapport? Did you do your best to facilitate discussion and probe effectively? Did you demonstrate active listening? Did you brief your clients on the process of observing a QRE?

In qualitative research, what counts is what cannot be counted, and that means asking questions that access feelings. When respondents invite you to visit them in their worlds and allow you to probe that area below top-of-mind, it is important to respect the privilege given and to honor that invitation with dignity.

Prepare a focus group

Keep the agenda manageable

It is the job of the moderator to be the captain of the research process, directing resources where they can do the most good.

The right number of “things” to test [concepts, prototypes, advertisement strategies, or ideas, etc.], should not exceed six. If more are desired, then it is best to pick a core set of four that are seen by every group and rotate in a different satellite set in varying groups.

Manage stakeholders

When there are too many “research partners” on a project, some should be limited to the true role of “observers”—adding nothing to process other than just watching—they do not contribute any questions for the guide and they do not send any questions into the research room during the process–they just watch.

On a practical level, I have learned to appoint a back room spokesperson (also known as a single point of contact or SPOC) who collects observers’ questions. This spokesperson is the only one talking to the moderator.

I also ask that no written questions are sent into the room until at least the forty minute mark, so that I have created deep rapport with respondents before having any interruptions.

Ask clients what they want

There are many pratical questions to cover during a preparation discussion : city sites, regional issues, recruiting specs, budgets, timelines, number of observers, stipends, prototypes, concept/ position statements, travel policies, and so on.

Make sure to cover a few additional questions to ensure critical time is spent on getting the key data wanted, not just some answers to some questions.

• If the session only lasted one minute, what must you know from respondents? what key insights would you want to take away?
• What baseline assumptions or hypotheses are already in place?
• What do you not want to hear about in the sessions—what data is of little interest to the team?
• What is going to happen after the study is completed—how will the data be used?
• What might change as a result of conducting the QRE project?

Understand the culture of the group

A good moderator should be able to lead an effective group regardless of culture, ethnicity, or age. However, there are some cases when matching will increase the likelihood of more than top-of-mind responses.

Multiple skills to handle diverse opinions without becoming judgmental, evaluative, or threatening. Working with hostile or diffident respondents. Handling dominators/“ milquetoasts”. Suppressing judgments. Avoiding preconceptions. Avoiding “finger-pointing” voice tone.

Write a good interview guide

Prepare the topic outline guide. It should contain:

• Introduction that covers purpose of the session and ground rules.
• Introduction of moderator to the group and group members to one another.
• Non-threatening questions asked first to establish rapport.
• Content questions on issues to be discussed, with notes to the moderator regarding cues or areas where probing may be needed. (It may be helpful to make notes on intent of some questions to aid in eliciting desired responses.) e)
• Stimuli, prototypes, samples, etc. for “show and tell.”

Write the guide to flow from general to specific. It should contain clear, logical questions that the respondents naturally want to answer. Imagine a funnel. Early questions are broad and general as the questions move down the funnel toward the narrow end, they become more focused and specific.

Try to favor the “universal guide” approach. This is a complete guide that any interviewer can pick up, so that all the hard thinking and planning are done in a non-sweaty place without hot lights. That way, when the lights are on, the microphones are running, and the client is peering intently through the mirror, the questions roll out easily.

When I write the guide for a qualitative research session, it seems to work best to enter the conversation where the respondent is—not where the client is. Starting in the respondent’s world makes more sense to them and creates a sense of safety. Later, when I invite them to come over to my world, they hardly notice the bridge they have to walk over. Ever since that early study, I have written questions from the respondent point of view by asking myself, what would respondents naturally want to talk about first on the topic of “X”?

Practice and be flexible

Practice Makes Perfect It is essential that you practice your questions aloud. Reading and speaking are two completely different activities. Questions are almost never asked exactly as they are written in the guide because the group experience is a dance of “I ask/ you answer,” and that is done aloud.

The last part of my plan is to resist the desire to whine and complain about how much uncertainty or requests for changes are present in the study. Instead I embrace the positive dynamism of having a chance to test materials that will be seen by millions of consumers.

Get away from the guide

When you are in the room with respondents, at least fifty different activities may be going on that requires your attention. Chief among them are the following:

• Establishing rapport Maintaining rapport
• Maintaining eye contact
• Reading the room
• Using active listening skills
• Probing for clarity and clarification
• Watching the time/ managing timelines for sections
• Making sure you hear from quiet respondents
• Managing dominant respondents
• Setting up/ showing interventions
• Remembering what was said by using linking and logic tracking techniques
• Varying tasks every twenty minutes
• Managing the climate of the room Making sure questions are on target
• Practicing unconditional positive regard (UPR) for respondents
• Watching process and content patterns
• Managing own body needs (e.g. enough water)
• Allowing and living with silence

If your head is stuck in the guide, you cannot successful moderate or manage group dynamics. You must be sufficiently familiar with the guide to have no need to read it word for word. Once you have started the group, the guide should function as a road map in the same manner as it does for a driver. A glance will give you directions, but you have to look up in order to drive.

Create trust

Rapport is the key to the game—if you take time to make that relationship with respondents—they will just about tell you anything you want to know. Rapport can take a few seconds or a few minutes, and if you do not take the time to create it, the quality of the comments and interactions is reduced. Without strong rapport, the research interview can often be just a long series of boring ‘I ask/ you answer’ events.

Once created, the focus group reaches an unspoken agreement to engage in conversation. The moderator has created a safe space for respondents to answer a wide variety of questions and has demonstrated a nonjudgmental listening style, and respondents step up to answer because they know that they will be listened to and respected.

The trick is making the group look like a two-way conversation when it is really an elaborate, organized event that guides respondents into a discussion of perceptions, opinions, beliefs, and attitudes (POBAs). Ultimately, a moderator should be able to create an environment, a mood, or an atmosphere and shift it when necessary.

How to create trust with participants

Levels of trust begin the moment the interviewer sees a participant. That first moment of eye contact made by the interviewer should be accompanied by a smile and a sense of welcoming by word and deed. Rapport deepens in the first stages of the interview event when these activities take place: State clear purpose Provide adequate disclosures about microphones, mirrors, observers Provide key ground rules for participation Ask easy opening questions

While many observers and some qualitative researchers often make fun of respondents, an attitude of respect is a coin that doubles and triples the investment made. It takes a respondent some measure of courage to come out, alone, to a research session in which questions are asked that reveal motivations and beliefs! A good qualitative researcher knows that when true respect is present throughout the interviewing event, the amount of data increases, and the depth of that information is often deep and rich. When these classic techniques are in place, the information pool is as large as a lake: Honor the world where the respondent stands Listen actively Suspend judgment

The “I ask—you answer” model of qualitative interviewing is interesting for about five minutes, and after that it is more interesting to watch paint dry!

Provide a Variety of Simple Activities to Keep Interest High Similar to the pacing activities in the preceding paragraph, the inclusion of very simple activities can keep interest high and forward client learning. The following are just a few:

• Ask short questions to get long answers
• Stand and ask questions from different areas in the room from time to time so authority is not rooted in the researcher’s chair
• Ask questions that access different models of listening Provide written and spoken instructions for all activities
• Find alternatives to charting responses on an easel
• Use manipulatives to forward discussion (e.g. file cards, product sorts, worksheets, etc.)
• Make abstract content more concrete through the use of easel drawings
• Use projective techniques that allow quick access to deeper thinking

Praise the first person to respond. And speak to respondents as life peers, not as a function of their education or ability level.

Stage your room entry

There are differing opinions on whether the moderator should already be in the room to greet the respondents, walk in with them, or walk in after they are seated. Each choice has its advantages and is generally a matter of personal style, although each sets up a slightly different kind of rapport:

• Being in the room first allows me to welcome respondents into the space and establishes the fact that it is a group discussion. The chitchat around the last football game for their home team or the sudden weather shift as they are taking seats contributes to rapport building. It has the same tone and mood as those times when I welcome guests to my home for dinner.
• Walking in with the respondents can put me into physical relationship with them and gets them thinking of me more as a co-participant than as an expert, a teacher, or a host.
• Walking in after they are seated tends to set me up in the role of leader or expert. When I enter like that, I sometimes look like an actress coming in to start the action in a play, and that is a different kind of rapport with the “audience.”

Tips to build rapport quickly

Some Tools, Tips, and Techniques

• Smile as soon as you see respondents, and be sure it is genuine.
• During ground rules and each respondent self-introduction, make eye contact lasting at least twenty seconds with each participant to form a bond or connection in a group. In those twenty seconds, no one is in the room except the moderator and that one respondent. Group reactions and interactions will come later in the interview.
• Imagine that you are making a physical connection with each participant during introductions similar to a handshake when you meet someone. As you greet them and say hello, picture handing them one end of a silk thread while you hold all the other ends in your hand.
• Asking a follow-up question to their introduction or referencing a similarity in your life (e.g. “I have a dog too”) is when they accept their end of the silken thread, via dialogue.
• During the session, keep checking the threads to make sure none have gone slack. At the end, picture taking the threads back for closure.
• Make an extra effort to respond warmly to each respondent when round-robin introductions are made. Make comments such as: “Welcome!”; “Glad you could make it!”; “Thank you for coming!”. This reinforces the value of their presence in a group.
• Think of the conference table as a clock with one person at approximately each hour. Make sure to check in regularly with the hours, to “sweep” the room like a second hand in a group.
• In an IDI, think of the respondent as your “newest best friend” that you are going to enjoy talking to and unpeeling the layers of very interesting onions.
• If some participants have problems or concerns as they arrive at the group—such as parking, worries about getting out later than told, a headache, worries about the session and their role, or a soda spilled on a dress—address these issues, either on your own or by enlisting facility staff help, before starting the group itself. People will appreciate your concern for them and on behalf of others and will reciprocate by doing their best to help you, creating powerful rapport.
• In an IDI, judge the situation and see if this question can be asked: “Is there anything that would keep you from completing this process in the next minutes.” If their answer is “yes,” set up a time for another interview. If no, their response to your question is the deepest level of commitment they can make.
• Use the time with them well! Include logistics with your introductory statements. Mention food and drink, where bathrooms are located, and ask if everyone is comfortable with the temperature. All these add to the group’s impression of you as a caring person, not just a taker of information.
• In an IDI ask questions like these to build rapport: “I can hear you clearly, can you hear me?”; “Do you want water or something to sip on in case you get thirsty?”

Strong rapport is not fragile. In any qualitative session, even the most experienced moderator makes some kind of mistake that briefly breaks rapport with an individual or the group. Effective rapport building early in the session will ensure that the uncomfortable moment passes quickly and is both forgiven by the individual or group and forgotten.

Make sure to come to complete closure at the end of your qualitative session. Even if the comments are brief, make them final. There are times when rapport is so strong that the group does not want to leave, or individuals want to stay in touch beyond the research setting. Learn to say a firm goodbye. Do not leave people hanging. A good way to bring closure in a focus group is to stand up, go to the room exit, and say, “I’ll shake your hand on your way out.” In an IDI, the best way may be to say, “I have asked all my questions, and I see our time is about up. I am going to walk you out and say goodbye unless there is something more you want to say.” Find a way to be fully sincere in the actions suggested. If you remain authentic and appropriate without phoniness or lies, you set the example and imply that the same behavior is desired from the respondents. I have learned that it is during the first three minutes of a group that participants form their first opinions about me and that is where the platform for rapport is built.

Remove your ego

Hold no personal opinions in the process of facilitating conversation and gathering data. There is no room for your opinions and attitudes during the two-hour group. At the same time, you must cultivate a healthy ego and hold the courage of your own convictions about doing what is right—not what is easy.

Maintain your researcher neutrality. You are the neutral party, gathering information from those around you. You naturally have opinions on everything: the observers/ clients, the facility, the respondents, the product or service, and the project. It is essential that you prevent your personal value judgments from entering into the data gathering process. You should be the model of objectivity and have a genuine interest in the respondents’ point of view.

Your role as a moderator is not to defend a product, service, or concept. Nor is it to educate or evaluate. Your role is to find out how respondents feel and what their opinion is. They may say some thing you know is totally erroneous. Your role is not to correct but to explore why they feel the way they do. These phrases may be helpful: a) “How did you get to that viewpoint?” b) “I hear you—can anybody support that comment?”

Start with easy questions

Make sure the first question is an easy one. This gives them time to talk aloud and hear their own voice in the room, and it allows the moderator to demonstrate listening. Questions that seem to work best include the following: “Tell me about…” “What is it like to…” “What have you seen, heard or been told about…” “When was the last time you did XYZ? What was that experience like?” “What are you looking forward to next year, etc…” These questions open the door to an interchange that gets the interview off to an involved start.

Listen to answers

A skilled moderator asks questions in a way that lets respondents tell her what she might already know—and she never reveals how much she knows. That means packing the ego in a metaphorical suitcase, and checking it at the door.

Trust is created when these factors are in place in a group discussion:

• Respondents are listened to, but not judged.
• All respondents are treated equally by the moderator.
• The moderator does not allow judgmental comments to be made by one respondent to another.
• From time to time, respondents are acknowledged for contributing with a “thanks,” “I see,” or “interesting, I never thought of it that way.”

Create the right atmosphere

Your goal is to obtain answers to questions and issues on your discussion outline in an atmosphere that is: Spontaneous Non-evaluative Non-threatening You have a larger goal in qualitative research, and that is: question the answers—

Be sure to tell the respondents something personal about yourself—become as much like a respondent as your role will allow;

Create a group discussion

The hallmark of a good focus group is respondents regularly talking to each other and not just responding to the moderator.

If individuals direct all their comments to the moderator rather than to one another, a focus group is not taking place. This is serial interviewing. The most difficult part of being a moderator is getting out of the role of leader (which is where respondents want you) and into the role of question-asker.

Invite participants to address each other rather than you.

 

Run the focus group

Make yourself available before the start

Do not bring mental baggage into the research room with you. Avoid rushing from office or client meetings or phone calls to the research room without taking a few minutes of mental preparation. Respondents will notice your stress and focus on it. A quick way to transform that kind of stress into a more peaceful attitude is to take three deep breaths, followed by actively dropping shoulders and humming for thirty seconds. Like magic, you are in a new zone.

Put other projects on hold. It may be difficult to focus when you have several projects running simultaneously. Seek ways to minimize, if not eliminate, phone calls from your office or from other clients during the time allotted to briefing for and conducting groups. Just because you have voice mail, texting, and instant messaging options does not mean you have to attend to them in the moments before a QRE.

Practice Unconditional Positive Regard

Mastery of UPR. “Unconditional positive regard” (UPR) is the ability to listen to viewpoints that are widely divergent from the thinking of the moderator and to hold these viewpoints as valid and appropriate because they comprise the belief system of the respondent. UPR is also the ability to let respondents “be exactly the way they are” and to avoid the desire to educate or inform them when they speak in error.

Lead and pace the group like a therapist

In a therapeutic model, a good therapist lets the patient set the tone or pace for the session’s first part by asking a question such as, “How’s it going since we last met?” The patient begins to talk, establishing the pace for the amount and type of information revealed. After a short time, the therapist subtly directs the conversation—via a line of questions—so that the therapist is in control, leading the patient to talk about other areas. (This is not always visible to the patient.) Pacing and leading is the primary tool for advancing the session and resolving the patient’s long-term problems.

In the early part of any pacing and leading incident, the moderator arranges the environment to let respondents make the first of many wide-ranging comments, and then settles on one to lead the conversation where she wants it to go.

I firmly believe that pacing and leading works best when the moderator has no “ego investment” in looking good or serving as entertainment for the back room.

Manage the introduction

Success means investing more in conducting good research than in looking good. A good opening could be: “Hi, I am interested in hearing your thoughts on our topic tonight. I am good at asking questions but I do not know much about how XYZ works in your world. I am going to need your help to understand how you feel about tonight’s topic, so I can accurately present your views to decision makers.” This opening asks for assistance and honors the group’s ability to be smart about what it knows.

Provide ground rules

This is a time for the moderator to calibrate the tone of the group by determining quiet, shy types versus those who talk more. It is also where the ground rules for participation are given.

Basic ground rules/ guidelines need to cover these requirements: Speak up. Speak one at a time. Honor the opinions of others. Excuse self (one at a time) for the restroom or for additional refreshments. Have the courage of one’s convictions. Allow for equal air time for each participant. Avoid side conversations.

The best advice I ever received about delivering ground rules/ guidelines is to make them sound fresh and new each time and to keep them short.

I have found it helpful to also pass out the ground rules as a handout so that when one is violated, I can simply say: “Remember guideline number _____?” Then respondents check their list and comply quickly.

Keep one discussion at a time

The following can be helpful phrases: i. “Ben has the floor.” ii. “One at a time, please. John, would you repeat your comment?” iii. “Jerry, hold that thought. Sam, can you finish what you were saying?”

Get a group back on track

People will wander off the topic track. When that happens, try the following: Hold up your hands and say: “Wait, how does that relate to _________?” Say: “Interesting point. But how about _________?” Say: “That’s a side issue. Let’s get back to _________”

Come down hard when discussion gets out of hand or off track, without losing group affinity. For example, uses verbal cues (“ How does that relate to…”) and non-verbal cues (“ stop sign” hand gesture). Is willing to eject a respondent (if need be) so that others can provide the needed information.

Make sure everybody talks

Encourage shy respondents to speak by calling on them by name and asking: “What do you think, Gary?” “Has that ever happened to you?” “What do you do, Martin, when happens?”

Neutralize participants who take too much space

Watch out for:

• The expert—he intimidates others
• The loud talker—he is garrulous, but his points are not relevant
• The pseudo expert—he wants others to believe he knows it all

Reflect questions back to the group

When you are asked for your ideas or views by a respondent, remember that you are not there to educate or inform. The client wants respondent views, not yours! Direct the question back to the group by saying: “What do you think?” “How did you come to feel that way?” “What would you do?” “What’s your hunch?” Avoid alienating respondents. Use some humor. “Hey, I am not getting paid for my opinion, just to ask the questions!”

“Well, (moderator’s name), what is your opinion about making dog food green in color?” The moderator deftly turns that question into another one and shoots it back to the respondent so that nothing sticks to the moderator causing them to have to provide an answer that would add to the data pool in the room. “Good question, Jack… I do have some opinions about green dog food but I would rather hear yours, since you have a dog and I do not—what do you think about your dog eating dog food that is green because of the algae added to increase nutritional value?”

Prevent biases

Rotate the order of stimuli shown across a series of groups to prevent first order bias. The simple act of passing out papers, using different colored markers to highlight elements of a concept statement, and using picture or product sorts to get more information are all illusions to help people project feelings outward to reveal innermost thinking.

Remain cognizant of order bias.

Another technique is to make sure that, where possible, each person has a copy of what is being evaluated—the concept, a mini-version of the storyboard, or the tag line. When respondents hold the item under discussion in their hands, they become invested in it, and the conversation about it is richer.

Be the time keeper

Take off your watch, and put it beside the guide, so you can see the face unobtrusively.

The best discussion usually comes in the last fifteen minutes of the session. A good group discussion usually runs out of time. Keep the recording going even as the sessions break up. People tend to say things to you that may not have been said in front of others. Sometimes it is a good idea to terminate “early” to force discussion. (This is called a “false close.”)

Hold your judgement

Do not get defensive—do not inform or educate. Do not be an expert—let them tell you what you already know.

Manage time

Keeping a little clock to see the movement of time and pre-thinking the timing for each portion of the guide allows calibration in an effort to avoid rushing the respondent through the process.

Make respondents research partners

When a respondent is made to feel like a research partner rather than a research subject, the benefit is a deeper level of communication and more fuel for the research furnace.

Make mistakes

Nobody runs a group perfectly. Some are better than others (this includes moderators and groups!). Basically, once you understand the dynamics, it is a matter of practice. Fight the fear of making mistakes. You will make some. Laugh at yourself. It is all just words.

Ask great questions

A true question is one to which you do not already know the answer. All internal dialogue humans have in their heads is in Q& A form. Questions reveal something about the person who is asking them. The one who asks questions in a dialogue is holding all the power. And they meet the SQLA rule: ask short questions to get long answers.

Prepare a set of good questions

Asking questions people will want to answer is a hard job. Here are a few rules.

• Write true questions—ones to which the moderator does not know the answer—let respondents take the moderator into their universe.
• Write questions that do not put respondents on the defensive—but let them answer fully and freely.
• Write probing questions that allow respondents to open up. Know that “what else?” is a probe to get divergent answers, while “anything else?” is a probe that shuts down a line of answers in a group discussion.
• Write a variety of probes—not just the same two or three standard ones that start with “Tell me more about that.”
• Write short questions that promote long answers (SQLA)
• Write questions that do not have part of the answer in the question (POAIQ). (For example: “What are some reasons you grocery shop after 11: 00 p.m.?—Is it because the store is less crowded or because they are restocking items?”—The moderator should have stopped the question at 11: 00 p.m. since what follows helps the respondent and shuts down their ability to express their own self-generated reasons).
• Write questions that flow logically along the path of how consumers think—not how researchers or clients think—that means sometimes asking context type questions to set an environment before diving into key issues.
• Write questions that create “rolls”—the group talking across the table to each other (one at a time) discussing the topic—not serial questions where the moderator asks and a respondent tells the moderator what he or she thinks, over and over in random order.

Avoid leading questions

The moderator’s role is to present questions that let respondents provide perceptions, opinions, beliefs, and attitudes (POBAs) that they generate from their own experiences. If a moderator asks a leading question, these are the possible risks:

• Respondents will step out of personal experience and present something that is rational and may not be true.
• Respondents will go for the easy “yes/ no” response, hoping the moderator will not probe for more information.
• The moderator is showing—by word and deed—that they do not trust respondents to share fully—the moderator feels he or she has to “help” respondents to the “right” answer—the one the moderator is looking for!
• The moderator wants to control the flow of information to fit an assumption base that he or she has already mentally created.

Leading questions can result in short answers. The goal of a focus group is a spirited detailed discussion on the key points of interest to the client. All leading questions are best avoided. Here is a list.

Assumptive Questions. Leading questions can use the “Assumption Principle.” For example, take this question: “Do you think prices will go up next year?” This question leads the respondent towards the idea of prices going up—if they answer “no” then this may mean they believe prices will remain stable—however, the thought of prices going down may not have occurred to them. Better question: “Where are you about prices for goods and services next year compared to this year?”

Linked Statements. You can lead questions (using the “Association Principle”) merely by comments you made previously and are still top-of-mind for the person being questioned (these comments will linger longer, if you put emotion into them). For example: “I really hate this government!… What are your thoughts about the XX party?” As well, insidious thoughts can be lodged within the question: “What do you think about John Richards? Many people are opposed to him, by the way.”

Implication Questions. Asking questions that get respondents to think of consequences or implications of current or past events links the past with the future in an inescapable chain of cause-and-effect. For example: “If you go to the party tonight, what will happen when you take your examination tomorrow?”

Asking for Agreement. A very direct leading question is one that is closed ended, clearly asking for agreement, making it easier for the other person to say “yes” than “no.” For example: “Do you agree that we need to save the whales?”

Tag Questions. Tag questions are short questions that are tagged onto the end of statements. These questions effectively disguise a command to make it look like a question. These are short phrases and often include a negative element such as “Isn’t it?”; “Don’t you?”; “Aren’t you?”; “That’s a good thing to do, isn’t it?”; or “You’ll come to dinner tonight, won’t you?”

Coercive Questions. Questions that force specific answers can include implicit or explicit coercion. Thus: “You are coming tonight, aren’t you? If you aren’t, then there will be trouble” or “You do love me, don’t you?”

Avoid asking why

Even though qualitative research is always trying to understand the “why” behind people’s choices, it is not advisable to ask that question directly. “Why” evokes the defense “because,” which does not tell us very much.

“Why” questions force short, rational answers that require further probes. Those secondary probes break up the internal dialogue the moderator wants to elicit without respondents editing themselves. If “why” had been asked differently, the probe would not have been needed and the respondent would have gone beyond the top-of-mind level more quickly. Q: “What are some factors leading you to drive a stick shift car?” A: “I have got more control. It lets me feel the road. I am more involved in the act of driving. It makes me feel cool, and it gives me options in bad weather.”

The secret is in the use/ non-use of the word “why.” “Why” questions invite rational, not behavioral, answers, ones that often begin the response with “because.” A question that starts “What is the role of…” allows respondents to enter the “answer arena” from a number of different directions.

Ask better questions than why

• Comprehension or Interpretation. What does the term outpatient mean to you? How do you refer to yourself—as a patient or as an outpatient?
• Paraphrasing. Can you repeat the question I just asked you in your own words?
• Confidence or Judgment. How sure are you that your health insurance covers drug as well as alcohol treatment?
• Recall. How many times did you brush your teeth yesterday? Follow-up probe: What were the reasons for each brushing? How many times did you go to the bank to make a deposit in person in June? Is June typical or not? If so, in what ways; if not, what are the reasons?
• Accessing self talk. You hesitated before answering… what was going on in your head in the pause? “Think back to the last time you went to Home Depot. Think about your state of mind and the reasons you were there. What can you tell me about the experience?”
• Accessing values/ beliefs. What makes you think that cancer is America’s most serious health problem?

Probe well after questions

Reactive probes are almost like natural probes, except they are more “knee-jerk” in character. The most common ones are: “What makes you say that?” Or: “What is the basis for that belief on your part?”

There are seven distinctions between probing strategies. They are listed here with some examples of the phrasing used, when they are chosen:

• Request for elaboration: “Tell me more about that.”; “Give me an example of…” Request for definition: “What do you mean by…?”; “What does the term_________mean to you?”
• Request for word associations: “What other word( s) do you link with_______________”; “Give me synonyms that also describe______.”
• Request for clarification: “How does that differ from…?”; “In what circumstances do you…?”
• Request for comparison: “How is _______similar to ___________?”; “Which costs more, X or Y?”
• Request for classification: “Where does fit?”; “What else is in the category of?”
• “Silent” probe: a non-verbal gesture characterized by such actions as raised eyebrows or hand gestures such as moving the right hand in a rolling motion that signifies “Tell me more.”

Manage silence

Good moderators have learned techniques for managing the silence so it becomes a powerful tool. Here are a few different ways to manage that silence, borrowed from RIVA Training Institute classes: Slowly count to ten before asking the question again or reframing the question on the table. After the count of ten, ask: “What were you saying to yourself as you thought about my question?” Throw the question out to the room like a grenade and metaphorically expect the shrapnel to hit a respondent, and focus your eyes on someone who has been talkative in the past as an inspiration to get them to answer. After waiting, ask an incomplete question: “And the answer is…” and smile. After the ten counts are up, make a fun statement such as: “Well… do not all talk at once,” and smile at someone to encourage a response. Learn from experience that the respondents are more uncomfortable with the silence than the moderator, and someone usually gets the ball rolling, giving others permission to piggyback.

Ask questions in line with the stage

When the moderator moves into the rapport-building stage, questions in this session should be easy to answer and allow respondents a chance to flex their answering muscles. “What factors make you keep a catalog to look at again later?” “Catalogs come in small formats (such as the size of ‘Reader’s Digest’) or larger (the size of a ‘Time magazine’). Which size is your preference?”

When the session moves into the in-depth investigation stage, the questions tend to become more precise and specific. In this stage, each question should clearly support the study objectives and ideally build on the other questions. “What items are missing from the catalogs you like?” “What do you think of this new fold-out page format?”

Questions for the closure stage are typically general in nature and are meant to close down the conversation: “What advice would you give companies who regularly send catalogs to consumers?” “What insights about catalog shopping are you taking away from the discussion today?”

Poor questions exact a price, sometimes a very dear one, and the research can suffer in a number of ways: Study objectives are not realized. Respondents do not have enough opportunities to deliver perceptions, opinions, beliefs, and attitudes. Respondents get bored. Respondents talk, but do not really answer the questions. Client sends in lots of notes in an attempt to focus the lines of questions. Moderator under a lot of stress and must “pull teeth” to get data out of respondents. Clients feel their needs are not being met. Qualitative research gets a bad name, pushing clients to rely only on quantitative measures.

Improve your questions:

Consider the example below. Think about your own questions and try to improve them in the same way.

Poor Q: You like sports utility vehicles, right?

Better Q: What do you like about SUVs?

Poor Q: Why do you grocery shop on the way home? Is that because it is convenient?

Better Q: When do you usually grocery shop, and what are some reasons for doing so?

Poor ethnographic question: “What do you like about your car?” o Better ethnographic question: “If you sold your car, what features or qualities would you miss most and for what reasons?”

Poor Q : “Why do you shop from catalogs?”

Better Q : “What role do catalogs and catalog shopping play in your life?”

Write the report

Focus groups provide insight. Thus, the ideal report has no numbers except pagination. The ideal report aknwoledges the fact that the focus group was biased: people come because they want to. They do not necessarily represent the target population. Thus, do not generalize to any population group.

Respect the point of view of respondents

The first rule is to not change that information or tell respondents that they are incorrect.

The second rule is this: if a respondent feels that something is true, then in their universe it is true, even if the moderator and the observers know that it is pure baloney. It is imperative that the report writer communicate this baloney in the context in which it occurred, indicating the depth of feeling about the POBAs expressed and the degree to which there was agreement or support for the viewpoint in the focus group, because that is the nature of qualitative data collection.

The analyst does not have to believe, like, or agree with what was said—he or she just needs to report it in context.

Run a debrief session at the end of the day

The post-focus group session occurs at the end of each day the project is on the road and for certain on the last night in the last city. A good moderator has been listening intently throughout the focus group sessions and answering these internal questions along the way: Do I have enough data to analyze this topic area for this group in the report? Have I heard from at least two-thirds of the room on this topic? Are the answers I am getting helping to fulfill the study purpose? Did I lead respondents to what I wanted to hear or did I let them take me to their world?

Ideally, at the end of the day (and the end of the project), pull the clients from the back room into the focus group room and conduct the post-focus group, asking them these key questions: What did you hear tonight that was an “a-ha!” or “surprise”—something you did not expect? What did you hear tonight that was a confirmation—you expected to hear it and you did? What new thoughts were generated by observing tonight’s groups (and at the end of the project)—across all groups? In what ways, if at all, did you hear any opportunity to meet new needs? How will the insights from these qualitative sessions fit into next phase of strategy planning?

What needs to be in the report

Twelve Key Factors in good reports.

• Clear purpose statement
• Clear understanding of what client wants in the report
• Analysis begins when moderator guide is written
• Data reduction permits streamlined reports
• A clear “analyst stance”—what do I believe is true?
• Judicious use of verbatim comments
• All key tools needed for analysis (notes, guides, stimuli, transcripts, outlines, etc.)
• Ability to organize disparate information into categories for analysis
• Able to detach self from findings and report only outcomes
• Good baseline writing skills and ability to turn dry data into readable engaging reports
• Willingness to accept client input for changes/ corrections
• Nurturing a positive attitude toward analysis and report writing

Appendix – Checklists

Appendix – Uses of Focus Groups

Most group requests cover one or more of these areas:

• New product prototype testing
• Package test
• Advertising strategy exploration
• Advertising copy formulation
• Testing the market place for product acceptance
• Taste test
• Idea generation
• Storyboard tests
• Ad-labs
• Pre-quantitative issue and language identification
• Membership issues

Given the limitations outlined in the last section, what would make anyone want to spend money on this kind of research? Some of the major reasons include:

• Short timelines—and the need for some research
• To test ideas and concepts in the decision stage
• To serve as a starting point—to generate hypotheses when none are known
• To support other research and confirm hypotheses
• To serve as a “disaster” check when all other research is over
• For security: when a client desires the limited exposure of a new idea
• To allow for client observation of real consumers grappling with issues, products, concepts, services, ideas, etc.
• Because what counts in life cannot be counted

Appendix – List of tools for focus groups

The following activities can be sorted into two categories: visual interventions and process interventions. An “intervention” is any activity that interrupts the “I ask—you answer” model of interviewing.

• Product obituaries or eulogies
• Forecasting
• Board of Directors
• Sentence completions
• Role play exercises
• Picture sorts
• Product sorts
• Collages
• Custom worksheets
• Debate teams
• Writing stories
• Drawing exercises
• “What if” scenarios
• Let’s pretend
• Build you own
• Product transformations
• Comparisons
• Associations
• Secret pooling
• Personifications
• Balloon drawings
• Easel drawings
• Show/ tell items (e.g. concepts)

Appendix – Alternative types of groups

• Known Pairs Research (husband and wife, father and son)
• “field trips” during the research period (going to the actual shop)
• Piggyback Groups This model is when one group of respondents watches another group and then the groups are interviewed in turn.
• Mock Juries In this QLMR process, two sets of “jurors” are convened and each hears a different strategy on the same case points. Then each deliberates, allowing lawyers to be “flies on the wall” in the jury deliberation process to see how case points and strategies affect the final votes.

Appendix– The 20 skills of a good moderator

By answering the following questions, an observer can determine whether or not a moderator is skilled. For most questions, the right answer is “yes,” but for questions 6 and 9 (bolded), the correct answer is “no.”

1. Flow from point to point without abrupt shifts?

 2. Keep the discussion on topic?

3. Probe for clarity?

4. Maintain UPR?

5. Ask questions that open up respondents so they can give full answers?

6. Lead the respondents, put words in their mouths, or inappropriately summarize/ paraphrase?

7. Establish and maintain rapport?

8. Include everyone in the discussion?

9. Conduct serial interviewing?

10. Read the room, stay with the respondents, and keep attention off of self and the guide?

11. Keep self/ ego out of the discussion, and avoid talking too much?

12. Attend to nonverbal communications?

13. Give clear ground rules/ purpose statement/ full disclosure about mirrors/ microphones/ observers/ stipends?

14. Use a variety of research tools/ techniques?

15. Pace respondents?

16. Provide linking and logic tracking for respondents and observers?

17. Listen rather than inform participants?

18. Vary voice tone during process?

19. Change location/ body position during QREs process?

20. Give clear instructions/ directions to respondents/ set up and appropriately introduce stimuli?

Appendix – Skills of a goof moderator

What to look for on a researcher DVD:

• Respect for respondents
• Ownership of the room–clear demonstration of “invisible leadership”
• Speaks clearly and loudly
• Sets expectations for the group and gives all the industry disclosures
• Asks short questions and actively listens
• Moves things along without rushing and handles tangents adroitly
• Avoids “serial interviewing”
• Shows creativity and adaptability in the moment–varies the “I ask–you answer” model
• Changes activities about every twenty minutes
• Moves around from time to time–does not stay glued to the chair
• Maintains an open body position–avoids clasped hands and folded arms
• Moves from general to specific questions within a topic area
• Creates safe opportunity for diverse opinions
• Works along a logical path showing planning of questions and does not appear to be “winging it” with most questions or slavishly following a “script”
• Handles thought leaders and shy respondents with ease so both can contribute
• Misses no opportunity to probe for additional information.

Appendix – A checklist for moderator

If the end-user is not getting the quality of data desired, the questions listed below may help determine if the problem is moderator driven or due to some other factor. Did the moderator:

• Give clear ground rules/ purpose statement/ full disclosure about mirrors/ microphones/ observers/ stipends ?
• Establish and maintain rapport, and create a safe place for respondents to share POBAs?
• Flow from point to point without abrupt shifts?
• Keep the discussion on topic and moving along?
• Probe for clarity?
• Maintain UPR?
• Ask questions that open up respondents so they can give full answers?
• Avoid leading the respondents (i.e. putting words in their mouths or inappropriately summarize/ paraphrase)?
• Include everyone in the discussion?
• Avoid “serial interviewing?”
• Read the room, stay with the respondents, keep attention off of self and the guide?
• Keep self/ ego out of the discussion and avoid talking too much?
• Attend to non-verbal communications?
• Use a variety of techniques to promote discussion?
• Pace/ lead respondents?
• Listen rather than inform participants?
• Vary voice tone during process?
• Change location/ body position during discussion?
• Give clear instructions/ directions to respondents?
• Provide linking and logic tracking for respondents and observers?

Appendix – The specific case of testing advertising (BRUM test)

Over the years, I have learned not to ask, “Based on what you see here, would you buy this product or service?” On the surface, it seems to be a good question—a way to prove the advertisement is effective. However, in the climate of qualitative research, the answer cannot be trusted because it asks the consumer to comment on an act that will take place outside the room and in the future.

We can measure what is important to advertisers in four dimensions: Believability, Relevance, Uniqueness, Motivation. Most advertising do not score well on the 4 dimensions. Three out of four is good enough.

B: Believability The “B” of the BRUM test is meant to see if the advertisement makes sense, supports the notion of “willing suspension of disbelief,” and hangs together as the story unfolds.

R: Relevance The R in the BRUM test stands for relevance and refers to those to whom the advertisement should speak.

U: Uniqueness The next area in the BRUM test is the “U” for uniqueness. A unique advertising differs from competition and is remembered long after it is seen. Advertising rarely manage to be unique, though.

M : Motivation. By asking questions such as the following, a researcher can uncover the drivers that make for a particular set of motivations now held by the consumer: “What did you hear/ see in the advertisements that make you want to know more about the product/ service?” “What, if anything, did you hear/ see that pulls you toward this product/ service?” “What, if anything, did you hear/ see that pushes you away from this product/ service?” Exploring the pull/ push factor with respondents generates a lot of information about elements of the advertisement that create interest/ appeal (pull) or confusion/ dislike (push).

Appendix – Focus group limits

Focus Groups Focus group research, a major tool in qualitative research, has several limitations important to note here.

• Non-quantifiable form of data collection
• Not projectable to universe of similar respondents
• Built-in biases (more risk takers participate/ non-random selection).
• Sample size small by necessity
• Non-independent responses
• Reports are subjective analyses of opinions, beliefs, and assumptions.
• Moderator variation