Getting customer feedback can be tricky. It’s not always easy to know who to ask or even the best way to phrase your questions. And even if you do manage to collect some feedback, what are you supposed to do with it?
For the team behind Intercom, a product that helps companies better communicate with customers, effectively collecting feedback is second nature. But they’ve also learned that helpful customer feedback doesn’t always come in the way you might expect.
For example, when the Intercom team built their product’s map feature, they were focused on allowing users to see where they were acquiring their own new users and tracking those trends geographically. But pretty quickly, Intercom co-founder and vice president of customer success Des Traynor says it became clear that wasn’t how the feature was being used.
"Recently we observed a lot of our users were using the map feature, but they were screenshotting it, cutting out pieces of data and tweeting it or using it in presentations," he says.
"That wasn’t really what we thought we were building, but we realized that because we were able to talk only to people using the map feature and see what they were using it for, we could get really focused feedback on what was actually valuable about the feature."
So they did. The team reached out to a group of its users who’d recently engaged the map feature and asked a few simple questions, things like, "How do you use the map feature?" "What do you like about the feature?" and "How could this feature be improved?"
And quickly a pattern emerged. Traynor says the users didn’t care so much about the geographic precision of the map—no one was zooming in on New York City to make sure they had engaged users in every borough—but preferred the feature for showing off the global impact of their business to would-be customers or investors.
"So we learned that the better looking the map is, the more people will use it," he says. "Because ultimately it’s about vanity and helping people by making (the map) easier to share, hiding commercially significant information and even allowing customers to tweet it.
"All of these things seem obvious, but you don’t get there without finding a way to talk to people using your product and seeing what they actually think."
All of these things seem obvious, but you don’t get there without finding a way to talk to people using your product and seeing what they actually think.
Des Traynor, Intercom co-founder
The lesson? If you consistently observe a certain behavior from your users, react to it. It’s a valuable type of customer feedback.
That being said, simply reacting to your customers’ behaviors and suggestions probably shouldn’t be the number one guiding factor when it comes to how you run your business. Frankly, that would be anarchy.
So what feedback actually matters? And how do you know you’re collecting the right type? We caught up with Traynor to discover what’s he’s learned about feedback over the years, how the Intercom team approaches customer interactions and what you can do to start collecting valuable feedback of your own.
Bridge the Divide Between Company and Customer
When Intercom was founded in 2011, the company’s co-founders came from varied business and software backgrounds. But Traynor says that all had noticed a common trend: there was a growing disconnect between the people building the business and the customers using their product.
Whereas a coffee shop owner can easily connect with customers on a personal level and establish a rapport with regulars, an engineer at a small software company isn’t afforded the same luxury. Traynor says that in his experience, the phenomenon manifested in a number of potential problems.
"First in that it was hard to see new customers," he says. "We used to get a PayPal notice of subscription, but even that only carried their email address. We didn’t know what was going on otherwise to try to piece together if they were a good customer or not."
The second big divide stems from a similar issue: when you don’t know who your users are or what they’re doing with your product, it’s hard to effectively communicate with them.
"The hardest part was that you could either talk to everyone or no one," he says. "Customers couldn’t reach out except through a support ticket system and it was all very transactional. The only real options they had were to consider the severity or priority of their issue.
"If you compare that with the level of interaction you can have over WhatsApp or SMS, they’re light years apart."
And as a result, sometimes the outcome and intent of that communication is skewed.
"Primarily, the only interactions there would be were with customers who were upset with shortcomings (of the product), which tends to lead you to biased view of your customer base," Traynor says. "When the only feedback you get is negative, it’s not great for organization. Your natural inclination is that if that feedback is bad, then all of your customers think that, too. And that’s not the case."
So they built Intercom, a simple, lightweight way for companies to connect with their users in personal ways. And through the process, Traynor has learned a lot about the best—and worst—ways to approach customers when it comes times to seek feedback.
Customer Feedback is Oxygen
More specifically, Traynor says, customer feedback is oxygen for the future of your business. So how do you establish a stable connection to that oxygen supply?
"One thing that’s very true is when people talk to Intercom, they feel like they’re having a conversation," he says. "As opposed to when you’re talking through a survey or ticketing product, it feels like they’re more freely willing to have chats when it’s an actual person.
"I think it’s that there’s more of an honesty that comes when they’re talking to a person, instead of rating on a scale. I think it’s viable to extract feedback from conversations in that way."
But before you even reach out to customers, Traynor says there are a few important things to keep in mind.
1. Start Small
At Intercom, a group of trusted users serve as a testing group for new products and software iterations. After the development team and then the rest of the company finishes testing a new product, they roll it out to the team of testers. These are "good, active users who are happy to accept new features that might not end up shipping or whose behavior changes after a few iterations," Traynor says.
When it comes to finding that group for your own company, what you want to do is start small. For example, Traynor suggests starting with 600 people instead of 6,000 to condense your feedback pool.
"You don’t want to scale the part that’s messy, you want to do the bit that works," he says. "You’ll see things like where the documents are stored aren’t clear or users don’t know what a label means. Any of those problems, they don’t get better with size. You don’t need 100 test people and have half of them come back asking what merge means. You just need five.
"This is where feedback is great because it helps smooth the path. People are able to just start using it straight away because you already asked 600 people who were a reflective sample."
That way you won’t get hundreds of people asking, "How does this work?" come the product’s actual release because you ran through those problems with a smaller sample.
"Oftentimes when you do it right, the release itself can feel anticlimactic because its been so gradual."
Similarly, in an Intercom blog post titled "Start with a Cupcake", Traynor uses the analogy of baking a wedding cake to talk about the importance of starting small and approaching feedback as a constant process.
The classic way people think about baking has them focussing of the individual ingredients, each of no value to the end user. You can deliver the base first, then the filling, and finally the icing. Only at the end of the final phase do you have something edible; something you can learn from.
Alternatively you could start off with a cupcake. You’ll learn the flavours you like, uncover any problems in your kitchen, and in general you’ll fast forward the feedback loop. You can then step up to a regular size cake, safe in the knowledge your ingredients are all fresh, your oven works, and your flavours are nice. Only then can you deliver a wedding cake.
The key difference in approaches here is how quickly you get feedback.
In short, sharing specific upcoming product features with customers to garner feedback yields no value. Instead, share a lightweight version of the pending product update that gives your customers a real taste of what's to come.
2. Find the Right Type of User
"The worst thing you can do is communicate with all of your customers at once or do something like say, ‘Hey, we’re taking feedback,’" Traynor says. "That’s unqualified feedback. You’re mixing in long-term, passionate customers with people who signed up yesterday because they saw you on Product Hunt.
"As one of my colleagues says, you might as well read the comments on Hacker News for your product strategy. That type of feedback doesn’t help you think about what inputs need to drive your business."
So when it comes to finding the feedback that does help drive your business, Traynor says it’s important to focus a similarly small, focused pool of users. For example, if you take a company's developer wanting to explore potential product issues, Traynor says conflicts typically stem from three main problems:
There aren’t enough people using a feature
People don’t know about a feature
People are using a feature but it’s causing them frustration
Let’s focus on the first option in the context of Intercom: "If there aren’t people using are a feature, talk to them about why they aren’t using it."
Maybe a user hasn’t generated any reports using Intercom. Traynor suggests reaching out to further explain how the feature works and how the reports are generated, followed by an inquiry into why they haven’t explored the function. Not only will this help you discover problems users may be running into—"I haven’t created any reports because I need to export them as Excel files"—but it also allows you to assess any untapped consumption for your product. So in this case, maybe that user would be willing to generate reports using Intercom if they were able to export them as Excel files.
"In that case, they’re ready to use the feature, but you just need to remove that barrier for them to do it," Traynor says.
3. Don’t Confuse Quality Assurance with Customer Feedback
It’s rarely the case that companies don’t communicate with their users at all. But often, Traynor says he’s seen quality assurance emails or surveys confused for customer feedback. Though there’s still value in doing quality assurance, he says it’s not always the most important way to connect with your customers.
"The bigger question is: Have we understood the job we’re trying to do well and have we designed a feature that mirrors or supports our users’ workflow?"
Traynor says one of the biggest lesson he’s stressed goes back to focusing on the right users. He references a variety of objective metrics for customer satisfaction that, while they can be flattering, don’t mean much when it comes down to extracting value feedback.
"They’re great ways to find out whether or not your product meets that metric, but it doesn’t tell you anything you’ve missed. There’s not any extra info on how to make a score of 8 into a 10," Traynor says. "You’ve no good data on this and its such a blended number—taking in your angriest and happiest customers with those who used your product once and forgot about it.
"Product teams don’t work on a product as a whole—they work on pieces—and that’s what your feedback should be focused on, too. The biggest lesson is only talking to the right people."
4. Remember, Environment and Context Matter
Sure, you can spend hours crafting a survey to email to your users and have it land in their inbox on Friday night. Let’s say it’s just four relatively-quick questions. Even if you phrase your questions correctly and target a smaller sample, you’re probably not going to get the responses you’re looking for.
Put yourself in the shoes of the customer who just received the feedback email. Maybe you see the survey when it arrives in your inbox that evening. On Saturday morning you revisit the email. You enjoy using the product and don’t want another thing to worry about come Monday morning so you decide to respond quickly. Surely some feedback, even if delivered hastily, is better than no feedback at all, right?
Traynor isn’t so sure.
Let’s imagine a different scenario. This time while you’re already using the product—let’s say it’s a web app—when a sidebar with a photo and name of a real person who works at that company pops up. Instead of emailing you the survey, you’re asked the four quick questions on the spot. The difference?
"Your response likelihood and response quality is infinitely better if you talk to users about something while they’re doing it," Traynor says.
"If you consider me giving feedback to Google Hangouts right now while I’m using it versus me talking about it yesterday during the Seahawks game, I’m going to be much more committed to helping make Hangouts better right now."
Especially if you can touch on a pain point that a user might be having, reaching out at a more opportune time could mean the difference between a thoughtful, quality response and a half-hearted answer to a survey.
"Feedback should be conversations, not structured forms," he says. "Asking at the right time and in the right place is much more meaningful."
6 Tips on Asking for Feedback
So you’ve done your research and determined the subset of users you want to ask for feedback. What do you do now? Traynor shared with us some of his most practical applications when it comes to forming questions and communicating with your users.
1. Start with the End in Mind
"Never ask a question that no matter what way (answered) won’t help you get to your end goal," Traynor says. "Always start with the end in mind."
For example, if you’re trying to figure out how to improve your product’s comment feature, don’t ask questions like, "How often do you use Intercom?" that don’t drive that purpose.
Traynor stresses that when you sit down to write a set of questions you have to ask yourself, "What types of responses will help me solve this problem?"
2. Avoid Dead-End Questions
This is a pretty easy one: Steer clear of any questions that are overly simplistic or could be responded to with a one-word answer.
"For example, if you ask, ‘Are you happy with Zapier?’ It’s easy to just say ‘yes’ and there’s no future to that conversation," Traynor says.
Instead, he recommends sticking to a tried and true reporter trick: ask questions that lead with "who," "what," "where," "when," "why" and "how." It’s an instant boost to quality responses.
3. Don’t Couple Independent Questions
Traynor says it sounds innocent enough to ask, "How fast and reliable is Intercom?" But really you’re doing yourself a disservice.
"On the surface it sounds like a perfect question to ask but it’s actually two questions. So both should be individual items," he says.
He likens the phrasing to asking someone, "Do you eat healthy and exercise?" While those two actions are oftentimes closely associated, you can exercise and eat terribly, and inversely, you can eat great and never exercise.
If you have two different features, it’s two different questions. Period.
4. Provide Meaningful Timeframes
Going back to our exercise question, if you ask "Do you exercise?" chances are most people are going to say "yes." But that still might not be the most accurate representation of your data set.
"Because no one really likes to exercise, but you’re supposed to say you do it," Traynor says. "But if you say, ‘In the past week, how many times have you exercised?’ it cuts through a lot of the bulls—t because it’s going to be zero from a lot of people."
By phrasing questions around meaningful timeframes you get a better idea of users’ behaviors when it comes to using your product.
5. Ensure Your Users Feel Qualified to Respond
"A lot of times people feel too humble to suggest they might have something valid to contribute," Traynor says. "So if you ask, ‘What should we improve?’ People will think, ‘I don’t know. You’re the expert who built this really amazing product.’
"Instead try something like, ‘In your opinion as a new user,…’ Always make users feel comfortable with what they’re trying to say."
6. Avoid Hypotheticals
The truth? Your users don’t know what the future holds. Neither do you. So it’s really not going to benefit you if you ask them to speculate.
"Say you ask, ‘If our product cost $49, would you subscribe?’ Well, our product has never cost $49 so it doesn’t matter," Traynor says. "It’s not a behavior question and to treat it like a data source doesn’t help you at all."
Finally, Treat Feedback Like a Hypothesis
Now that you’ve collected feedback, talked to users and filtered out many of the anomalies that occurred, take a deep look at that pool of data and start clustering it into groups.
"Say your goal is to start providing features or support that larger companies want," Traynor says. "If that was your strategy for the month or the quarter, you would want to find slices of feedback along those parameters. You segment that data, but in accordance with your strategy, and to prioritize that you can use simple tactics."
Traynor says you can find relatively informed opinions and trends within your structured and unstructured data by clustering samples.
"If you do all that, you’ll end up with six or seven key points, but a point I always make is a pearl of information or an anecdote is not data and data is not direction," he says.
For example, if you have 20 people out of 70 say you should be able to modify something within a particular feature, Traynor says that’s indicative that a change might be to your benefit. But before you go and make it the most important project of your quarter, he says it’s important to validate it.
"Sometimes it makes sense to (make those changes), but you have to realize those findings are only ever a hypothesis," he says. "You almost have to act like a detective and say, ‘It seems like this is true, how do we verify it?’"
So maybe you go back and ask another 200 people a quick, focused question to see if the trend resonates with a larger segment of users. Traynor says this will help you validate that a need exists and supports your hypothesis that a change should be made.
All hypothesizing aside, Traynor still believes it’s important to trust your gut too when it comes to your product’s direction.
"It’s worth saying that if you really know your product and customer base you can bypass a lot of this if you have a good feel of what you’re trying to do," he says. "Inevitably as the product grows though, you’ll begin to attract people less like yourself who may not fall into that area."
Even with all the improvements you could possibly make to your product, you'll still need to help some customers. But don't wait for them to contact you—get in touch first when it seems like there's a problem, and help them before they think to ask for help. That's the idea behind proactive customer support.
Image Credits: Suggestion box photo courtesy Andrea Williams. Des Traynor photo via LinkedIn. Intercom graphics from the Intercom blog and Intercom.io.
