The Iceberg Theory of User Feedback
There’s valuable feedback about your product and your business hidden beneath the surface.
Above the water sits the easily visible feedback: metrics you’re collecting in analytics tools, NPS survey responses, customer complaints, angry Twitter rants—and hopefully a few vocal evangelists. Often, this vocal and visible group is the squeaky wheel that gets the grease and gratitude and drives change. The problem is that this type of feedback is simply the most accessible rather than representing the reality of how people feel about your product.
Two kinds of feedback tend to rise to the surface most prominently—feedback on your product’s most well-defined problems and its well-defined, helpful solutions:
Well-defined problems take shape with a critical mass of specific and clear feedback, shining their own light on next steps for the product. Well-defined solutions attract customers and feed word-of-mouth. They should be easily identifiable as the beating heart of your product. You’ve made something of particular value for which people pay you with time, money, and attention.
But what about everything below the surface—all the little problems that people forget to voice or don’t even bother to complain about that, over time, drive them silently insane?
Think about your own experiences: you come for one game-changing feature so you can deal with a few bugs. But you leave if the experience is fraught with annoyances—slow page load times, confusing navigation, a wonky feature, bad help documentation, or slow response times from support.
You’re probably hearing from just a sliver of your overall customers. One customer service expert has found that only 4% of dissatisfied customers voice complaints to businesses. If you get 40 support emails (which tend to focus on immediate troubleshooting) each day, but you have 1,000 daily active users, for example, then you have plenty of “under the surface” feedback to explore.
You might not be heading into the frigid waters to look at what’s underneath the obvious surfaces. Sometimes what you excavate won’t amount to much, but sometimes the feedback can crystallize, with some effort and dot-connecting, into some “Aha” moments for you.
Here are a few ways to dive in deep and collect all the feedback you need to keep your customers from walking away.
Silence Isn’t Always Good
Minor annoyances usually don’t spark angry emails, but that doesn’t mean they hurt your product any less. You may satisfy the loud complainers who’ve gotten your attention, but the quiet ones may be struggling with diminishing patience.
Upon encountering minor annoyances that don’t feel worthy of an email to support, the quiet ones get irritated and feel ignored. “How many times do I have to reformat this CSV before the damn uploader will work!?” If customers are quiet, don’t assume they’re loving every minute of your product. Maybe they are, but that’s a dangerous assumption. (Ever notice customers cancelling and churning without a peep? It happens a lot.)
Check in with your customers and open up opportunities to have a helpful conversation. Unless you’re just starting out with a product, you probably don’t have the resources to coordinate and schedule check-ins with every single customer. Instead, respond to behavior and signals of intent.
series of clicks in which your users are pummeling their mouse buttons in frustration. It’s like punching your site in the face, usually because it’s not doing what the user wants or expects it to.
Talk to your customers
Automate a simple automated email to reach out based on behavioral signals like inactivity trends, lack of communication, or an abnormal amount of exports. For instance, you can trigger an email to users who haven’t logged in a week or 2 or have viewed your support documentation X number of times within 2 days after already having been onboarded.
If you’re tracking support requests (as a user event), you can create an automated email campaign for whoever hasn’t contacted support in awhile.
As people enter this segment, we could trigger a message like this one to spark a conversation (while providing a helpful resource).
Subject: How are things going?
Silence is either golden or deadly—I guess it just depends on whether you’re a librarian or a ninja. In any case, we wanted to say hi and ask, how are things going?
If you’re experiencing problems with [our product] or have questions—anything from trouble with your data integration to needing inspiration for a new campaign—just let us know. Seriously, we’d love to help.
Oh, and here’s a [blog post/guide] we created recently that you might find helpful!
This is a simple way to keep accountable to your quiet customers. You can, of course, get more sophisticated than this by tracking usage against support tickets to help you better understand how often to expect to hear from customers based on their level of usage—and then create segments to target people who fall outside those ranges.
Find Bad in the Good
It feels really good to get an email from a happy customer. In Customer.io’s case, there’s nothing that makes us happier than hearing about a customer whose behavioral emails cut churn, drove adoption of a new feature, and helped their customers.
But just because a customer has a success story doesn’t mean there isn’t constructive feedback to be had. These are the people that got results—and if you work in SaaS you know that getting people from a non-customer to customer to successful customer is really hard.
So, here’s what we recommend. Ask if you can write a case study about them, then schedule 30 minutes to interview them about their success. They get featured on your site, and you get a glowing review of your product. But you can also get something else way more powerful: a look behind the scenes. Just because someone ran a successful campaign doesn’t mean it was easy—it just means they persevered. It probably wasn’t as easy you’d hope it could be.
So assuming that Customer.io is doing the interview, here are the types of questions we’d ask to find out how this project got from A to Z:
- Did you do this alone or did you have help? Where did you need a hand from a developer/designer/copywriter, etc?
- What were the hardest parts about pulling this off? Was it wrangling people and deadlines? Or was it our software?
- What would you do differently next time?
- What could we have done to make this easier for you?
You know the problem areas in your software. Dig in. Find out all the ways this could have failed—it’s likely that other customers are getting snagged in the same areas.
Think Bigger—Way Bigger
It’s easy to understand how problems may lie below the surface in any situation, but that’s really just the beginning. Consider another type of iceberg model. Imagine that the bottom of an iceberg—the part you cannot see—is also its foundation. What you see at the top is directly influenced by the base.
As explained in the book It’s All Connected: A Comprehensive Guide to Global Issues and Sustainable Solutions, an event is connected to underlying behavioral patterns. Those patterns are the result of systems, which in turn are created and underpinned by mental models, beliefs, and values.
Take, for example, an oil spill, which is an event that is easily witnessed above the surface:
If you looked only at the event, you might think that we should just build stronger tankers and better pipelines. But if you look at the root cause of such spills, you can start to understand and address long-term, sustainable solutions such as developing energy sources that do not rely on oil transportation.
Changing the models, systems, and patterns that influence your employees and customers isn’t easy, but it is the only way to make structural changes that naturally result in a better company and product.
How are your employees incentivized? Do you encourage them to make the best product possible, or are they constantly under the pressure of due dates? Do they feel comfortable taking extra time to do their best work, or are they required to sit through endless meetings? Perhaps more than anything else, this is why some products are truly great while others get lost in the day-to-day obligations of work.
“What gets measured gets managed.” It’s the oldest business cliché in the book for good reason. Measurement and incentives go hand in hand. For example, do support people track the number of tickets closed or the percentage of happy customers? Every person and team at your company should have a North Star number that helps them understand if they are making meaningful progress or just ticking items off a to-do list.
# Customer Interaction
Talk to customers early, often, and face-to-face. Chris Savage, founder and CEO of Wistia, recommends doing this in person when possible. In-person customer visits sound like the quintessential thing that doesn’t scale, but Chris has a great point that scale can actually help as customer numbers local to your team rise.
This in-person feedback was more honest and changed how Wistia did user research and product design:
In person, customers tell you things they would never tell you over the phone….[In-person visits are] the best way to understand if you’re building something of any value, and my only regret is that we accidentally convinced ourselves, at some point, that we didn’t have time for it.
The world’s best companies are self-aware and realize that it’s not great features alone that grow a company but a great experience. They understand that the iceberg is large and unforgiving. Look deep, look often, and never stop chipping away.
Have you ever been surprised to learn about under-the-surface feedback? Share your story with us in the comments!