This panel brought together VoC specialists to share insights on effectively managing customer feedback, from collection to implementation, in various business contexts.
Key topics included:
Introduction to VOC programs and their maturity levels
Prioritization strategies for customer outreach and feedback implementation
Closing feedback loops with limited resources and without advanced tools
Success metrics for closed-loop feedback processes
Ideal scenarios for closing the feedback loop
Identifying and addressing customers with misaligned expectations
Alternatives to traditional NPS surveys for B2B contexts
Timing and implementation of in-app NPS surveys to minimize user disruption
For a comprehensive look at the panel's insights, the full video is available on YouTube.
Speakers:
Guest speakers:
Aicea Yalcin
Min Gao
Kevin Kimoto
To learn more about Monterey AI, check our website and LinkedIn page.
Transcripts:
Aicea:
But let's get started, because it seems like this is a huge topic for everybody here, so no pressure. As we all know, we gather feedback, we look at feedback, we make product changes, and when it's great, it's up 44%, which is an amazing number. So impressed. But how do we tell our customers that we've made these changes?
So, before we get started, I'd love to hear from you guys. How did you get started with VOC? What do you do? And then I think most importantly for closing the loop, how mature is your VOC program?
Kevin:
Hi A little loud. Kevin I've been with Procurify for seven years. So I've done five Five fundraising rounds and I'm I'm beginning to see the gray hair I actually got started being obsessed with the customer from a go to market angle.
I've served basically every go to market team from marketing, sales, top of funnel, and I've noticed in my time that even though these are the front line folks talking to our customers or our market, they often have only their insights. And it just ends in their organization and it doesn't go back to the product team.
It doesn't go back to our leadership. And especially, you know, when you think about tech and you think about seven years and you think about the turnover, that insight and that wisdom is just institutionalized. And then these lineages of themes, I've seen mass extinctions so to speak. So that's actually how I got involved in this.
Min:
Hi, my name is Min, Min Gao. I've been with UiPath for also seven years. So I'm running the voice of customer for about a year or something. But before that, I'm doing the voice of customer in an unofficial way. So this one is a bit more supported by the leadership. They see a very important strategy toward running official VOC centralized.
And having all the side of VOC program. Join me together into a single program. So, similar problem but a little bit different. So, we have a lot of feedback from customers. However the general sentiment from the customer is saying, we provide you all the feedback. Where, where are they? Are they just going to the black hole?
So we, we, we really need to solve the closing the loop with the, with the customer, right? Either we committed to solve something in the roadmap or we don't do that. But that, that's, we just mentioned about we need to kind of say no sometimes. Perhaps it doesn't align with the strategy, perhaps resource constraint.
We just need to be honest about our decision so that customer is not always waiting for us. We should be more proactive in communications.
Aicea:
I love that bit. The term black hole has been my number one thing I hear at work from all teams. So really want to, we have to fix that. That's really important.
But you mentioned really quick that we get all this feedback from all of these different channels. How do you prioritize if you're going to reach out which team or which customer are you going to let know first? So what's your prioritization?
Min:
Yeah. So it's a, it's like art. Most of the time we try to move art to science with more data and number driven.
But at the moment we are looking at feedback be that source from different location. And then we'll look at actually look, the, the, the highest impacting metric will be like customer AR. And the, for prospect, we look at the opportunity size, and then we also look at the case volume, so we're doing a lot of collaboration with support team with the sales team, with the SE team, right, just to bring all those numbers together, cause, like, everything has different way of measuring impact, so basically the VOC program is about connecting all these teams together, and we made some hypothesis on what the top impact feedback should be.
And then, going back what we say, alright, these are the top item that we need to fix, how can we provide the data to support these things? But obviously, we wanted to move a bit more in the future where we can use some kind of tool or general capability to say, to help us to identify those hypotheses, right, instead of one person saying, this is what we want.
Aicea:
What about you, Kevin?
Kevin:
I would say that there's a mixture of both weighing out the opportunity cost. So really common. We use ARR and I've recently. And I'm trying to continuously plant these seeds that ARR can't be the defining factor because when you think about it, you are only weighing out things that are known.
So for example, when we think about our account executives, if you have a sales team, they will only pipe ARR if they believe that they could reasonably close that business. So when a product team takes that and ingests that information and weighs this out as saying, Hey, We will only build this if there's a dollar figure associated with it.
We then get into this trap of building for only what we know we can win. So, there's a, you know, Min, you said it's a mixture of art and science, and this is definitely the art part of the science. And really we've tried to move away from I forgot who, who said this, I think it was Behzad. The, let's go look at the highest paid person in the room, and then they'll tell a story about how their friend's gonna buy the software, and, and we should listen to what they want.
We're really trying to move away from that, and, and that's like that narrative driven focus of like, hey, we could tell and craft a great narrative, and, and we could fit most of the pieces together. Now we're beginning to start to unpack a lot of that, I would say early nascent stuff into, hey, can we thematically group things?
Can we weigh this out in, in respect to the existing roadmap and really use different frameworks to, to remove our own natural biases that we're introducing into the system?
Aicea:
Okay, great. And let's get to that actual question I think people want to hear. But Without looking at AI and all these cool tools, which would make our job so much easier when you're looking at a very small program, a new program, how are you going to be able to close the feedback loops with the customers without all these fancy tools?
Do you guys have a success story of, like, the most successful pathway you've found?
Kevin:
Yeah. Okay. I'll, I'll, I'll start with something really simple and it involves two products that I, I know and love and, and I'm happy to hear that they're here and it's slack and notion. The first part with slack is that that's, that's our main, you know, internal communication and especially since moving from being in an office to being fully remote during COVID.
Slack is our only. Island that we can ensure that people are being updated and I'm I recall an earlier panelist mentioned that there's both this internal and external need for Closing the loop and so with slack one of the things that we've managed to do is all of the NPS all of the surveys any kind of like core piece of customer feedback that isn't buried inside a transcript gets pumped into a respective Slack channel and everyone in the organization can see it.
Because that is being made visible. Everyone in the organization can also see how we are responding to it, who is sending that communication, and what that communication is. And we never even had that level of visibility when we were in the office. So this is, this is definitely something that I've seen help both the internal and the external and it, and it's using software that I would assume that everyone here, here has.
Min:
So for us, I, I think I would answer this in two fold, right? One fold is our internal customers and the other one is our external customer and how to close those feedback loops. So for internal, it's, it's a little bit easier, right? We, we can. Partners, we partner with the product operation team. So they do those monthly product roadmap sessions.
So they announce what the feedback are, they announce what their, what their roadmap, and what kind of feedback they already addressed. And they also send out the release note they make sure all the GTMP and sales team are aware of all the product changes and, and the roadmap. For the external customers, right, for external customers.
Is it still good? Is it, is it? Yeah, yeah. Okay, cool. So we rely on the GTM team and the sales team to do the communication because obviously we don't, we don't, we're not staffed, right, to do that kind of level of communication ourself. So we rely on the GTM and sales team who are mostly close related to the customer to deliver those informations.
This is not the best approach right now. In the future we'll try to. Do a bit more sophisticated way. But at the moment, this is what we have. So we make sure our communication we capture all the information, right? So we, we use Jira to keep track of all the major customer information and requests.
And then in that, we have, we, we track of the specific customer name and whoever the internal folks that are assigned to those, those accounts. So we make sure the feedback are close with the customer through the, through the GTM and sales team to communicate with the customer. If the customer have any kind of additional feedback follow up those information will generally come back with in the survey.
Or they can use, they, they they, they have a, probably a separate way of tracking those customer input using another product. So we're, we're looking at those additional feedback from there.
Aicea:
Yeah, I think both of you guys said something that's really important where you said it's not that clean, but it is getting information out there I think we kind of forget from time to time that customers just want to hear even if it's not going to be the easiest way.
We're always thinking about automating automating processes and making it like super scalable, but sometimes you just have to start off small. And I love the concept of like using your teams and those customer facing teams, telling them what's going on, keeping them up to date. Love Slack. We also use Slack.
It's a great format of passing along feedback. We're moving along. I think the next thing that really comes to mind for me and that where I've struggled with VOC is when we do close the feedback loop, how do you measure success? Because are you going to be following each customer to see if they've start using the product feature or if their NPS is improving?
How do you guys, what's your measure of success?
Kevin:
I would say that there's several things that we're always really weighing out. the first one and I, I don't necessarily always agree that whatever's easy should be the thing that you're measuring. So like NPS I think is a great kind of like waterline, but for a lot of and like for, for those of you who don't know what our company does, we, we started with purchase order automation and now we're into spend management and where.
Touching on the accounts payable process and for a lot of our users who've never really had to deal with structures when it comes to buying something. they might not actually be too happy that their CFO is saying, Hey, we're going to actually have a formalized requisition process, which means that you're not just going to go take the company card and randomly buy shit.
And then our AP team is going to reconcile it. It's like we're going to have a structured process. So there's a little bit of nuance to our NPS where we're actually going to change some of the things. Cause you know, for, I think a lot of us just buying the thing that we need without any, you know, input or kind of oversight is, is the easiest path forward, but not for an organization.
So we really have to weigh out with our organization is this NPS move like even if the NPS gets lowered, is it because the product isn't doing what it's supposed to do or is it because the structures that the original business objective was built around are being achieved? So that's one of them. And then the other aspect is actually just around product and feature adoption.
That's, that I would say is like our gold standard that we're really trying to drive towards because at the end of the day we exist as a business to fill a need and a gap for our customers and you should see a very tangible impact almost immediately. as they begin to use and explore the platform. For us we have, I'd say like a fairly robust set of metrics that we could be tracking from requisitions, approvals, purchase orders created, the, the, the type of stuff that that is the very sexy part of procure to pay technology.
And, and from that, The adoption piece is the main thing that we're just trying to drive towards. And that's how we measure whether we're getting close to our customers or whether we're actually straying away.
Min:
Similarly, adoption utilization is actually the big thing, right? So we have, we generally have a big problem where we sell a lot of license to customers, but a lot of time we, we know that the utilization is pretty low, right?
So a lot of feedback are coming from customers saying, all right, we have all these products, but we, we have a hard way of using them. Hence, that's why they're requesting those features, so we can, so they can use it more. So utilization is a big metric that, that we're measuring. Along with that, there are NPS, like supporting data, as well as revenue data.
So as a company, revenue is a huge thing, right? So it's always they always wanted to get revenue. And then we also start building this kind of similar to what Kevin was doing, like a top 100 priority list. So every biannual VOC session that we're hosting, we bring in the top 30 items, like, these are the top 30 absolutely needed feature requests from the field.
So we're looking at that items to see, like, how many I'll be addressing, like, going forward and continue to build on that list.
Aicea:
Okay, cool. And I believe we only have a few more minutes. Okay, so last question. In an ideal state, in an ideal world, what does closing the feedback loop look like?
Kevin:
I think it's not only this very clear internal aspect, because I actually feel that the internal teams are, that's, that's the first group of people that you have to make sure they understand that you're addressing the customer need. Okay. If our frontline support team members don't feel like we're building towards usually their, their biggest challenges, then, then why would they have buy in for any of our other initiatives externally?
And I love some of the initiatives that I saw here today. I'm definitely stealing all of them. It's to be in abundantly clear with the customer that Even if we're not going to road map this right now, even if we're not going to address it in this form, it's to explain it quite clearly because I think for a lot of us here in the room, our customers know a lot, but they might not know exactly how to build it or what's legal or what's even possible.
And so, It's on us as facilitators and practitioners to kind of guide them to, yes, I understand that you want faster horses, but here's, here's the car.
Min:
Yeah. For me, I think for us, it's about a customer centric, right? So being understand what feedback are coming from which customer is, it's important, really important, right?
Building a central database that can host those kinds of data is a key step. Yep. Outside that we can say, knowing which customer and what feedback they have, we can individually address each of them. Sometimes we need to bring in, like, the leadership team to connect with the customer on the product strategy discussion.
I think that's very important. A lot of, a lot of information you don't get it in the data, right? You only get it through conversations. So, for certain curated lists of accounts. We try to host those sessions where we bring the actual end user, the power users or decision maker from the customer side, together with our C level or leaders within our organization, physically or virtually, to have a joint product discussion, product strategy discussion, right, to, to align on the what, what, what to build and what not to build.
Aicea:
Okay, and I think we're at time, but are there any questions? Oh boy.
Q&A:
So I'm curious about the customers who are asking for faster horses when you give them the car. How do you identify them? Because they don't necessarily fit into the car list.
Kevin:
That's a great question. I think, I think part of it, so I'll go incredibly tactically. tactical and granular in this example, we have people who are asking for a certain thing that fits their exact workflow.
It's probably not even in the realm of possibility and there's probably actually a better way of doing it. It's often on our product teams to kind of weigh this out and facilitate this discussion. But what's interesting, especially about our field is the subject matter experts, I mean, like, I don't know how many of you have met people in Procure2Pay or Spend Management.
It's like, it's pretty niche for, for FinTech. The subject matter experts are often not the people who are inside of our product organization. And so they have to also lean on some of the experience that, and usually it's from our implementation team, our customer success managers, like the people who live and breathe the same air and water with our customers.
to actually determine, is this actually what we should be doing? And then can we start prototyping and testing it with a select group of customers to see if that's going to drive their goal.
Aicea:
I think your super users become really important VOC and advocacy go so well together. Cause you need those advocates to kind of help fill in the gaps where you can't personally.
Q&A:
I Have a question about NPS surveys So in b2b, I think you I pass and procure if I are kind of in the same path of like When you ask people would you recommend this product to your friends and family? They're like, oh No, I would never tell my mom about a B2B SaaS tool that I actually didn't, you know, I wasn't even in the purchasing decision making process.
So what are some other survey questions that we can ask in the B2B world?
Aicea:
I mean, we just use, we just say colleague or business and then also the way you send out the survey. So at ActiveCampaign, we've actually moved to in app. So instead of being an email, it's just You send an in app message, which makes them think more business, instead of just, yeah, your mom or dad.
Kevin:
We've done a few things we used to send out in the email, did not go well. Actually blacklisted our own domain one time. Like, oh, okay, let's, let's not send the NPS through email. We moved it to in app, and there was actually a small, like there's slight tweaks that we did that dramatically changed some of the user behavior.
So, an example was actually where it was located on the page would actually change how people are responding. If they had an ability to include comments inside of the NPS, we also started changing behavior. And then this last one, which I don't know if it's cheating or not, we colored the 9 and 10,
And then we colored the other ones red and yellow, like just really faintly. And that actually, that actually drove up engagement for people to engage with the NPS. So it's like just these small, subtle things, and, and we're always just trying to iterate on it. Because I agree with you, no one asks their friends and family for B2B software advice.
Min:
Same. I mean, it's, it's it's not a reference to friends or family, right? So it's more for other business leader that they, they're connected with. So, I don't own the survey. Actually the survey is owned by a whole, but many, many different, I think we kind of covered that, right? Different, different team runs their own survey.
So surveys, surveys NPS survey is run by CS and then there's another NPS survey, which is the in product NPS survey run by product. Essentially this is one of the problem we need to fix anyway. So, so, yeah, so NPS survey is one of the indicator There are a whole bunch of other survey that the, the, the other organization team are running.
My job here is to bring the survey result together, so we have a centralized view of what that look like. So can't answer your question, but yeah, so this is what the situation is.
Q&A:
I have a question for you about your NPS. Hopefully they're fair questions. One is like, how do you think about I'm not, I'm unsure about what your scale is, right?
But how do you think about when? to launch an NPS question, especially in app, right? Because you have a few things going for you or against you. One is timing, right? And the other is whether or not what you're flashing up a survey is going to have an interruptive or negative experience. If the user's going through their journey or trying to get something done, how do you think about that?
Kevin:
For us, we have, I would say like fairly robust segmentation of our user base. So, of course, like our super users, they're actually differentiated from our champions, and they sometimes are the same people, sometimes they aren't. We segment that those different user groups. Additionally, we have kind of like a, what, what most people would consider like a basic user, and they're like, they have one or two use cases inside of the platform, like I'm trying to request something.
I'm trying to receive something. And so with this segmentation, we will change the timing of when we do the NPS. And of course we'll also weigh out a little bit more like our champions and super users and their feedback. Of course we're trying to take everyone's feedback, but again, the nuances we might actually change the organizational behavior and inherently it should be for the better.
We know that there's going to be some people who used to be able to grab the card, the corporate card, and just buy, buy, buy. And because they can't do that, they will let us know, right? It's like, you made me spend longer because now I actually have to go through this process. And, and it's often on our CSMs and support to guide them through.
Trying to give them that perspective that this is probably healthier for your organization as a whole Even though there's like some changes that that have happened. So In addition to that just to tie it in Developing strong internal stakeholders is a huge must for our company because you are talking about changing behavior Well,
Min:
like I said, I don't own the survey, so I don't have input on that, so unfortunately.
Aicea:
I will, I will say, like, depending, you also asked about, you know, how often, when. I, I found it that it's most important to really look at your industry and your software. If it's an in app issue, it's a software that's being updated by the product team on a regular basis, you ask way more often. And Active Campaign, we now ask every 90 days because our product team is really active on fixing issues and changes.
At other companies, we used to ask yearly because it, it was a buy now, pay later. Not a lot changed because it's a lot of compliance issues. So there's no reason to ask them multiple times because they're gonna be like, I was declined because of a compliance concern. So I think there's flexibility Which I like with NPS because it really, you can improve your the feedback that you're gathering based on that.
