Have you heard? My new book Continuous Discovery Habits is now available. Get the product trio's guide to a structured and sustainable approach to continuous discovery. When you’re building a product, you can easily get overwhelmed by ideas. There are the ideas your product trio comes up with based on your discovery work, the ideas that come from your customers in the form of specific requests, and the ideas that come from stakeholders within your company like your customer-facing teams or CEO, to name a few.
While some of these ideas are unsolicited and may not relate to your current outcome or the opportunities you’re pursuing, that doesn’t mean you want to ignore or discard them completely. Just because an idea isn’t something you’re going to pursue today doesn’t mean it won’t be worth pursuing at some point in the future.
Just because an idea isn’t something you’re going to pursue today doesn’t mean it won’t be worth pursuing at some point in the future. – Tweet This
So how exactly do you manage all these ideas?
For today’s Tools of the Trade, we caught up with Leann Schneider, Product Manager at Plum. Leann found herself in exactly the situation we described above—her small product team was getting overwhelmed by insights from customers and internal stakeholders. They started using Pendo to manage these insights and make sure they were giving everyone a voice without disrupting the product team’s workflow.
We’re changing up the format this time around. Teresa sat down with Leann to record a video so Leann could walk through what Pendo’s tool looks like and how it works. You can watch the video or read a lightly edited transcript below. Keep in mind that as always, Tools of the Trade is intended for educational purposes and does not constitute an official Product Talk endorsement of the tool.
Meet Leann Schneider: Our Discovery Champion from Plum
Teresa Torres: Hi, everybody. Welcome to the next edition of Tools of the Trade on Product Talk. I’m here today with Leann Schneider. I’m excited to learn about her experience. Leann, welcome.
Leann Schneider: Hi, thanks for having me today.
Teresa: Do you want to start by just telling me a little bit about your role and company? Let’s set the context.
Leann: Sure. My name is Leann Schneider. I’m a product manager at Plum. At Plum, what we do is we use industrial organizational psychology or I/O psychology science to assess people’s potential. What that really means is that people who are either applying to jobs or employees at a company, we assess them to see what kind of jobs they will most thrive in.
Teresa: Oh, interesting. Okay, so I’m assuming a lot of your products are assessments?
Leann: We have a singular assessment. It’s called the Plum Discovery Survey, and we use that for a number of different applications like being able to fit people into jobs, but then also for development and career transition.
Teresa: Okay, great. Then tell me a little bit about your role there.
Leann: Sure. I’m one of two product managers at Plum, and we also have a product operations person and head of product. Basically, I do what any product manager would do, I help listen to our users, our internal stakeholders, and our customers, to learn about what types of things they want to see in our product, and then work with our engineering and design teams to make it a reality and include it in and hopefully make things better for them.
How Plum Uses Pendo to Collect Structured Feedback
Teresa: Leann, you’ve been a member of our CDH Slack community for a little while now, and one of the conversations that’s come up quite a bit is: I’m inundated with all these other sources besides customer interviews—whether that’s feedback from customer success teams, or sales teams, or individual customers. One of the things that you shared is that your team is using Pendo to collect some of those insights. I was hoping you could walk us through what you’re doing with Pendo and how that works.
Leann: Yes, absolutely. It really just started because we were inundated, getting it in meetings, getting it over Slack, and we were like, we need some sort of way to automate that or at least streamline it.
First, I’m going to show the public-facing side of how we collect feedback, and then I’ll show what it actually looks like in Pendo.
This is our application, and when you scroll down at the bottom, we have this widget called Resource Center. Although it looks like this Resource Center is made by Plum, it’s actually hosted by Pendo, and when you click on it, you can see there’s a bunch of links to different types of articles about Plum, so you can learn more about it or get the information you need. That’s also partially controlled by Pendo, but the one I’m going to focus on today is “share your feedback.”
Teresa: Just to clarify, so we’re looking at the Plum product?
Leann: Yes.
Teresa: I know you mentioned that you have an assessment tool. Is it the people that are using the assessment, not the people taking the assessment, that would see this interface?
Leann: Right, that’s a really good clarification. This “share your feedback” is available to our customers. The widget itself looks different depending on the user.
Typically, 99% of the people that come in take the assessment to apply for a job, and they’d be able to see some of this, like Plum Basics or My Plum, so they can get information and that we can answer their questions, but our clients themselves would be able to see the “share your feedback” part of it, and they’d be able to click in and provide feedback on their end of things, like the ones that pay for the service.
Teresa: If I’m an employer and I’m using your assessment to evaluate candidates, this would be the view of a recruiter or the hiring manager. They could come in and see the results of the assessment, and then they have access to this ability to give you feedback.
Leann: Yes, that’s exactly it. I should also mention that our internal stakeholders can log into Plum and they provide feedback through the same form as well. We’ve introduced that, if you have feedback on behalf of a client or even yourself, please use this forum so we can triage all the pieces of feedback through this forum.
We’ve introduced the idea that, if you have feedback on behalf of a client or even yourself, please use this forum so we can triage all the pieces of feedback in the same place. – Tweet This
Teresa: Okay, yes, great. Both, your client customers and your internal stakeholders have access and can submit feedback?
Leann: Exactly, because everyone internally at Plum has their own Plum profile and can go into the app and do the exact same thing.
Teresa: Okay, great.
Leann: Yes, basically, if they get to that point where they want to make a suggestion, either because they’ve decided they’ve seen this or maybe, our customer success rep has said, ‘Hey, that’s a great suggestion, why don’t you provide it through our feedback forum?’ You can go in there, and then we have a little form here, just a title, what you’re trying to achieve and what is blocking you from achieving it, and your current workaround, and then a little bit about themselves, if they want to provide that.
Then, they just submit the request, and that’s what happens on there, and it goes into the black box, and then I could show in a moment, unless you have any questions, what we see on our end.
Teresa: Yes, can we go through the questions you’re asking again and tell me a little bit about how you came up with these questions?
Leann: Sure. I won’t take credit for coming up with these questions, because this was done when I was on maternity leave, but the idea of trying to get at the underlying issue here. We don’t want people to say, ‘Can you do this for us?’ And then we have no context for why they even want that. We want to dig a little bit deeper. That’s why what you’re trying to achieve and what’s blocking you from achieving it, and current workaround, this gives us a bit of an idea of the severity of the issue. Is there even a workaround, or are they just dealing with it right now?
Then, this information is so that if we want to contact them to do some additional discovery, we have a bit more context of the type of persona of the person that we’re contacting.
Teresa: I love this. It’s almost like you’re creating a template to help them give better structured feedback.
Leann: Exactly, yes, and that’s for internal stakeholders as well. They have given that well-meaning feedback, but we want them to dig deeper in that initial request, too.
Reviewing and Upvoting Incoming Customer Requests
Teresa: Yes, great. Can we look at where this goes?
Leann: Absolutely. This is inside Pendo. Pendo Feedback is one of the products. We use the insights and guidance, but we also use the feedback portion of it as well.
There’s a number of views. We usually use the browse view, and it just lists basically all the pieces of feedback here. You get the request name. I can go into the statuses in a moment, but basically as they come in, they just come up over time in this view.
Teresa: I think you mentioned you have two product managers at Plum. What’s the volume of incoming requests that you’re getting?
Leann: It’s interesting because when we first introduced this tool, I think it was about three or four years ago now, the volume of requests was intense. We were getting stuff like, I would say weekly, for us that’s intense, getting things weekly. I would say it’s actually petered out within the last year. I think because either maybe we’re doing a better job of addressing things or making our product better, and that might be part of it, just the stage of our growth right now, or in the stage of our product. Also a lot of the requests that we know of are already in there, and people can upvote them.
You can actually see the number of votes here. Some of the features that have come in have a number of votes, and for whatever reason we’ve actioned them or not actioned them after having that decision. I found it’s actually petered out. Now we only get things maybe once a month or so that we have to triage.
The process that we use is that our product operations manager has a recurring meeting with me and the other product manager, and we review the requests on a regular cadence so that we could see, okay, is there anything new coming up? Has something that we decided to shelve for now received more upvotes and we need to take a second look at it? Is something maybe more strategically relevant than it was in the past? Before we said, no, we’re not going to do that right now because it’s just not one of our priorities for this quarter, but maybe it is now.
Teresa: Is the voting all internal? Do your customers get the option to vote?
Leann: Our customers can do it. I’ll be honest, I’m not exactly sure how it works when they see it. They can actually go in and upvote different pieces of feedback that they see.
Teresa: They have access to an interface that’s similar to this, where they can see all the ideas that came in, or the requests that came in? I can imagine you get duplicates. When they submit through that form that you showed, does it give feedback that this looks similar to these other things?
Leann: It doesn’t say that, but that’s certainly happened in the past. I was just going to show this merge request, so I don’t know if you could see that. We’ve had that in the past where it’s a request, or something we’ve seen before. Then you can merge the two requests together, so you’re not just getting people upvoting the same thing, but in different worded requests.
Creating a Regular Cadence and Process for Reviewing Insights
Teresa: I can imagine this can very quickly—it’s almost like what we see with bug databases, where you get a lot of duplicates, it ends up growing over time, we forget to look at it until something’s urgent.
Give me a sense of: How does this fit in your workflow? Is it hard to manage duplicates? You said you’re now getting one a month. Give me a sense for, how does this fit in your workflow in a given quarter?
Leann: I believe it’s monthly, we review the requests. Our product operations manager will download the results. You can download everything in an Excel worksheet, and then you can do sorting and you could do it here too, like sorting by status, or how many upvotes there are, or time as well. We basically try and prioritize the ones that obviously we haven’t reviewed yet, or maybe have the most votes.
Then you can also just filter by different statuses. We’ll basically get out the other requests that we don’t need to look at because it’s already released, or we’ve decided right now it’s just not a priority, and then try and focus on the main ones, because you’re right, there can be a lot of noise, and we just don’t have the time to review every single request monthly once again.
Teresa: Is the idea that you’ve got a theme in mind, and you’re just looking for, do we have requests that are relevant to this?
Leann: We have two trains of thought when we’re looking at this. Of course we have our quarterly priorities, and we’re thinking, ‘Okay, we have a few outcomes we’re hoping to reach,’ and we can go back into the feedback and be like, ‘Okay, this piece of feedback is related to that.’
I’ll just pick one for example. You can make them bigger too. You go into it, and we may say, ‘Look, this is an internal person, but this person has submitted this piece of feedback. At the time we didn’t follow up on it, but now it’s more strategically relevant. This is actually a good person that we can follow up with and ask if they want to do an interview.’
We’ve done that before, where it’s like this is a more strategically relevant thing, and we’re going to follow up with these people, and use them as discovery targets. That’s how one train has worked.
In some cases, new feedback requests will come in, and we’ll say, ‘Look, it’s not related to an outcome we want to do this quarter.’ The amount of work involved would just be way too much, and some of that triaging is just figuring out this would be a huge amount of work, and it’s just not relevant right now. We’ll set it to open for voting, and see if other people upvoted, and we can review it again next time.
Then there are some things actually much like this, where it’s a small change. We have this concept of different cards in our platform. A job rec card, for example, is one of these. In those, the piece of feedback here was, instead of having to click on specific links on the card, can you just click on the entire card?
This is a thing where we’re like, ‘Well, that’s like a two-point ticket for an engineer maybe. Why wouldn’t we just do this?’ It doesn’t need to be related to a product outcome, but it would be a delight for our customers to just have a little bit more of a simple experience going on.
We sometimes get requests that aren’t related to a product outcome, but it would be a delight for our customers to just have a little bit more of a simple experience going on. – Tweet This
In that case, when it’s a smaller thing, then we’ll triage it, and in this case, we assign it to a designer, and they were able to look at it and say, ‘Yes, this would be pretty straightforward to do, and we agree that this is a good idea.’
We almost have two tracks. It’s more like, is this related to a product outcome track? Then we evaluate it, and either set it to open for voting, so we can see if it’s upvoted more, or it’s more like the, well, this would be a quick win track. Let’s see if we could just fit it in or maybe put it into the backlog, and if an engineer has time, they can fit it in within the next release, and there’s that track, too. Then in that case, we’ll set it to building or interviewing users, if that’s the case, and then, of course, released when it’s eventually released.
Teresa: What I love about this quick wins track, is like it’s so easy to take this ethos of ‘customers don’t know what they want’ too far. Should we be building, just responding to feature requests blindly? Of course not.
There are all these little, tiny things in products that you don’t know bother a customer until you give them an opportunity for them to tell you.
My course platform does this—every quarter they release 50 to 100 little changes that were all driven by customer feedback. I love it, right? Because it’s a way for them to say, we’re listening to you. These were all easy changes. They’re not things that are going to make it onto our roadmap, they’re not things that are going to impact an outcome, but they’re little delighters. They’re like things that remove a little bit of friction here and there, and I think it’s really easy to forget. Sometimes teeny-tiny changes go a really long way for having happy customers.
Sometimes teeny-tiny changes go a really long way for having happy customers. – Tweet This
Leann: Yes, I completely agree. There’s a sense of satisfaction, too. It’s like often we do need to say for certain requests here, it’s just not now, or this would take a full quarter to do, and it’s just not going to happen.
There are a few things where it could feel like, yes, you know what, we can show that we’re listening, and you can have that sense of satisfaction, or maybe it’ll take me like a week or so to do a little bit of discovery to figure it out, and it would be like a two-week initiative.
We can just get it all shaped up and ready to go for when there’s a little bit of a lag on the team, and they can just work on it. I do find a lot of these types of requests end up doing that, or we just find it’s a good source of people if we need to do broader discovery on bigger initiatives.
Keeping Stakeholders Informed of Updates
Teresa: That’s great. This idea of like, now we also have this non-urgent backlog for when somebody has downtime, they can just grab something, and improve something for a customer is also a really great idea.
Tell me, when you change the status on one of these requests, does the person who submitted it get an update about that?
Leann: They do. I don’t want to actually do it… Let me see, this is more of an internal… Yes, here you can see. If you click on a different status, we have these like pre-baked messages that have been written by a member of the product team, and you can customize the response if you want to. Then you can choose whether or not to email the people who have submitted it or upvoted it, so that they know that it’s being actioned in a certain way.
Teresa: You have a way to, if a customer submits a request and it ends up getting moved to that build category, you can let them know, ‘Hey, we’re building your idea’?
Leann: Exactly. It’s nice, too, because instead of having to spend time in internal channels like Slack, messaging this person or that person, saying, ‘Hey, let this client know that we’re doing this,’ it’s just all automated for us.
Instead of having to spend time in internal channels like Slack, messaging this person or that person, saying, ‘Hey, let this client know that we’re doing this,’ it’s just all automated for us. – Tweet This
Teresa: Yes, that’s great. What impact does having this set up have on your work as a product manager?
Leann: I maybe will be a little selfish in response in saying that, now I’m not getting random Slack messages saying like, ‘Hey, here’s a piece of feedback,’ or during a meeting, we’re not surprised with a random piece of feedback, that’s well-meaning, but it’s like you’re almost left having to defend yourself in a meeting and having to explain it or ask questions about it.
Instead, there’s a process now where it’s like, ‘That’s a great piece of feedback, can you submit it here so that we can review it and consider it and talk about it as a team?’
In that sense, it’s a big relief, and we’re not left dealing with ad hoc requests like that. I think from a shipping product perspective, it’s nice because if we decide that we’re going in a certain direction and we need to figure out who to talk to while we have an avenue where we can see, have there been any requests about this? Are there people there who upvoted it that we can talk to and get in touch with? Because their email addresses are right there, we could just get in touch with them, of course, we might need to talk through our customer teams before contacting some people.
There’s either that, or like we were talking about earlier, you might just have those quick wins that we can do and really feel like when we’ve shipped a release that we’re doing a good mix of long-term initiatives that we’re making progress on that. Just some of the quick stuff that we feel like we’re making the typical user happy or a typical internal person happy that something that they asked for is getting actioned. It’s nice to have a bit of a balance of both.
Teresa: Yes, that’s great. I love this idea of like, you’re not constantly getting peppered because there’s a place for the feedback to go.
Leann: Exactly.
Teresa: I think years ago, I worked at a company where we used to have an internal idea management tool. It was the same thing, it was a way for people to feel heard, they could visibly see the status of it, people could vote it up and down, we could decide as a team what we wanted to invest in and what we were going to table for later. I think that transparency just goes a long way. We can never do all the feature requests. It goes a long way for creating some transparency around it.
Leann: Yes, I agree. That’s been the biggest benefit for sure.
Teresa: That’s great. All right, well, Leann, thank you so much for taking the time to share your story. I really appreciate it.
Leann: No problem, happy to do it.
This conversation was inspired by a discussion in the Continuous Discovery Habits community. Are you looking for a safe space where you can discuss tools and tactics with like-minded peers? Come join us there!
