Have you heard? My new book Continuous Discovery Habits is now available. Get the product trio's guide to a structured and sustainable approach to continuous discovery. If you want to make continuous interviewing part of your discovery, you have to automate the recruiting process. It sounds simple, but that doesn’t mean it’s easy.
If you want to make continuous interviewing part of your discovery, you have to automate the recruiting process. It sounds simple, but that doesn’t mean it’s easy. – Tweet This
While many product teams want to talk to customers every week, they struggle to make this a reality. Whether they face blockers from within the company, difficulty identifying and connecting with the right people, or potential interviewees who don’t show up to their scheduled sessions, the road to continuous interviewing is full of challenges to overcome.
For today’s Tools of the Trade, we caught up with Yury Oleynik, the VP of Product Management at HiveMQ, who has managed to make continuous interviewing a reality on his team.
Like many of the stories we highlight here on Product Talk, Yury’s features a few false starts and frustrations. Yury shares a few of the tactics his team tried to automate recruiting before they came across the tool they’re currently using, Orbital.
You’ll hear how Orbital addresses many of their needs and helps the HiveMQ team generate a steady stream of customers to speak with every week. You’ll also hear some of the challenges Yury and his team continue to face with their recruitment process.
As always, this post is intended for educational purposes and should not be considered an official Product Talk endorsement of the tool.
Meet the Continuous Discovery Champion, Yury Oleynik
Meet Yury Oleynik, VP of Product Management at HiveMQ.
Yury Oleynik is the VP of Product Management at HiveMQ. Founded and headquartered in Landshut, Germany, HiveMQ empowers businesses to transform with the most trusted MQTT platform (a messaging protocol used for connecting sensors and devices, such as in the Internet of Things).
The HiveMQ MQTT Platform is designed to connect, communicate, and control IoT data under real-world stress and is the proven enterprise standard. HiveMQ powers use cases in automotive, energy, logistics, smart manufacturing, transportation, and more. Leading brands like AirFrance/KLM, BMW, Matternet, Liberty Global, Mercedes-Benz, BraveHeart, and ZF choose HiveMQ to build smart IoT projects, modernize factories, and deliver better customer experiences.
As the VP of Product Management at HiveMQ, Yury oversees HiveMQ product discovery and delivery by the Product & Engineering team. In his role on the leadership team, he is responsible for shaping and executing their product strategy.
Yury joined HiveMQ in January 2022 and says he was attracted by the culture, sophistication of the MQTT platform, and the diverse and intricate IoT market they serve. Fun fact: A single HiveMQ broker can pump up to 1,000,000 messages per second—more than a half of global WhatsApp message traffic per second!
Describing the purpose of his team, Yury says, “We are here to 1) discover and understand the problems (need) our customers have; 2) shape a strategy to navigate the problem space on our journey to our vision—to build the Central Nervous System for enterprises; and 3) enable our peers in engineering, product design, and product marketing with a clear understanding of the problem to discover and deliver the product our customers love.”
We are here to enable our peers in engineering, product design, and product marketing with a clear understanding of the problem, so that we can discover and deliver the product our customers love. – Tweet This
Yury says he was fortunate enough to start his career in product management at a company with a strong product culture, where regular customer interviewing, assumption testing, and a problem-first mindset was the norm.
At HiveMQ, Continuous Discovery Habits has been required reading for the team. “I first read it back in 2021, and it helped to formalize much of what my old team had already been practicing at the time,” explains Yury.
Understanding the Need to Recruit Interview Participants and Initial Steps to Automate the Process
When Yury joined HiveMQ back in January 2022, they had a large engineering organization and only a single product manager at the time. “Although the team had many interactions with customers, there was no strong methodology in customer interviewing,” says Yury. “One of my first focus areas as a product leader was to evolve the product culture of making customers the source of truth in our product discovery and delivery efforts. Increasing the amount and the quality of customer interviews became my first priority.”
One of my first focus areas as a product leader was to evolve the product culture of making customers the source of truth in our product discovery and delivery efforts. – Tweet This
Yury quickly realized an interesting aspect of the HiveMQ product: As essentially the data backbone for their customers, one critical measure of success of the product is realized when customers do not need to touch it. “HiveMQ should flawlessly pump messages 24/7 in the background without a single user interaction. This aspect made reaching out and scheduling calls over the Customer Success (CS) team a low-return exercise for the customers who have already installed and configured our software,” explains Yury.
Because there wasn’t a lot of interaction between the PM team and existing customers, Yury tried having the product team join existing customer touch points and leverage support tickets to schedule interviews. “But that also only got us so far,” says Yury.
While the initial tactics didn’t yield great results, Yury wasn’t ready to give up on discovery. “As we were launching our new HiveMQ Cloud plan, discovering new problems and collecting feedback became essential in order to find product-market fit.”
Yury’s next step? Putting together a team of two product managers and one product marketing manager to design and execute a customer outreach campaign. The campaign the team came up with included reaching out to a list of around 5,000 users from HiveMQ’s free plan, as well as several hundred marketing qualified leads (MQLs) from inbound leads.
They divided this pool into three categories: about 100 of the most promising leads (who they’d send personalized outreach via email, LinkedIn, or phone when applicable), around 300 leads (who they’d send a personalized email), and the remainder (who they’d reach out to via email).
Unfortunately, this campaign didn’t lead to the huge influx in interviews they’d anticipated. Yury says, “It was quite disappointing to learn that we managed to get only about three interviews as a result. That was pretty disheartening, and we didn’t see a repeatable path for setting up automation here. So we continued to have infrequent, reactive customer interviews for a few months.”
It was time to go back to the drawing board when it came to customer interviews.
Around this time, HiveMQ rolled out a new Cloud plan that created more in-app touch points with customers. They began to consider building their own system for in-app recruiting, so Yury reached out to his network to see how other people were approaching this problem. This is when Yury first learned about Orbital, which seemed to be exactly what they were about to build themselves.
“Orbital was still in early access back then, but we saw that as an opportunity to help shape it moving forward,” says Yury. “They already had the functionality we urgently needed to automate recruitment in-app for our cloud offering users, and their roadmap addressed our (assumed) future needs.”
Turning to Orbital to Meet Their Customer Recruiting Needs
It’s no exaggeration to say the results with Orbital were pretty instantaneous. “Once we rolled out Orbital within our new cloud offering, we got six interviews scheduled within the first week after going live,” says Yury.
Once we rolled out Orbital within our new cloud offering, we got six interviews scheduled within the first week after going live. – Tweet This
Orbital allows you to set up pop-up messages that look like they’re coming directly from members of the product trio. Click the image to see a larger version
Once a user clicks the “Schedule chat” button, they’re able to select the date and time when they’re available for an interview. Click the image to see a larger version.
Once a user has selected a time when they’re available for an interview, Orbital prompts them to provide their contact information and confirm the interview time and date. Click the image to view a larger version.
But it wasn’t a perfect solution. “After the first excitement, we noticed a problem—a high number of people didn’t show up for the call. Some weeks this was as high as 50%. These were new users on our free plan, so in hindsight it shouldn’t have been that surprising. The next problem we ran into was that many of the people we interviewed weren’t within our target users.”
It was time to refine their approach yet again.
They went back to Orbital and made some modifications and feature requests to address the following issues:
- Targeting – they waited until users had been using the product longer before recruiting
- Screening – they added screener questions to qualify people in/out
- Reminders – they enabled email and SMS reminders
- Personalization – they added the interviewer’s name and role to the intercept, so users would know they were speaking to a real person (product manager/designer/engineer)
These changes did the trick. “With these improvements in place, we now have a consistent flow of interviews every week with our target users from our self-serve cloud offering,” explains Yury.
In this short video, Yury walks through how his team sets up a new discovery campaign in Orbital:
And when things change (as they often do), the HiveMQ team can easily update the targeting, screening—or anything else—from within the Orbital admin interface. “This has been super important for us to be able to move quickly and adapt based on our discovery learnings,” says Yury.
The Orbital user interface makes it easy for any member of the trio to update many aspects of in-app recruitment, such as interviewers and their availability, interview length, and the text that appears on the initial pop-up message.
One of the keys to their continued success according to Yury is that each interviewer has their own Orbital account with their calendar connected so they can freely move interviewers around and set up round-robin scheduling without coordination. “That was one of the big selling points for us—we wanted to make sure that anyone on our product and engineering team could ‘self-serve’ interviews, without needing to ask for permission or worry about coordination.”
This has been a game-changer, says Yury. “As a product leader trying to foster a culture of discovery, it’s been valuable for me to see which customers/accounts each team is talking to, and remove barriers to the insights we need to do our job.”
As a product leader trying to foster a culture of discovery, it’s been valuable for me to see which customers each team is talking to, and remove barriers to the insights we need to do our job. – Tweet This
Orbital makes it easy to get both an overview and detailed understanding of who is scheduling interviews with the product trio. Click the image to see a larger version.
Continued Challenges with Recruiting Customers
While their initial success with Orbital is undeniable, Yury admits that they haven’t yet figured out how to regularly speak with highly targeted individuals at large enterprise customers that don’t frequently visit their product. “To be honest, we still haven’t managed to solve this yet, as this segment poses a different set of challenges for our recruitment efforts. In many cases we can’t reach them in-app because of the nature of our product.”
While they’ve tried leveraging CS and sales teams to put them in touch with the right individual within an enterprise customer account, that person is often not their main contact. This requires several steps or emails forwarded from one person to another to get to the right person. “This takes ages and often goes nowhere,” admits Yury.
For the time being, they’ve been returning to the same handful of people within the group of customers they’ve already established contact with. “It’s unfortunate because we don’t want to over-index on these customers, but at the same time we feel it’s better than nothing,” says Yury.
Another challenge with enterprise customers is that the CS team tends to be a bit more protective over these relationships and doesn’t want the product team to reach out to customers directly at the wrong time. “For example, our CS team does not want us to reach out to specific customers when there’s an unresolved support issue,” explains Yury.
Compared to other customers, Yury says it’s been harder to get a consistent flow of interviews within the enterprise segment. “Because our outreach here is still heavily manual, some weeks we have too many interviews (pure luck) and other weeks almost none.”
HiveMQ is currently working with Orbital to address these concerns for email outreach to enterprise customers. The tool has the ability to take CS-defined rules to block specific accounts and users, so that trios automatically respect those constraints and safely recruit without having to go ask permission on a case-by-case basis.
And because Orbital handles both the outreach and the scheduling, it can dynamically change the number of emails sent throughout the week to adapt to how many interviews have been scheduled, to get as close to the target as possible without sending customers to a scheduling page with no remaining availability.
In the interim, the HiveMQ team has been using Orbital’s shareable links that basically give them the in-app intercept flow as a link that they can share in Slack, support ticket responses, and also have CS send out for them. “That allows us to automate everything but the outreach itself,” says Yury.
Key Learnings and Takeaways
Yury shares a few of his key learnings and takeaways from HiveMQ’s continuous interviewing journey.
Automating recruitment is key
“Until we automated recruitment, we were not truly able to practice continuous discovery,” says Yury. “It was a huge unlock, and it was super simple to set up for us because we found Orbital and had enough users.”
Until we automated recruitment, we were not truly able to practice continuous discovery. – Tweet This
Work strategically with CS and sales
“While ‘riding along’ with CS or sales can be helpful for customer exposure, it doesn’t really allow you to interview,” says Yury. He adds that in HiveMQ’s B2B setting, CS wants to help but has their own job and has better things to do than act as the product team’s automation. Having a tool that makes customer outreach “safe” is critical to establishing trust with the CS team.
Experiment with different recruitment channels, intercepts, and interview lengths
“We couldn’t reach every user we needed to through the same channel—in our case we decided that we needed both in-app and email recruitment,” Yury explains. Similarly, small tweaks to the in-app intercept (e.g. targeting, screening, personalization, copy, incentive) can make a huge impact on conversion rate. They also found that reducing the interview length from 30 minutes to 1 hour significantly increased their conversion rate. Yury adds that while they haven’t needed to experiment with incentives yet, Orbital does allow you to track that within the tool.
Scheduling interviews within a shorter window reduces the likelihood of no-shows
“When customers schedule a week or two in advance, the likelihood that they don’t show up for the interview is significantly higher than if they schedule for the current week,” says Yury. The HiveMQ team used Orbital to set a “sliding window” for scheduling, so users can only schedule based on availability within the next five days, for example.
When customers schedule a week or two in advance, the likelihood that they don’t show up for the interview is significantly higher than if they schedule for the current week. – Tweet This
Do you need help with finding customers to talk to week over week? Check out our brand new self-paced course: Customer Recruiting for Continuous Discovery.
