Many times when buying HR software, a vendor will offer to put you in touch with a reference customer as a way to showcase how great their offering is.
While getting the opinion of someone in our network is always preferable, that’s not always possible. So, we’re left with the vendor’s contact.
Unfortunately, the vendor is incentivized to put us in touch with someone who has a positive view of their offering, and so this reference customer is probably someone who is either not going to be fully truthful, or simply has a very high opinion of the vendor for reasons that may or may not be relevant to us.
This can be a warranted opinion, but most times there is very little signal as to whether or not you should buy this product when talking to these reference customers. This is true despite a feeling you may have of “well, this person is very similar to me and really likes this so maybe it’ll be good for our company.”
The Right Questions Make the Difference
Reference calls can be much more effective when you ask the right questions. Simply asking for a customer’s experience with a product will allow them to stay high level and focus on the positives.
They clearly like this product or else they wouldn’t take time out of their day to do this favor to the vendor. Of course, it is worth noting here that sometimes there is a financial incentive for the reference customer - dinners, trips, gift cards, or they may even have equity in some cases (especially true for “industry thought leaders”).
Here’s a pro tip: get as specific as you can with your questioning. This is true when talking to a vendor intro’d reference. It’s also true when talking to someone in your network.
We see a lot of people asking “what do you think about XYZ vendor?” and getting shallow answers like “LOVE IT!” or “Stay away!”
I’ve seen these happen online and offline, and I’ve probed many times into what’s behind the gut reactions. You’d be surprised how many times the people giving effusive answers have never even used the product!
Of course, someone may love or hate a product for reasons that don’t pertain to you. I talked to a very smart CEO last week who HATES their ATS. Why? The email templates don’t allow rich media, otherwise he’s a fan. That one feature isn’t relevant to everyone, but you’d only know that’s the source of his disdain if you ask for the details like a toddler would.
10 Questions We Recommend
When you’re getting on the phone with a vendor customer reference, here are a few questions to keep in your back pocket. They’re designed to find the signal through the noise.
Please beware, they are very much trying to get the negatives out. So, your call may leave you with a poor impression of the vendor. It’s up to the astute buyer to understand how this information fits into the entire picture you’ve created throughout your research process.
- What’s the one thing you’d change about Vendor X?
- Tell me about the last time Vendor X’s technology broke.
- How surprised were you by your last QBR/annual checkin?
- When’s the last time you referred a friend to Vendor X? What was the time before that?
- Tell me about the last time you had to deal with customer service. What went wrong and what was the response?
- Tell me about a time that you suggested a product improvement - was it implemented? How long did it take?
- It’s two years from now and you aren’t a customer anymore, what happened?
- Who’s your main point of contact at the company? Was that different from the sales process?
- If you look at your calendar, what date was your first meeting as a customer, and what was the go-live date?
- This is really nice of you to do these calls, do you do many? Does the company at least fly you out for conferences or throw you a gift card? So, you don’t have any financial tie to the company?
Here’s another pro tip: pick one thing you really care about (an integration, a workflow, a feature) and ask a very concrete question around it, ie how many emails per month do your recruiters send through the automation module and what is the response rate to those emails?
You Get More With Sugar Than Salt
Some of these questions can start to get aggressive. It’s important to remember the person on the phone is most likely doing the vendor, and you a favor.
You’re not trying to out them as a paid plant by the company, you’re trying to figure out if this is the right vendor for your organization. So, it’s important to stay cordial and thank the person for helping you out.
If you’re at this point in the process, there is a high likelihood that you’re going to sign the contract. But, asking the right questions can help decide between two vendors, or illuminate risk areas that you hadn’t yet identified.
