A year ago we wrote about how an unknown application tracking system (ATS) from South Africa was the #2 rated ATS on Capterra. The TL/DR of that article is that a very small company was able to manipulate their reviews in order to seem like a global player that could compete with the best ATS software.
Of course, you’ve probably seen the many scams that companies have perpetrated on sites like Amazon, Yelp, Glassdoor, etc to enhance their online reputations and get more customers (or hires) via reviews.
While many of these various strategies are well documented, we ran into one that is slightly more novel and specific to software reviews that we wanted to make our audience aware of.
The $1 million rule
On SSR, we generally don’t review solutions that have less than $1 million of annualized recurring revenues.
After a company has $1 million in revenues, they are much more likely to have a fully developed product, at least decent customer service, a sales team that can handle different types of customers, and they are much more likely to be around in 5 years vs a company with $200k in ARR.
We chose the $1 million cutoff as a VC friend had done an extensive analysis of their portfolio spanning a few decades and hundreds of companies. They found that once a company hit $1 million, they have a MUCH higher chance of making it.
“Fake” reviews on software review sites
A few weeks ago a vendor emailed us asking to be reviewed by our team. Their basic pitch was that they had over 250 reviews, almost all of which were 5 stars.
I asked how many customers they had, and what their revenues were and was a bit perplexed at the answer. They had <150 customers, and around $600k in revenues. Since they weren't at the $1 million marker, we started digging into their reviews to better understand if we were missing something and should feature them on the site.
The first question we had was: how could you have 250 reviews if you have <150 customers?
Well, this company had asked their customers for reviews, but mostly asked the employees who use the software to review it as buyers. Definitely a smart marketing tactic, but in my opinion a bit devious.
First, the users of the software have a very different perspective vs buyers. Their "love it!" is relevant to the buyer, but doesn't give the full story. Secondly and more importantly, if your review data is what you lead with in your pitch, you are marketing yourself in a misleading way.
To give you a sense of how much, Greenhouse Software is rumored to be around $80 million of revenues (133x the revenues of this company!) and has only 2x the number of reviews. They’ve also been around for 10 years, vs 2 years for this company.
Workday is a $60 billion market cap company and has 730 reviews on Capterra….only 2.9x this $600k revenue company!
If you were to read the 250 reviews, you would start to realize many of them are irrelevant to the HR team buying the software (except for understanding the employee experience - important but just part of the puzzle).
Of course, who has time to read 250 reviews? What we’re most likely doing as buyers is internalizing that number as a point of social proof to justify a purchase.
And, if your brain has internalized that hundreds of reviews = vendor at scale, you’ve been duped by this inflated review count from a tiny company!
So, just like we should dig a few levels deeper on many vendor claims (and generally buy software like a toddler), having a deeper understanding of what the true signal of a company’s online reviews are is essential to the buying process.
