The review process described below has been in effect on eBiz Facts since April 2021. Reviews published before April 2021 followed a different process.
The Goal
The goal of our review process is to ensure that the overall rating displayed for each course is fair and accurate.
Fair and accurate ratings help our readers make better informed decisions about which courses to invest in and which courses to avoid.
To achieve this goal, we research each course according to a strict process and combine our findings with reviews from real students.
Each review on eBiz Facts goes through three stages…
Stub Review
This is a short article about a particular online business course, providing basic info such as description, price, and alternatives.
This information is based on brief research by a member of the eBiz Facts team.
Once a stub review is published…
- The overall rating for the course is set to incomplete.
- Students of the course can submit their reviews via links on the review page
Editorial Review
This is a more in-depth review of a particular course.
The eBiz Facts team will spend several hours researching the course and going through the training materials (assuming we have access).
We end up with a 10-50 page document filled with notes about the course…
Based on our research, we rate the course on 31 separate factors, and our proprietary algorithm uses those factors to calculate an editorial review score.
We then edit and publish our research as an editorial review, replacing the earlier stub review.
(At this point we also send the review to the course creator for feedback. More on this below.)
Complete Review
A review is considered complete when we have published…
- Our editorial review of the course.
- At least 3 student reviews above a certain trust threshold.
At this point, the overall rating displayed for the course is determined by combining our editorial review score with the weighted average student review score.
As we publish more user reviews above a certain trust threshold, our editorial review score has less influence on the overall rating.
Ultimately student reviews have the biggest impact on the overall rating, since students know best how good (or bad) a course is.
- Example of a complete review: The Affiliate Lab Review
🤔 Frequently Asked Questions
What are the 31 factors you use to evaluate each course?
We evaluate each course based on factors such as…
- Refund policy
- Hidden costs
- Frequency of updates
- How the course is marketed
- How comprehensive it is
- Fixed cost or recurring?
For example, we evaluate the refund policy of a course like so…
- Good = minimum 30-day refund period, no questions asked.
- Decent = less than 30 days and/or a conditional refund policy.
- Bad = policy is too complicated, doesn’t exist, or isn’t honored.
That evaluation converts to a number, which plugs into our algorithm to calculate the editorial review score for the course.
Not all evaluation factors have the same impact on the editorial review score.
For example, the refund policy of the course is given more weight than whether or not there are subtitles on the training videos.
How often do you update your reviews?
It depends.
The overall rating of a course is updated whenever a new student review is published.
As for the editorial review of a course, a few things can trigger an update…
- Feedback from the course creator
- Feedback from students
- A significant update to the course itself
How do affiliate links work in the reviews?
For every course we review, we apply to be an affiliate (if an affiliate program exists).
If approved, the main call-to-action (CTA) on the review becomes an affiliate link to the course.
For example…
If we are not approved or no affiliate program exists, the main CTA becomes a link to view alternative courses…
With this setup, there is an incentive for each course creator to establish and maintain an affiliate partnership with us: such a partnership results in more leads flowing their way.
However, whether or not we are an affiliate for a course has no impact on that course’s rating. We evaluate each course using the same process, regardless of partnerships.
Often this will result in us earning far less money than we could otherwise.
For example, one of our top-rated Amazon FBA courses is Proven Amazon Course…
Proven Amazon Course – Key Points
With such a low price point, we only earn a few dollars for each new student we refer to that course.
Meanwhile, many other review sites highly recommend an FBA course called Amazing Selling Machine, which pays affiliates upwards of $1000 per referral 😳
But, thanks to our strict review process, we don’t rate that course highly at all…
Amazing Selling Machine – Key Points
Sure, we are still affiliates for that course, and we will still earn a commission if someone signs up for it via our affiliate link.
But that is unlikely to happen given the rating.
And we’re totally okay with that.
In the long run, we’re confident that things will work out financially so long as we do our best to review each course accurately and fairly and never lead our readers astray.
What if you get something wrong?
Despite our strict and lengthy review process, sometimes we do make a mistake and publish inaccurate info about a course.
We’re only human, after all 🤷♂️
But we make every effort to ensure that such mistakes are found and fixed ASAP.
A few ways we accomplish this…
- We email a link to our editorial review to the course creator as soon as we publish (and before any promotion), asking if they have any feedback or corrections.
- We quickly update the review and rating if the course creator provides compelling evidence that we made a mistake.
- If the issue is more a difference of opinion with the course creator, we offer to quote them in the review so they can have their say.
- We search online for students of the course and invite them to leave an honest review. We also ask the course creator to send students our way. If the feedback from students conflicts with our findings, we investigate further and update the editorial review as needed.
Unfortunately, some course creators find all the above to be insufficient, and simply threaten us with legal action if they don’t like our review 😱
Here’s an email we received from a course creator after we sent along a link to our review, asked them to let us know if anything was factually incorrect, and noted that we’d be happy to update any errors…
We have received several such threats over the years and have never removed a review in response.
Usually what happens in these situations: the course creator realizes that such bullying tactics are ineffective, and they eventually tell us what parts of the review they take issue with.
We evaluate their feedback and update the review if appropriate 👍
How do students submit a review of a course?
There are multiple links to submit a review on each course page.
For example…
Click on such a link and you’ll be taken to a review form, where we ask you to provide information about yourself and your experience with the course.
How do you ensure student reviews are legit?
We do a few things to filter out fraudulent student reviews.
First of all, each reviewer must verify their email address before they can submit a review.
They must also provide a link to at least one active social media account, which we use to verify that they are a real person.
(A reviewer can opt to have their review published anonymously, but we still require them to provide a link to a social media account so we can verify their identity on the back-end.)
Lastly, we calculate a trust score for each student review, based on a number of factors.
For example…
- If they provided proof of purchase
- Time in the course
- Past experience with online business
- Any contradictions in their responses
The better the score, the more weight that student review will have on the overall rating of the course.
If a review seems suspicious or doesn’t provide enough insight, we follow up with the student via email and/or social media to ask for more info.
Through this process, we end up rejecting 30-40% of all submitted reviews.
We do all this to avoid the Trustpilot trap, whereby even the most meaningless / spammy reviews get approved and apparently have equal impact on the overall rating of a course.
For example…
Why are some student reviews marked as biased?
You may see the following message atop some student reviews…
This reviewer has a business relationship with the course creator or a competitor. Not included in overall ratings.
We mark students as having a relationship with the course creator if they answer YES to any of these questions…
- Are you an active affiliate of this course (or a similar course), regularly recommending that other people buy it through your affiliate link?
- Are you the creator of this course, their employee, or business partner?
- Are you the creator of a similar course, their employee, or business partner?
We still publish reviews from students who answer yes to one of those questions – so long as the review is insightful – but their rating doesn’t factor into the overall rating of the course.
We do this to ensure the integrity of our reviews.
Otherwise it would be too easy for a course creator to, for example, get their army of affiliates to submit positive reviews and boost the overall rating of their course.
Can course creators respond to student reviews?
Yes.
A course creator can send us their response to any student review and we will publish it so long as it adheres to our editorial standards.
(We never provide course creators with identifying or contact info of a reviewer unless we receive explicit permission from the student to do so.)
Can course creators remove student reviews?
A course creator can request that a student review be removed, but we rarely oblige such requests.
The main exception is if the course creator provides proof that a student review contains misinformation.
Questions or Comments?
If you have questions or comments about our review process, contact us here.
