Do agent rating sites make the grade?

Former New York City mayor Ed Koch used to stand on street corners asking New Yorkers, “How’m I doin’?” It was a great PR idea, but did it really provide him with an accurate picture of public sentiment?

Fast forward 30 years: If the current mayor wants to know what people think of his performance, he can find out here. Or here. But each site measures different things in a different way, so it’s still tough to really know if any of those performance reviews are more reliable than any others.

There have been plenty of articles talking about the need/value/inevitability of online real estate agent reviews and how more agent ratings would (in theory, at least) lead to a higher overall level of professionalism. Indeed, many different agent review systems have appeared on the scene during that time, with more to come. But with so many systems out there, how can you tell which ones are really doing a good job of identifying and promoting professionalism?

Back in 2011, PAR started considering whether the association should create an agent review system of its own. We put together a member task force that spent lots of time discussing this general issue, identifying various metrics, dissecting some existing rating systems, and eventually sketching out their ideas for the ideal agent rating system.

However, after considering the political and practical hurdles to launching such a system from within the Realtor® organization, the task force recommended not moving forward with a new rating system. Instead, PAR produced a report card evaluating existing rating systems against some of the main criteria considered by the task force. We will be helping members and their clients to understand which existing systems are the best fit for their needs.

PAR looks at the publicly available information for each system and rates each of the 10 criteria as one star, a half star or no stars, based on the task force’s sample rating criteria. For example, our criteria suggest that the best rating systems focus on obtaining and posting ratings from actual clients or customers. On that basis, a site earns one star if it verifies the business relationship before posting; a half star if it requires self-identification as a client (but no verification); and if you have a site that is open to anyone without identification… no stars for them.

Each system gets a total rating of somewhere between 0 and 10 stars, along with a report card that explains each score.

Our first release includes report cards for 10 agent rating systems. Not every existing system is included, but we had to start somewhere, and others will likely be added over time (though we won’t waste your time with any of the more scammy sites.)

Is our methodology perfect? No. And it will be no surprise if members, consumers, and possibly some of the rating systems disagree with our criteria and evaluations. But we expect this to be a living and developing project that gets tweaked as time goes on, and we look forward to delivering regular updates.

We welcome your comments below on the idea behind the project and any thoughts you may have about the broader question of promoting and improving the professionalism of agents.