A fascinating new research paper from a PhD student and two professors from the management and computer science departments at Boston University that focuses on online consumer reviews in the Airbnb era gives a nascent peek into understanding and interpreting nuances of user-generated ratings in the context of the sharing economy.

The paper, titled “A First Look at Online Reputation on Airbnb, Where Every Stay is Above Average,” was authored by Georgios Zervas, Davide Proserpio, and John W. Byers and released on Friday last week. The authors collected reviews on about 600,000 Airbnb listings and compared the “star ratings” on them with equivalent ratings on TripAdvisor. They also considered several thousand properties that were listed on both platforms and hence easier to compare variance in review patterns.

The key question the authors tried to answer with this research: are the reviews on sharing economy sites — essentially open marketplaces — different than those on traditional review sites like TripAdvisor?

The topline results are fascinating:

  • Nearly 95% of Airbnb properties boast an average user-generated rating of either 4.5 or 5 stars (the maximum); virtually none have less than a 3.5 star rating. Compare this to TripAdvisor, where there is a much lower average rating of 3.8 stars, and more variance across reviews.
  • For the cross-listed properties on both Airbnb and TripAdvisor, the paper found that more properties receive the highest ratings (4.5 stars and above) on Airbnb than on TripAdvisor.

Some reasons put forward for this difference: perhaps sociological factors are at work,
whereby individuals rate other individuals differently or more tactfully (hence the positive reviews), than they rate firms such as hotels, independent of the platform.

Airbnb has a two-way review system, where people renting these accommodations also get ratings from the property renters, this there is more incentive on both sides to stay civil, and hence positive. Or as the paper puts it, “bilateral reviewing systems, as used in Airbnb, inflate ratings by incentivizing hosts to provide positive feedback so they are positively judged in return.”

And as New York Times puts another spin to this theory in a story: “If Airbnb guests seem too critical they might get turned down by future hosts who worry they will be too demanding. Who wants a cranky guest complaining about the noise at 3am? A better approach is just to shower everyone and everything with praise.”

Download (PDF, 2.6MB)

Photo Credit: An accommodation in San Francisco on Airbnb, lot more positive reviews than reviews for same property also available on TripAdvisor.