Short-Term Rental Data – Behind the Complexity and the Nuances

Much ink has already been spilled (including our own) over questionable data that showed plunging Airbnb host revenue in several big cities. A tweet about it took on a life of its own, leading to a slew of news articles, social media posts, and investor concerns.

Other data providers said the tweet was wrong: Yes, host revenue was down a little, but nowhere as much.

With so much riding on this data, we wanted to understand how these firms gather, sort and process their numbers. 

While most data companies scrape the internet for new listings, most of them also source data from integrations with Airbnb, Vrbo, Expedia, property managers, and individual hosts. They then apply proprietary algorithms.

For instance, AirDNA’s machine learning model is good at identifying blocked versus booked nights. PriceLabs prides itself on nailing hyperlocal data that can tell hosts how other listings are priced in their neighborhood. Beyond is testing pricing based on search data.

“All these companies are attempting the same kind of problems and they are thorny problems to solve,” said Drew Patterson, co-founder and CEO at travel startup Thermal. “Running data sets around short-term rentals is a complicated exercise, and it’s hard to get right. A lot of things can compromise analysis.” Patterson co-founded Transparent Intelligence, which was acquired by OTA Insight in March 2022.

What Do We Want? Standardization. When Do We Want It? Yesterday

There are a couple of reasons why analyzing short-term rental data can be complicated. Chief among them is the lack of standardization. The typical parlance for anyone using short-term rental data would be RevPan or revenue per available night, comparable to revenue per available room used by hotels. It’s key to calculating the earning potential for a property based on its availability and daily rate. 

“Where the industry needs to come to is using standardized metrics,” said Richie Khandelwal, co-founder of PriceLabs, a revenue management platform for Airbnb hosts and vacation rental owners and property managers. “I have no idea what RevPal is, what are active listings? Are they available to book or block?” said Khandelwal, referring to the AllTheRooms data that Nick Gherli claimed he used — and later said he repurposed — in the viral tweet, which depicted revenue per available listing. Other data vendors vociferously disputed those numbers.

“We all (companies) get the same data, but how we use it is different,” he said. With a headcount of 30 spread across its engineering and data science teams, PriceLabs started in 2014 providing dynamic pricing options for vacation rentals and short-term rentals. The company’s clients today are individual hosts as well as property management companies.

“We did not know about the vacation rental market when we started out,” Khandelwal said. “We thought the large part of the market was Airbnb, it took us a couple years to understand that the traditional vacation rental market has existed outside of Airbnb and is significantly large.”

The fragmentation Khandelwal is referring to is another factor that contributes to non-standardized data and metrics. There are a number of vacation rentals that have survived without needing to be listed on Airbnb, Vrbo or Booking.com — and operate successfully through direct booking — which makes it harder to gather data on them. 

AirDNA, which claims to have the world’s largest dataset for short-term rentals also claims 95% coverage and 95% accuracy. 

“It’s hard for any company to get 100% accuracy and coverage because there are some operators that don’t distribute their listings online,” said Jamie Lane, chief economist and senior vice president of analytics at AirDNA.

“Short-term rentals industry is still fragmented and the data companies all have different strategies and integrations to gather accurate data on them,” said Jeffrey Breece, manager of revenue management and data science at Beyond. “Everyone is having to do so many integrations, and it’s a big challenge, as not much is standardized.” 

Integrating with individual hosts, online travel agencies and property management companies has become imperative for data vendors to stay competitive. “There is some measuring contest when it comes to integrations,” Breece said. “You won’t survive in this biz without it. Integration is getting easier, and has expanded over the last couple of years with a lot of nuance to it.”

Beyond prides itself on two-way integration where it pulls information from a database but also pushes information to it. Put simply, Beyond will provide data to clients as updates become available versus its clients always querying for data. 

Secret sauce: Transparency

The jargon and the methodology aside, the data companies all share one tenet: transparency. That’s ultimately what’s important in ensuring accountability as well as trust.

AllTheRooms was purportedly the source of the data in the tweet that showed plunging Airbnb host revenue. It has not spoken publicly or engaged with journalists seeking to confirm the data.

It’s also a question of how often a vendor’s data is put through the ringer and by whom. AirDNA, for instance, counts hedge funds tracking travel companies as its clients. Lane said that they’re particular about digging into the accuracy of its data. 

“Our data engineering team is larger than the entire staff of other companies. We invest massively in our technology,” Lane said. “Other companies might say they have equivalent data to us, but it’s just not true. Ours is of much higher quality which is put to test often.”

Do Your Best, Let Others Do The Rest

Discounting for the default setting of any business to be competitive, all the companies who were contacted for the story agreed on one thing: Whose product you choose to buy depends on what you need. 

Lane of AirDNA conceded that KeyData has good benchmarking tools. Jeff Breece at Beyond tipped his hat to the top echelon of property managers that are investing in data science and building cool tools. Richie Khandelwal said that his company uses KeyData for some functionalities. 

And users of these software products, too, are aware and do the comparing-and-contrasting as needed for their business.

“Each of them (data providers) have different algorithms and signals,” said Liz Marie, chief product officer at Revedy, which offers consultancy services and underwriting for short-term rentals as an investment asset. “We use AirDNA, we are discussing using Transparent as well. The thing to me is that you have to understand the assumptions being made and be clear about it — What does revenue mean? What fees does it include ? You have to know what you’re looking at.”