Meta Says Its Newest AI Model Rivals ChatGPT. We Tested it for Trip Planning

Meta AI last week released the newest version of its generative AI model, which it claims outperforms that of OpenAI in nearly every metric.

While those metrics refer to technical measures, Skift is interested in how these models perform for travel. 

A Skift test in 2023 of Meta AI’s chatbot — based on the second version of its AI model — showed the tool was lacking, especially compared to competitors. We tested the new version to see how it performs now. 

The new model, called Llama 3.1 405B, is accessible in the U.S. via WhatsApp and the website Meta.ai.

Unlike ChatGPT, the Meta AI chatbots are not yet capable of reading images (great for translating menus) or listening to audio (for translating speech). And unlike Gemini — which has a direct connection to Google Flights — Meta AI does not have as reliable of a connection to flight information, based on a Skift test.

Meta says the model is also “open,” which means its code is available for software developers to download and adjust as needed. All of the models powering ChatGPT by OpenAI, Gemini by Google, and Claude by Anthropic are closed. 

The tests below were completed through Whatsapp. 

Things to Do 

Based on a prompt for activities focused around hiking and cultural experiences, the chatbot gave multiple suggestions for visits in and near Vancouver.

It wasn’t in the form of a traditional itinerary. They were grouped according to the type of the activity along with a suggestion for which day to complete each of them. 

Most available trip planners don’t take logistics into account, but a couple of tests seem to show that Meta AI is getting better at that. For example, it suggested visiting Cypress Mountain and Stanley Park on the same day, which would make sense because the route to the mountain cuts through the park. And it suggested visiting Garibaldi Lake, a challenging hike about 90 minutes away from Vancouver, as the only activity for that day. Also notable, the chatbot didn’t make suggestions for visits that are too far outside of Vancouver.

Finding Restaurants 

The chatbot did provide some solid suggestions for restaurants in the area. Two of the suggestions, however, are permanently closed: West Restaurant and Heirloom Restaurant.

The chatbot did not provide links to any of the restaurant websites. It began typing each of them out when asked, but once the response was finished, each of the URLs were replaced with “link unavailable.” 

That the means the user can take restaurant suggestions from Meta AI but then needs to complete a traditional search to learn whether it still exists and get other details.

Finding Flights

A flight search didn’t work well. 

The prompt was for a flight from New York City to Vancouver in September. The chatbot said it could not find an option. 

It did, however, provide a link to a Bing search that showed options for a week in the middle of September. But at that point, it makes sense to just begin the flight search in a traditional search engine.

Finding Hotels 

The hotel search was even less useful than the flight search. 

The prompt was for a hotel or short-term rental near downtown Vancouver. It gave a couple names of properties, paired with inaccurate pricing information without links to those places. 

It did provide a link to a Google search for a hotel tonight, not in September. Again, it would be easier to just complete a traditional search to find a hotel. 

Packing List

When opening a new conversation with Meta AI, the chatbot makes several suggestions for topics to ask about. One of them is for a packing list for an upcoming trip. 

After selecting that option, it asks for the destination, number of days, and what type of trip it is. For our test, we responded: “Vancouver, one week, hiking and city.” It then gave suggestions that made sense for that type of trip. 

Most of the answers are common sense, but it could be useful for those who tend to be forgetful when packing.