Those of you who know me will agree that I travel quite heavily. I am also not a fan of staying at crappy hotels, so I am a frequent user of hotel review sites such as TripAdvisor (and so is my assistant!). Despite claims that they're clean, I have always suspected that a large number of the reviews in such sites are fake. It simply makes sense: paying a guy $10/hour to write fake reviews is well worth it when the return is many more visitors to your hotel.
I started using TripAdvisor in ~2002. Although I have no proof of this, I feel that it's gotten worse over time. This is probably due to an unfortunate predicament of popular social review sites: they start crappy (because they don't have enough reviews), then they improve for a period of time while they gain more traffic, and finally they start getting worse because, due to their popularity, it's worth money to manipulate them. The sweet spot for us readers is when the sites have enough users for the data to cover most choices, but not enough users for businesses to care about manipulating them.
TripAdvisor has recommended a number of great hotels for me, such as the one we went to near Cancun:
But it has also recommended at least 4 that pretty much sucked. I didn't feel like doing my own work this morning, so I spent it writing a simple script to find evidence of fraudulent reviews on TripAdvisor by using the following conjecture: fake reviews will tend to come from "users" who have made very few other reviews. The intuition is that, if your goal is to increase the average rating of a specific hotel, it takes too much extra work to write reviews of other hotels just for the purpose of making your account seem more real.
So my script does the following: it downloads all the reviews for a hotel, and then for each review it measures how many other reviews that user has written. I ran my script on 25 hotels that I've recently stayed at, 4 of which I would classify as bad, and the others as good. All of the hotels have a high rating on TripAdvisor.
To my surprise (things rarely work when I do them), the data perfectly matched my hypothesis: the fraction of 5/5 reviews made by users who made no other review is significantly higher for the crappy hotels than for the good hotels. (For the scientists out there: the difference is statistically significant.) In fact, if you remove reviews made by accounts that have made no other reviews, all the good hotels keep their high rating, whereas the bad hotels drop by a lot.
I'm not in the business of outing fraudulent hotels, so I won't publish their names here, but I will say that I wish TripAdvisor implemented some form of Karma system. For now, I will make good use of my script every time I travel.
Oh I also know of an amazing restaurant recommendation site, but I won't link to it here because it's currently in that sweet spot with enough but not too many users!