“It’s a completely subjective process,” people whisper. “The top ten is just one person’s opinion,” others say. No doubt, Cincinnati Magazine creates a lot of buzz every March with their Top Ten Restaurant list. I’ve heard the accusations, read the assumptions and want to clear a few things up about how the list is determined.
I’m in no way affiliated with the magazine, by the way. And if the list were up to me, it’d be jiggled around a bit and other restaurants would be added. But the point here is to get down to the facts so we can at least have intelligent debate. And the facts may surprise you …
Here’s how Cincinnati Magazine really determines Cincinnati’s Best Restaurants every year.
- Eight months prior to the March issue, dining editor, Donna Covrett, determines a master list of approximately 25 restaurants. The master list is based upon her dining experiences throughout the year.
- A dining schedule is organized with a group of approximately 12 diners that represent a cross section of food knowledge and dining experiences. (None of them are employees of the magazine.)
- They dine at all of the restaurants over approximately a 6-8 week period. The groups have a minimum of 2 people besides Covrett, but usually 3.
- Reservations are never made under Covrett’s name.
- If it is a restaurant where she is known (there are usually about a half dozen or so), the unknown diners are sent in ahead with instructions to begin without her. She joins them 15 or 20 minutes later. If she is detected, it is determined in the post-meal discussion whether or not there was any discernable difference in food or service. If there was, it is reflected in the final tally.
- As much food is ordered off the menu as possible (generally 3-4 courses each), without duplicating any dishes. Unless there are specific dietary requirements, everyone is required to taste every dish. If anything such as wine service or craft cocktails are part of the restaurantâ€™s profile, that is ordered as well.
- No freebies are accepted. If a chef sends out a dish that wasnâ€™t ordered, the diners ask the server to put it on the bill.
- After dinner, the group retires to another site for discussion and their experiences are plugged into a rating sheet.
- The rating sheet consists of four categories: food, service, atmosphere, and miscellaneous. There are five criteria in each. The rating sheet itself was created with the help of a statistician and psychologist (two data gathering professionals).
- Each criteria has a Leichert scale of points. The points for both individual criteria and categories are weighted. In other words, the more important the criteria, the higher its point value.
- With the other diners speaking first, each of the criteria are discussed and points are determined.
- Lots of notes are taken and nuances discussed but ultimately the top ten list is simply determined by whoever garnered the most votes. If there are ties (and there always are), they return to the restaurants for a 2nd or 3rd time.
That’s it. Though it has been printed before, I was surprised to learn how much effort is put into being as unbiased as possible. So, what do you think about the process?