“It’s a completely subjective process,” people whisper. “The top ten is just one person’s opinion,” others say. No doubt, Cincinnati Magazine creates a lot of buzz every March with their Top Ten Restaurant list. I’ve heard the accusations, read the assumptions and want to clear a few things up about how the list is determined.
I’m in no way affiliated with the magazine, by the way. And if the list were up to me, it’d be jiggled around a bit and other restaurants would be added. But the point here is to get down to the facts so we can at least have intelligent debate. And the facts may surprise you …
http://fhaloanmichigan.org/fha-loan-in-new-haven-mi/ Here’s how Cincinnati Magazine really determines Cincinnati’s Best Restaurants every year.
- Eight months prior to the March issue, dining editor, Donna Covrett, determines a master list of approximately 25 restaurants. The master list is based upon her dining experiences throughout the year.
- A dining schedule is organized with a group of approximately 12 diners that represent a cross section of food knowledge and dining experiences. (None of them are employees of the magazine.)
- They dine at all of the restaurants over approximately a 6-8 week period. The groups have a minimum of 2 people besides Covrett, but usually 3.
- Reservations are never made under Covrett’s name.
- If it is a restaurant where she is known (there are usually about a half dozen or so), the unknown diners are sent in ahead with instructions to begin without her. She joins them 15 or 20 minutes later. If she is detected, it is determined in the post-meal discussion whether or not there was any discernable difference in food or service. If there was, it is reflected in the final tally.
- As much food is ordered off the menu as possible (generally 3-4 courses each), without duplicating any dishes. Unless there are specific dietary requirements, everyone is required to taste every dish. If anything such as wine service or craft cocktails are part of the restaurant’s profile, that is ordered as well.
- No freebies are accepted. If a chef sends out a dish that wasn’t ordered, the diners ask the server to put it on the bill.
- After dinner, the group retires to another site for discussion and their experiences are plugged into a rating sheet.
- The rating sheet consists of four categories: food, service, atmosphere, and miscellaneous. There are five criteria in each. The rating sheet itself was created with the help of a statistician and psychologist (two data gathering professionals).
- Each criteria has a Leichert scale of points. The points for both individual criteria and categories are weighted. In other words, the more important the criteria, the higher its point value.
- With the other diners speaking first, each of the criteria are discussed and points are determined.
- Lots of notes are taken and nuances discussed but ultimately the top ten list is simply determined by whoever garnered the most votes. If there are ties (and there always are), they return to the restaurants for a 2nd or 3rd time.
That’s it. Though it has been printed before, I was surprised to learn how much effort is put into being as unbiased as possible. So, what do you think about the process?
{ 11 comments… read them below or add one }
WOW, thats way more effort than I was expecting, but Donna is very thorough.
When does the list come out?
Donna is the only person in town who always dines at restaurant multiple times before reviewing them– I have a tremendous amount of respect for her and her process.
no, I’m quite sure she’s not the only one.
Thanks for posting this. My husband and I use the Cincinnati Magazine list a lot when picking where to go when we want to eat out. We often wondered if we were responding to a popularity contest or if the restaurant owners influenced the list. Knowing how objective the process is makes me feel much better. I would also kind of think that restaurant owners will pay more attention to consistency since we now all know the process. Cincinnati Magazine should be commended.
Barb, I completely agree! I am often skeptical of lists such as these because it is easy for the lists to be biased by a variety of factors. Knowing Donna goes to great lengths to ensure that as little bias as possible influences the final list is comforting.
Donna give Orchids restaurant the first place on the top 10 list because the Chef gives her FREE FOOD AND DRINKS AT THE BAR.
She as NO INTREGITY!!!!!!
No, Cincinnati Magazine does not accept free food. Discourse welcome, but make sure you check your facts.
Super cool about the way the ratings are calculated:
# The rating sheet itself was created with the help of a statistician and psychologist (two data gathering professionals).
# Each criteria has a Leichert scale of points. The points for both individual criteria and categories are weighted. In other words, the more important the criteria, the higher its point value.
I’m glad to hear the reviewer (an expert in her field) goes to other experts for what she needs. In this great recession-web 2.0-diy world, it seems people have a false self confidence and disregard experts, or don’t have the time or money to use them. The beauty of specialization of labor still lives!
i love this. for us “dummies” reading about food, i really helps to understand the behind the scenes in this process. It adds to the overall credibility of the publication. Everyone should read this before diving into that issue of the magazine…
Funny, the above process has been detailed in the Mag before but apparently lots of people skip over it. I am told it is one of the most common questions Donna is asked in interviews as well. Just goes to show how many times a message needs to be repeated before it sinks in.
the best restaurants usually serve super delicious foods at a very low price ~