Both Anthony Kaufman and Anne Thompson have written about this new movie review site called Criticker. Their descriptions intrigued me, and today I finally popped over to their site to check it out.

The idea is pretty straightforward. You rank movies on a scale from 1-100, they plug your rankings into their "Taste Compatibility Index (TCI)," and then you get a list of movie recommendations and film critics whose taste matches yours. It's a fun toy and a good way for movie buffs to while away a few minutes or hours.

But being obsessive-compulsive (and stuck at work and bored), I decided to really run the site through its paces. Specifically I wanted to test this claim: "The more films you rank, the more accurate your TCIs will be!"

Methodology:

For my experiment I paused and recorded data after ranking 10, 25, 50, 100, and 250 films. I kept track of the critics that Criticker told me were my kindred spirits and the movies that Criticker recommended to me. As the number of films I ranked increased, I gradually followed more recommendations, but I tracked the same base group of films. For each film I noted Criticker's Probable Score Indicator, my own score, and the difference between the two.

Predicted Outcome:

As I rank more and more films, Criticker's recommendations should become more accurate and a stable group of kindred spirit critics should emerge.

Data:

Results:

Critics:

I'm not as concerned here with the second predicted outcome (the emergence of a stable group of kindred spirit critics) because the TCI depends on the number of films you've seen in common with the critic in question. Even after ranking 250 films, that number is still typically very low. John Hartl is number one on my list, but that's based on only 10 films. Number 2, Mark Caro, is based on 30.

Also, this number depends greatly on which films you've ranked. Rank different films and get a different list. Variety's David Rooney ranks 86th on this list, but after 250 different films he was number 1. Still, some critics do appear quite often in the upper echelons of all of my lists: Mark Caro, Scott Foundas, Marjorie Baumgarten, Rick Groen. All of these are critics that I read regularly and whose opinions I respect. And there are few surprises at the top of the list — the only critics in my top 25 with whom I frequently disagree are V.A. Musetto (11), Peter Travers (17), and James Berardinelli (24).