Technologies of Taste

napoleon-dynamite300Ever wondered what kind of magical juju lurks within ‘recommendation engines?’ Anyone with a Tivo knows just how wrong it can be about your preferences, but I’m sure I’m not the only one who has fallen into an if-you-like-this-then-you’ll-like-that click-hole, pleasurably stumbling across stuff that you can’t believe you hadn’t heard about before. It’s like souped-up serendipity and it’s very addictive.

I find this technology fascinating, not only when it’s accurate but when it’s not. Whether you view the technology as a craven marketing gimmick or as a slightly magical personality diagnostic tool, it certainly raises some questions about the nature of taste and personality, the delta between pleasure and boredom, and the possibility that fickle humans could be decoded.

One reason I believe that entertainment research is so important is because it’s a powerful way to explore identity, both the private self with its eclectic predilections, and the social self, which enjoys its pleasures as a part of various ‘taste communities’ that literally circle the globe. Surveying people about their entertainment preferences (as we do periodically with Zogby International) is one of the better ways to ascertain the state of any human cohort — its desires, its fantasies and its attitude toward itself. When social networking tools like Facebook ask us to define ourselves by the cultural products that we consume, it literally inserts us into a vast ‘taste database’ where, for instance, any other person who put PJ Harvey in their Favorites is just a click away from becoming a friend.

But to think about taste as a simple database — or even a complex relational database — doesn’t begin to describe the bizarre internal logic of our own predilections. Why do I like Buffy and scoff at True Blood? The connections and disconnections among these datapoints are so complex that even those closest to me can misjudge what I will and won’t like. So how on earth can a computer do it?

Well, they don’t do it very well, but online retailers like Netflix have found that a recommendation engine has a very positive effect on their bottom line (mainly because it encourages customers to watch movies they’ve never heard of). Netflix launched a public contest to try to improve Cinematch, their proprietary engine, by 10%. The competition’s been going on for over two years, but the top team has only been able to improve the accuracy of the recommendations by 9.44%. Despite stiff competition, and a lot of information sharing among teams, no one can get over the hump. The New York Times Magazine reported recently that one competitor, Len Bertoni, realized that one movie — out of the 17,770 in the sample — was increasing his error rate by a whopping 15%. The culprit? The 2004 indy hit Napoleon Dynamite.

After the article appeared, the discussion boards for the Netflix prize lit up. Bertoni mentioned that he had identified 25 difficult-to-predict movies, only some of which were mentioned in the article. All were described by reporter Clive Thompson as ‘culturally or politically polarizing and hard to classify.’ Generous programmers started sharing their own lists of the most difficult to predict movies and every list included Napoleon Dynamite, Sideways and Sin City.

It’s not just that these are ‘love it’ or ‘hate it’ movies — I’m sure many people, and I include myself here, have pretty moderate feelings about these flicks, whether they liked them or not. They aren’t, for instance, as polarizing as some of the other films that showed up on a few hard-to-predict lists, such as Fahrenheit 9/11, Crash or The Passion of the Christ. Nor are they particularly hard to classify. None of them commit any serious genre bending, and I know I could find them lickety-split in my local video store. What they all have in common, though, is that several computer programs have not been able to predict how many stars they received from past Netflix customers. Why?

It’s certainly not due to a lack of data: Netflix released 440,189 customer ratings. The problem may be that most of the teams in the top tier of the contest are using a mathematical technique called ‘singular value decomposition,’ which identifies groups of films that share a certain predictive factor such as graphic violence or profanity or sci-fi geekiness or chick-flickiness. The most disarming thing about this technique is that the programmer doesn’t identify the factor beforehand. They just run the algorithm and then hope they can figure out what ‘factor’ the program has isolated in each group. Sometimes the answer seems very clear but other times it’s utterly inscrutable. The reporter Clive Thompson argued that when the result is illegible it could be that ‘the machine may be understanding something about us that we do not understand ourselves.’

Really? When I asked a friend of mine — David Ortega, a former colleague at Vivendi-Universal Games, who went on to get his PhD in psychology — what was up, he offered a compelling critique. He argued that the Netflix recommendation system is nonlinear but the programmers are depending upon a linear means of investigation. Turns out David wrote his dissertation on nonlinear and chaotic behavior in multi-member therapy sessions (family therapy to be exact). The insight David offers is that the Netflix data is nonlinear, which makes it incredibly hard to predict, not because it’s random, but because it’s chaotic: people tend to use the words ‘random’ and ‘chaotic’ interchangeably, but chaotic sequences are not random, they’re just really, really hard to predict. Here’s David:

The trouble is it’s difficult enough to demonstrate that certain behavior is chaotic — it’s nearly impossible to predict it, due to “sensitivity to initial conditions.” One “wrench in the works” (a predictable family Netflix queue that only rents Merchant and Ivory films suddenly picks Napoleon Dynamite) propagates through the whole system.

There’s something very appealing about the idea that human entertainment preferences may be too complicated to compute. But, on the other hand, it’s kind of charming when the machine gets it right. One programmer on the Netflix Prize discussion board offered his list of the films that were the easiest to predict:

The Lord of the Rings: The Fellowship of the Ring: Extended Edition
Patriot Games
Star Wars: Episode V: The Empire Strikes Back
Lethal Weapon 3
The Shawshank Redemption: Special Edition
Toy Story
Lethal Weapon
Lord of the Rings: The Fellowship of the Ring
Sex and the City: Season 4
The Godfather, Part II
Lost: Season 1
Finding Nemo (Widescreen)
Six Feet Under: Season 4
North by Northwest

Do you have any problem recognizing who would like or dislike these movies? Yeah, me neither.

Related Posts