Think back to a book that had a big impact on you. Maybe it changed the way you look at your life or the world. Maybe it changed how you approach your relationships.
Gödel, Escher, Bach. Guns, Germs, and Steel. Infinite Jest.
If you’re like me you’ve probably chased after that feeling of insight by looking for other books in the same vein, and more often than not they’ve left you cold. It’s particularly frustrating when these blah books come so highly recommended, maybe by a trusted friend or the masses on Goodreads, maybe by a mentor or celebrity role model.
The way we discover books is flawed and I think we can do better.
To illustrate one manifestation of the problem, imagine that books A and B both contain similar insights. Alice reads A and Bob reads B, their minds are blown, and they go off to evangelize for their respective books. One day Alice and Bob meet and each recommends their favorite book to the other. Alice reads B and finds it boring and insipid; Bob reads A and has a similar reaction. What happened?
Now it’s possible that A and B are written in different styles, and that A appeals particularly to Alice and B to Bob. But for the sake of argument let’s suppose that if Alice had read B before reading A, she would have found it equally insightful, and similarly – reversed – with Bob.
This scenario is contrived, but I think similar things do happen in real life. Sometimes I even do it to myself: I revisit a book I thought was great and wonder what the big deal was. I think what’s going on is this: that books (and other media) seem interesting or insightful only under certain rare conditions.
Insight on the Edge
Every book requires background knowledge. If I am too inexperienced to understand a book I won’t get anything out of it. On the other hand if I’ve already encountered the ideas in a book I might nod along, but I would have no need to change my worldview. You can only blow your mind once in a given way.
Insight lies in the fertile area between incomprehensibility and yesterday’s news. It’s rich in meta-information, information that – perhaps indirectly – tells you how to better model the world.
Hints for Better Book Recommendations
For a certain kind of book, the kind that is meant to explain how the world works, rating systems like Amazon’s or Goodreads’ are inadequate. The reader cares less about finding a book that other people have found insightful than about finding one that they will find insightful.
In other words, insightfulness is not a property of books alone, but of book-reader pairs. Book quality is a two-place attribute.
Further, ratings are subject to a host of systemic biases, e.g. the halo effect, political bias (of which more below), or a kind of survivorship bias whereby readers are more likely to finish and rate books they enjoy reading, particularly long and difficult ones (think Ulysses).
With this in mind, we can start talking about what a good recommendation algorithm would have to keep track of.
Ideas in books build on one another and we’d like to recommend books in the right order. To this end, we can to identify introductory works that need no background material to be understood. We can to identify prerequisite ideas (to read and understand C, you must first read A or B). Given these links we can find paths through idea-space towards increasing complexity and sophistication.
We can compensate for memory decay with spaced repetition.
We can infer the reader’s current levels of understanding from their prior reading list and ratings, possibly also asking them questions about their background knowledge or learning goals.
If this sounds like curriculum design, that’s because it is.
This approach is not without problems. How do you catalogue all ideas in all books? How do you deal with partial understanding, or verify understanding in the first place? Or deal with circular dependencies, where I depends on J depends on K depends on I again? How do you handle the truly great works, the ones that can be understood and appreciated on multiple levels, that stand up to multiple re-readings?
My sense is that we don’t have to find an optimal solution, only a pretty good one. At any point, for a given reader, there are likely many books that they would find insightful. We as readers can be forgiving in other ways: sometimes I read a book that doesn’t contain any new (to me) ideas and I still get something out of it. Maybe it’s just been a while since I’ve thought about the topic. Maybe the new book phrases things differently or uses a different set of examples. Maybe I understood the ideas before but they’re only now gaining relevance in my life.
All of this amounts to an approximation of empathy, a thoughtful model of the reader. Your friend who makes uncannily good recommendations for you? They’re not just saying, I liked this so you’ll like it too (although many friendships are indeed based in shared tastes). Rather, they’re taking your personality into account. They’re anticipating your experience. This feels good when it happens, for the same reason it feels good to get that perfect gift – you get a sweet new trinket or gadget, sure, but you also get an affirmation that your friend gets you.
Competing Ontologies
In the example with Alice and Bob the two books contained compatible messages, two ways of presenting the same model of the world. In practice real-world complex phenomena afford many explanations, and books will present different, competing ontologies. As with optical illusions like the Necker cube or the Rubin vase, the territory may be represented by multiple maps. However these illusions are designed to be ambiguous with both interpretations presented at once, while most people’s models for complex phenomena are well enough established to avoid spontaneous perceptual reversals.
To say this another way, explanatory models exhibit a kind of first-mover advantage. Because of this, the wrong book may close off as many opportunities for further reading as it opens. This effect most spectacularly emerges between opposing religious ideologies or bodies of political thought, as between liberal and conservative media in the US. In recent years there has been growing concern over the so-called filter bubble effect, in which tailored search results lead to increasingly misinformed readers.
Can our algorithm counteract this effect? Even supposing that we can identify books that change minds, how do we get people to want to read these books? Maybe changing minds is too ambitious, and we can only aim for an acknowledgment that, as Bret Victor puts it in a slightly different context, “there exists a worldview in which all of these statements are consistent.”
I don’t have a solution for this problem.
Beyond Insight-Seeking
Why not fiction too? Let’s take into account upbringing, life situation (is the reader going through a breakup? buying a house? nearing retirement?), personality traits (MBTI/OCEAN), archetypes, identity, etc. Can we effectively use calls to action by emotional appeal to increase the likelihood a recommendation is actually taken? (“Want to feel sophisticated, comforted, inspired, etc.? Try reading Gone With Gravity’s Shades of 1,000 Splendid Years of Solitude in Jamaica!”)
Notably Netflix’s recommendation algorithm offers the movies you’ll actually watch, which might have a mediocre ratings, rather than the ones you aspirationally say you’ll watch – your to-watch list is for the movies you really should get around to, just not tonight.
Perhaps this is all taking things a bit too far. Such attempts are necessarily inauthentic, not based on a true understanding of the reader and therefore unable to match the abilities of a skilled human recommender with direct personal knowledge. This, coupled with the larger time investment inherent in the medium (movies are quick, books are slow), leaves existing book recommendation services lacking. For now.
We are likely nearing the point, especially with increasing quantity and variety of personal data available, where the imitations become almost as good as the real thing, not to mention readily available and cheap, and the market will shift. And as often happens we’ll gain and lose something in the transition.
An Addendum
Some years back my friend Orion corresponded with Ursula K. Le Guin with about book discovery and she wrote about the exchange on her blog. Orion points out (and I agree) that the non-physical nature of e-books robs the reader of a certain kind of serendipity. Perhaps the “Accidental Discovery” of the library shelf could be emulated by appropriate technology, but even if implemented I suspect the experience will not be quite the same.
On the physicality of books, Le Guin writes of her 16th-century copy of Ovid’s Metamorphoses:
“For all my lack of the collector’s instinct, I handle that little book with reverence. It is the oldest book I have ever touched, by far. And touch does mean a good deal. So does time....
The margins are full of comments and the close-printed lines are interlineated with translations (mostly into German, or with another Latin word) in various colors of ink, some very faded, and many different handwritings, all tiny and mostly illegible to me. This book has been a scholar’s treasure and perhaps a schoolboy’s torment, it’s been bought and sold and given, lost and found, it’s been jammed into the pockets of greatcoats, thumped about in rucksacks, pored over in student lodgings, it has gathered dust in attics, crossed many waters, and changed hands a hundred times; it contains four hundred years of obscure human histories right along with the two-thousand-year-old words of the poet. Would I prefer it virginal, encased in plastic? Are you crazy?
But the question I can’t answer has to do with content. It’s this: To what extent is the Metamorphoses in e-book form the same book as the one I’ve been describing?”
Along the same lines, Martin Fowler writes about the difference between “book” as a physical entity and “book” as a literary work.
Photo credit: Library Bookshelf by twechy
comments powered by