A couple of years ago, a research paper from Uber AI came out and obliterated the previous algorithms on the Montezuma’s Revenge Atari game. Montezuma was one of the hardest Atari games to crack as it belongs to a class of games called "hard exploration problems", where the agent has to "learn complex tasks with very infrequent or deceptive feedback".
Disclaimer: I've never played the game but i think the idea that made the algorithm do so well is simple and generally interesting.
Here's the insight:
"We hypothesize that a major weakness of current intrinsic motivation [adding an exploration bonus to actions] algorithms is detachment, wherein the algorithms forget about promising areas they have visited, meaning they do not return to them to see if they lead to new states."
By adding a component that keeps an archive of promising visited places, the algorithm was able to beat previous competitors by two orders of magnitude.
That made me think of open tabs on internet browsers. Open tabs is a recurring problem for people who engage in exploratory search. While they act like an archive of promising areas for future exploration, they mostly become a dump of articles never to be touched again.
"That feeling when one finally closes a browser tab, after weeks of keeping it open in search of a moment to read it (which never arrived). Repeats 3 times: *The web is a river, don't try to hold on*." — Samim
As we explore a new exciting domain of knowledge, we quickly become overwhelmed by the different possible avenues to explore (*opens new tab*). As one goes down one rabbit hole, we are quick to forget the multiple other avenues that had caught our interest early on. These different branches of exploration quickly lose their meaning, as we forget the context in which they were found, and so the tabs remain there until we reluctantly close them some months later.
We know the tabs have value — why otherwise such pain in preciously keeping them open? But that value is gone when we go back to them, lost in the rapid stream pouring out of our information bottleneck. If only we could keep those promising sparks we once saw, and re-visit them when the time is right!
My hunch is that much of it has to do with how we represent information, and one of my biggest grudges is our obsession with lists.
1) A list of tabs:
2) A list of songs:
3) A list of interesting people:
4) A list of search results:
Lists are often great, but they're terrible when exploring new or complex domains of knowledge, and they're a very poor scaffolding for exploration. They're the equivalent of a sad pile of papers, when what we need is maps. The documents could instead be spread out on the floor of a very large living room in thematic islands, or organised in enticing library alleys.
Evidently, many people have spent a lot of time thinking about these things, like Vannevar Bush's memex, or Ted Nelson's Xanadu. I touched upon this a bit in a previous blog post on annotations and linked notes.
Related Projects
- Every Noise At Once — Mapping music genres.
- Fingerprint — Visual explosions of GitHub repositories.
- Tree Style Tab — Keeps tabs as trees where newly opened tabs are "children" of their parents.
Maps empower active search processes, providing people with a greater sense of direction. It's also about time we move on from repeated queries in a search box and doom scrolls. Who knows, perhaps better interfaces and intuitive representations could lead to improvements in exploration orders of magnitude greater than what we have today?