It never fails: The cooler a hero is, the likelier he or she is to start tossing out references that only mean something to a handful of fans. The longer an epic story goes on, the more self-referential it gets.
Why does this happen? And is there just a natural law that stories have to get more inside-baseball the longer they continue for? It's a form of narrative decay. Here are some theories about why and how it happens.
Think of it as a spectrum. There are two extremes of serialized heroic narratives: expansive and self-referential. At one end, you've got a set of stories in which the hero is constantly meeting new characters and discovering new stuff, and everything is freshly created for just that story. At the other end, you've got heroes who constantly re-meet old villains, friends and other supporting cast members. The self-referential end of the spectrum also includes in-jokes, fanservice, and obsessive references back to old continuity points.
Most epics fall somewhere in between the two extremes — very few ongoing stories have either no reference to the past, or nothing but references to the past. Most of the time, a long-running storyline will sit somewhere in the middle — still introducing new stuff, still expanding the universe, but also harkening back to earlier stories. A good example of this is the most recent few seasons of Doctor Who, which have reliably brought back old villains and also given nods to 40-year-old stories, but also kept introducing new stuff.
It definitely seems to be true that most ongoing or serialized epics start out more expansive, and then become more self-referential over time. Most genre television shows start out with more "monster of the week" episodes, and then tilt the balance towards "arc" episodes over time. But also, a character in any medium will start to reference older stories and re-meet older characters as the adventures go on, and on, and on.
Narratives decay as they get older — or maybe it's more accurate to say that they ossify. Structure replaces energy. All of the inventiveness and newness that animates a new story get replaced by walls and props and girders. This can be a beautiful thing, if the structures are really elegant or moving — but there's no substitute for having a story that pops with newness and keeps introducing new stuff.
It also seems to be true that whenever an epic hero makes the jump from one medium to another, the level of fan-service and other sorts of self-reference goes up massively. Especially if a comic-book character gets a movie, you're bound to see lots of fan-pleasing nods and easter-eggs, and sites like this one will point them out for you if you miss them. But this also happens a fair bit with comics-to-television or television-to-movie translations. Partly, this may be because fans of the original stories are entrusted with bringing them to a new audience — but also, it's a way of bribing the fans of the character to like his or her new incarnation. And also, you can't discount the fact that someone making a movie of a comics or television character will be grasping at straws somewhat, to try and make sure they capture what made the character so successful originally.
The different types of self-reference
So what makes heroes get more self-referential as they get older? The most obvious reason is that people grow up being fans of a character, and then they finally get a chance to create new adventures for that character — so they pay tribute to the stuff they loved as a kid. But also, there's the obvious fact that the longer a hero has been around, the more old stuff there is to refer back to, and the more layers of concepts get placed on top of the original premise. And also, the longer a hero hangs around, the likelier he/she is to have a vocal fan base, who really want to see references to all the little bits and pieces they used to love.
The most obvious example of a story being self-referential is fan-service — can you spot the Tribble in J.J. Abrams' Star Trek? Will you keep your eyes peeled for G'Nort in Green Lantern? — but really any narrative that refers back to the past becomes self-referential. And you can even be self-referential without referencing the past at all — any time a heroic story refers back to its premise, it's self-referential by definition.
For example, for a while there, it seemed like Spider-Man was only fighting spider-themed villains in the comics. And in the new Spider-Man musical, there's the spider-goddess Arana, who turns out to have some murky link to Spider-Man's origin. When Superman gets dragged into lots of stories about Kryptonians, that's also a sign of self-referentiality. And I think one reason why I get bored with Wonder Woman fighting Greek demi-gods and mythological stuff is because it's dragging her down in to self-reference land.
What makes an epic hero iconic is that their origin is not their premise. I mean, that the premise of Superman isn't that he's from Krypton and was raised by the Kents — the premise of Superman is that he lives in Metropolis and defends the Earth from monsters and city-bottling maniacs. The premise of Doctor Who isn't that he comes from Gallifrey and used to be a Time Lord — the premise of Doctor Who is that he travels around time and space fighting evil. And so on.
An epic hero whose origin and premise are the same thing is going to have a narrow set of adventures — like if all Batman ever did was try to avenge his parents. I think this is something that superhero movies, in particular, often get wrong — they confuse origin and premise, so that the movie's villain is part of the hero's origin and the whole thing plays out as ramifications of the origin.
Don't go to extremes
But like I said, the division between expansiveness and self-referentiality is a spectrum, rather than a either/or thing — and either end of the spectrum can have its own drawbacks and perils.
Like, if you go too far towards expansiveness, you risk winding up with a boring succession of "monsters of the week." And too much newness can become samey, paradoxically — you roll out too many "new" ideas that are the same ideas dressed up in new rags, and you lose all credibility.
At the other end of the spectrum, too much self-referentiality will shut out anyone who's not already a fan. And that you make your world smaller and smaller — every time Chewbacca or Tarkin shows up on Star Wars: The Clone Wars, the galaxy far, far away gets a little bit more claustrophobic. That galaxy starts to look like a town with 100 people in it.
A shrinking time window
I feel as though the time window between a series being expansive and becoming self-referential is shrinking all the time. Now it only takes a few years, tops, for a new television show, book series, comic book or movie series to start feeding on its own past continuity and referring back to itself. (And that's not even including the way new geek-friendly genre entertainment tends to reference other stuff.)
It's easy to see why the slide towards self-referentiality is getting faster — fan groups form way more quickly than they used to, thanks to the internet. And creators know that they have to court these nascent fan groups — a new franchise has to gather a fan base quickly, or it's in trouble.
Fans create energy and get people talking about your creation, and this can lead to more coverage in the mainstream media — but fans also demand that you repeat yourself, feature favorite characters again and again, and refer back to older stories. Fans love mythos, so the more you deepen the mythos, the happier they'll be. And "mythos" is a form of self-referentiality, because it ties together all of the past stories.
What's the alternative to becoming more self-referential over time? Well, there's the tried-and-true "reboot" method, in which you toss out the bathwater and about half the baby in order to try and create something new and exciting. This is usually a short-term fix, since you end up having to reboot again after a few years. In the worst case, you keep hitting the reboot button over and over, as your epic storyline blue-screens and you're left with just a blinking cursor and a bios that never quite finishes loading.
But J.J. Abrams may have been stumbling towards another alternative to self-referentiality and entropy — the endless curtains. You pull back the curtain and reveal another curtain. And behind that curtain — another curtain. It can go on forever, and it never gets tangled in its previous versions. Even if you eventually reuse one of the earlier curtains — like Lost trotting out the Dharma Initiative for its fifth season — you've gone through enough curtains in the meantime that it feels like something totally different than what it originally was.
Maybe the bottomless rabbit hole of J.J. Abrams is our best hope for having stories that provide enough mythos to please fans, without ever starting to look backwards too much.