The more I think about it, the more one thing about last month's Battlestar Galactica finale sticks out at me: It had a happy ending. And that still seems odd, unexpected and almost unfair. Why?
Perhaps I'm imagining it, but it seems, sometimes, as if downbeat sci-fi has somehow become accepted as more... highbrow may seem the wrong word - "realistic" definitely is - but intellectually acceptable, if that makes sense; people dismiss more positive sci-fi as fluff and popcorn viewing best done with your brain turned off, while happily gathering around for the latest SF that eagerly demonstrates just how fucked we all are, whether by our own doing or by outside forces. Early sci-fi efforts included their share of cautionary tales of things that could, and will, go wrong, of course, but there was still a sense of wonder and positivity about this whole "technology!" thing that is almost entirely missing from today's science fiction (Even the aforementioned BSG finale had an essentially "everything's great, and we've been given another chance - but watch out for those robots!" coda attached). But how did that happen? When did we become so resigned to our dystopias and our demise that it became what we embraced? Some theories:
Happy Is Boring
Who wants to follow happy people doing happy things? We watch/read/listen/play characters who have grand adventures and terrible experiences, because those are the ones that have interesting things happen in them, and because escapism is always more exciting when there's an element of "I'm glad that's not me" in there, as well. Conflict is key to drama, after all; when you're just paying attention to contented people feeling content, you're left watching one of the more boring episodes of Star Trek: The Next Generation, and nobody wants that.
We're In Our Teenage Angst Period
Along similar lines, culturally, we've reached a point where optimism doesn't seem mature enough anymore - Happy endings are the stuff of fairy tales and we've grown past those a long time ago, thank you very much. It's not just science fiction; stories of all media and genres - well, okay, maybe not musicals - have found that, to be critically acclaimed, it's better to focus on the more serious, downer things in life. Of course, sci-fi has that problem more than most, considering the general cultural apathy towards it in general (See "Why Aren't Critics Insanely Excited About Terminator: The Sarah Connor Chronicles And Other Related Questions").
Optimistic Sci-Fi Doesn't Try Hard Enough
This isn't entirely true, of course; I think there's an argument to be made for Flex Mentallo to be the equal in almost all ways to Watchmen, for example... But for the most part, even the good "happy" SF (Doctor Who, Fringe, Transformers) seems to content to "merely" be good entertainment instead of trying to make the grand statement about something larger, like a Battlestar Galactica. Mind you, this may be because...
It's Impossible To Be So Optimistic About Science Anymore
Is the bloom off the rose? We've seen how technology can be abused and used for evil ends too often to be too eager to celebrate it fully anymore. Even in the high tech utopia of Star Trek, the positivity comes more from the human spirit - even in the alien races surrounding us - than the science (Remove the humanity, after all, and you're left with the Borg). It's much easier for us to worry about technology than it is for us to believe in it, perhaps; we're more and more convinced that we'll create Skynet before Data (and even Data comes with evil prototypes).
In the end, perhaps it's just a trend; maybe one day we'll collectively wake up and realize that our creations haven't managed to overpower us and cause nuclear annihilation after all, and that science has slowly continued to improve our lives even if it doesn't look like the glittering prize that it may have done fifty years or so ago. There's always been as much to be in awe of as in fear of, in terms of scientific discovery; it'd be nice if our fiction remembered that more often, and delivered stories that compel, tell us about ourselves and also remind us that, sometimes, things can work out after all.