One interpretation is that all content should be there to help the players play the game, to help them experience the kind of experiences you want them to as the game designer. Taken to extremes, this question can be asked of all content in your game, from every piece of art to each and every word that makes up the text. Art, mechanics, fiction, world building; ultimately all content can and should be able to answer this question definitively and if it can't, then one might consider it an unnecessary or even harmful distraction from delivering to the players the tools they need to fully experience the kinds of things you made the game for in the first place.
Another interpretation is that as well as providing your game product as a tool to enable players to achieve the experiences you want them to, it is also to help immerse them in the ideas of the game itself. Rather than providing specific tools, you use art, fiction and other content to provide inspiration, without explicitly giving tools to help players deliver similar experiences to those depicted by the content. Instead, you focus on providing enough immersion that the people consuming the media gain an insight into the ideas you've tried to imbue in the game and thus equipped, their imagination can build them tools of their own.
I think both approaches have merit, and like many things, a balance needs to be kept. I admit, I'm a fan of short-form fiction in roleplaying game material, but I'll admit, it's not always good enough to provide the inspiration I require to start building my own mental framework of tools for the game.
Equally, as a programmer, I find well-defined specifications extremely rewarding. I can't think of any case where clear instructions and tools to help me play a game (and more importantly achieve the kinds of experience the game intends) would be anything less than extremely useful.
I don't think dismissing all media from a game product in favour of providing those tools is doable though. Beyond serving the needs of the game itself, media is also the language of marketing and I imagine many potential players would never investigate something that appeared somewhat bland at first glance.
Anyway, I'm interested in your opinions on what I see as the trade off between media as inspirational and media as instructional. This isn't to say that instructions can't themselves be inspirational, but where is that tipping point and how does one get the best of both?
" I think however, with proper annotations, fiction can be extremely useful because it can be a narrative not only enjoyable in its own right as a consumable product "
The original post came from a discussion where asked about short stories in roleplaying games. The major issue I have is that very few people can write in an engaging way. Very few roleplaying designers are good authors. Just compare some of Robin D Laws's earlier work (ex. Hero Wars) to some of his kick-ass products today (ex. Feng Shui 2).
That said, I had the pleasure of reading three products that had good authors, and it worked well. Two of them had an ongoing story that presented the world, and one was Shane Hensley in Deadlands (the other was a Swedish game module). The first story in Deadlands, that introduced the game master to the world, was one of very, very, very few stories that engaged me as a reader. I can think of three games of the 200+ roleplaying games I read that managed to do that. It also gave me, as a game master, an idea of how to present that information to the players.
So it's not impossible, but it takes skill.
A Swedish game called Hammarn och Trollspöt (Eng. The Hammer and the Wand) presented the world in form of letters, traveling journals, political documents, recipe, newspapers, and such. What was unusual was that all of these were open for interpretation so the game master had to actively puzzle things together. The author then presented an adventure creation model, also open for interpretation, that the game master then should mix together with (its own interpretative) setting material.
So I do know games that can both have fiction that engages the reader, and make the reader taking an active part in how it can present the information to the players. But it takes years of learning the skill to do that, and few game designers have actually thought about this. I guess that's why it's so largely underrepresented in roleplaying games. That, and that some people assume that what works for one medium (literature) is beneficial for another (games).