The Form of Formats

From the inception of art on computers, intellectuals have breathlessly theorized about the endless new possibilities for creating new media. This has persisted through many advances in technology, from the primitive pixel art of the ’80s through the modern-day interest in AR/VR. Computers are able to overcome many of the limitations of analog mediums, and open up avenues for brand new modes of expression that give artists higher and higher levels of control over their output.



Despite this excitement about tearing down old barriers to make way for new art, the most engaging new media is situated in emerging formats which limit possibilities by the introduction of artificial constraints. 



What is a format?

When you pour water in a cup, it becomes the cup. When you pour water in a bottle, it becomes the bottle. When you pour water in a teapot, it becomes the teapot -Bruce Lee

A format is a container for content that gives it its shape. The boundaries of a format are defined by the constraints it imposes on the content. These constraints are not the physical or practical limitations of a transmission medium, but are restrictions made by choice. Take for example the Haiku:



this is a haiku

with specific syllables

on each of the lines



nothing can stop me

from adding an extra line

it might be a poem

but it’s NOT a haiku



In addition to these limitations, a format generally makes it easier to do things in a certain way; it has a grain to it that shapes the content.



You have to go out of your way to go against the grain, so those features that follow the grain tend to become tropes and expectations, and features that go against the grain are salient and "subvert expectations". Much content is playing into and subverting the expectations of the format.



A format shapes content in a similar way to the much-celebrated medium, which is the channel through which an idea is transmitted: Acrylic paint, slide projector, bubblegum wrapper. Marshall McLuhan’s famous phrase “the medium is the message” makes the claim that the nature of a medium, and how it shapes content, is more important than the content itself. Formats, like mediums, are typically discovered rather than invented and gain their own welled-up history that drives and informs creative action:



“[…] Which is another consideration to take into account in this matter of “inventing a medium”. Artists do not, of course, invent mediums. Carving, painting, drawing, were all in full flower before there was any socially distinguishable group to call itself artists. But mediums then individualize their practice; they intensify the skills associated with them; and, importantly, they acquire histories. For centuries it was only within and against the tradition encoded by a medium that innovation could be measured, just as it was in relation to its reservoir of meanings that new ranges of feeling could be tested” - Rosalind Krauss, ‘… And Then Turn Away?’ in James Coleman, Wiener Secession, 1997, p.5



As computers tear down more and more old barriers, the choice to continue to work in any particular medium is the choice to work within the “format of the medium”. It is possible to reproduce a polaroid picture on a computer as an image effect; the colors, border, and grain of the film are simulated without ever involving actual film itself.



Ebooks are a format designed to replicate paperback books. Ebooks are not a 1:1 equivalent to the paperback, but are a useful representation of books that allows content created within them to be displayed and consumed in a similar standardized way. When you buy an ebook or paperback off of Amazon, you have a strong sense of what to expect: you know the content reads left to right, is linear from beginning to end, is separated into chapters. You don't need to relearn these things every time you open a new book. Formats work to make content largely predictable.



Haikus, paperback novels, .png files, and VHS cassettes are all swappable, and this quality is very useful for both discovery and consumption. Anyone making art in completely alien formats—often bucketed together into miscellaneous categories like “experimental” or “new media”—has to completely go it alone. Life on the frontier is hard.



Worlds

How do formats interact with worlds and worlding?



Formats, in a sense, are an orthogonal concept to Extended Media Universes; they intersect with these worlds to create a bounding box for content:





The Harry Potter universe can release content in many different formats that share the same world. The content inherits the context and expectations from both the format and the universe.

In the same way that Extended Media Universes contain specific franchises, formats themselves contain other, more specific formats:





All media inherits the form of its parent contexts, but can be said to have a format and a world of its own specific to that work as well. All shared formats and extended universes start in the context of one project before they are expanded outwards.



How are these one-off project specific formats and worlds created?



Improv Theater

Improv theater is about creating worlds and content within those worlds on the fly in a very short span of time. Improvisers need to create a minimum viable world as quickly as possible, and then maintain that world while also generating the meat of the content.

The improviser must embody several different mindsets at the same time to create a fully realized scene (a la Ian Cheng's masks):



The Content/Mechanics axis is interesting because it highlights the core difference between worlds and formats which are otherwise extremely similar: a world is a context that shapes what you are saying (content), and a format is a context that shapes how you are saying it (mechanics). In Scott McCloud's Understanding Comics he outlines the stages of development for an artist, and defines two types of artist capable of creating masterworks: masters of form, and masters of content.



Base Reality

“Base Reality” is another term from improv that means the world of a scene. Because the goal is to create a shared base reality as quickly as possible, the fastest way to achieve this is to accept and build on all ideas and information proposed. This is where the famous improv phrase “yes, and” comes from: yes what you've said is true, and here's an additional piece of information.



Many painters find it difficult to paint on a blank white canvas, so they will lay down some initial textures or splatters in order to get themselves going. Then they progressively build on those details by allowing themselves to assume that they are representative of something larger (basically a form of practical Pareidolia). Imagine you are an artist, and at the center of your canvas you see a small dot.

•



Maybe you think the dot looks like an eye, but what does the eye belong to?



Now you have a fish! The fish was always there, you just needed to see it. In this way, the best worlds are discovered (in a certain sense at least).



The Game of the Scene

Once the minimum viable world is established, the performers switch to generating a "game" that will create the meat of the scene:





The game, at least in UCB style improv, is a pattern of activity that is optimized for generating jokes. It is useful to think about this in terms of Dan O'Shannon's theory of what a joke is:

The base reality sets up rules and expectations, and the performers try to identify a potential wrinkle or anomaly in the set of facts that doesn't match our expectations (an "unusual thing") and to create a game of exploring and heightening it. The audience is left to face how their expectations were wrong and incorporate these new insights about how the world of the scene works into their mental model of the situation; which is often experienced as “funny”, or at the very least interesting.



Let's run through a scene:



Scene: We're shopping at a department store, but I'm a witch

Base Reality: Department store

Unusual thing: Witches

The Game: Witches as Shoppers

Example Moves:

• “Do you need help getting that down from the shelf?"

  "No I've got it" *levitates*

• "Excuse me, do you sell cosmetics for women with Green complexions?"

• "My coupon is expired!? Good thing I know necromancy!"



What might be the unusual thing in one scene can be baked into the base reality of another:



Scene: We're witches, but I don't like using magic for health reasons

Base Reality: Witches

Unusual Thing: Not using magic

The Game: Witches as overly health conscious people

Example Moves:

• "We could fly there on broomsticks, but actually I would rather walk. Good exercise."

• "We're always making potion after potion. You know what else cauldrons are good for? Good old fashioned soup."

• "I actually prefer to wipe my own ass instead of disapparating my shit away, it's more natural that way"



There are other flavors of games in improv based around following narrative formulas or examining and heightening status conflicts between characters, but the general idea is the same: quickly establish a world, identify the grain of expectations for the scene in that context, then play off of those expectations.



The name of the game

Any well-developed piece of media has a game at the center. The game is built around the pursuit of interestingness, creating an engine to uncover insights in the form of jokes, truths, or cool guitar riffs. These insights update your mental models about the parent contexts: the format or the world (up to and including truths about “life” itself). The game is a conversation.

All content is either games to play yourself or recordings of other people playing games. Narratives are simply the recounting of events in a game already played (or the prediction of events in a game still unfolding). Watching other people play games is a way to play vicariously; your detailed knowledge of how the game is played serves to enhance your experience of watching it by allowing you to understand the conversation. I don't enjoy football, but only because I don't have any existing mental models about how football works; I may be interested in the lessons to be learned about triumph and self-sacrifice, or marvel at the spectacle of the half-time show, but I cannot find interest in the technical specifics of how a pass was made or a play was executed (in exactly the same way that a literal conversation about those topics would be boring to me).





Most media we consume is the result of multiple games played simultaneously, and people gravitate towards the games they understand. When I listen to music, I engage directly with the singing and the lyrics because of my background in choral music and writing poetry. Friends of mine who play guitar or understand music production will hear totally different things in the same music based on their experience. When you watch movies, do you pay more attention to the actors in their roles, or the writing, or the cinematography, or the score?



Games can exist within other games, just as a conversation about a football game can be a part of the larger conversation about football, or sports. The game itself becomes just a single “game move” in the larger game. Every game move has a price, and every game rewards its players. Memes take only a few minutes to make, with a potential reward of social status and bragging rights; a blockbuster movie can take millions of dollars and years of effort by hundreds of people to make, with a potential reward of billions of dollars. These factors determine who can play, and who wants to play.



Zombie Apocalypse

There is plenty of media created with no game at the center: it simply has nothing to say. Perhaps there once was a game, but the conversation has died down and it is no longer able to produce new insights. The content is produced as a result of the joyless pseudogame “how can I copy existing media to get the most reward with the least effort”. Typically we think of this type of media as soulless, bland, formulaic, and dead.



Factory pop music is a familiar example. This is basically a catchall term for music that seems like it was written by a robot and packaged primarily as a product, and excludes albums that are “pop” but still have an interesting angle or things to say. Many outstanding pieces of music (and probably most really “big” hits) jump to the top of the charts containing a small piece of shiny, interesting, fresh insight packaged in an accessible way. Sometimes this is arbitrage: taking, stealing, or appropriating ideas from more obscure or inaccessible artists, but at least there are ideas to be found. Given the money-making potential, however, for every good pop song there are a dozen pieces that go through the motions of following the formula with no new ideas at all: these are “bland” and “unoriginal”. These unoriginal songs can still chart, in the same way that if you repeat something you've heard before to a person who hasn't heard it, they may mistake you for being smart.

Meme formats also highlight this uneven progression of death:





Meme Formats show the tight relationship between formats and games. The low cost of participation allows them to evolve at lightning speed, and their simplicity allows for all manner of complicated format games:





Once the game isn't fun anymore, memes follow a predictable progression from funny to “normie” to “dead”. This reflects the uneven propagation of new insights through the fabric of society: if it's new to you, it's still interesting. Even the deadest of dead memes can be funny to people who are completely out of the loop. My brother‘s high school had a contest to create memes to promote consent, and in an extremely cynical (and entrepreneurial) attempt to win all three top prizes of $500 he created over 100 low-quality formulaic submissions attempting to pander to the teachers’ poor understanding of meme culture, and submitted them under his friends’ names. He took the top prize under his own name with one of the first and most obvious memes he created. 



It's easy to see how the potential for reward can outstrip the pursuit of interestingness as a motivation, and how this can lead to soulless content that goes through the motions without really saying anything. It is possible to turn any game into a formula: a game where the rules are so strict that the moves are predetermined. In order to uncover new insights, games need to cover new territory.



How to play a game that doesn't exist

In the same way that nobody invents a medium, truly vital and living formats are discovered organically. Nobody sits down to write the rules of the game until after the game has been established; even game designers (who represent the most intentional, authorial version of creating games) rely heavily on play-testing to understand and shape the game before committing to anything. Other types of media makers, however, are not primarily in the business of defining formats or genres: they are in the business of making standalone works.

The first work made in a new format usually only intends to be standalone, or weakly continues the conversation about an existing format (the first movies were just filmed stage plays). To borrow some more wisdom from Improv Theater: the game only begins on the second move, and is cemented on the third—It doesn't become a game until a new pattern of expectations is established, and a back-and-forth conversation is started. A meme is just a standalone funny image until someone fills it in with new content and establishes it as a format. The problem is not actually how to create a game—it's as easy as striking up a conversation—the problem is finding people to play that game with.



So, as we look forward to the endless possibilities enabled by new technologies, how do you get people to want to play a game that doesn't already exist? This is complicated, but is basically the same question as "how to build a scene/community/public". It’s probably possible to do this all on your own, playing solitaire or having a schizophrenic conversation with yourself over the course of a lifetime, in the same way that extended universes often start as the work of one person.



But it's probably easier and more fun if you find collaborators.