Video as a Data Type
A reference architecture for AI-era media
The reference architecture I’ve been circulating privately with founders, operators, and standards bodies for the better part of 2026 will go public this week, as my keynote at Silicon Valley Video Summer Camp. This is my written version.
Video is becoming a data type. I want to be careful with that phrase, because it gets used loosely and I mean it precisely. A data type is something you can address, query, permission, settle, and attribute. A clip that behaves like one is addressable frame by frame. You can ask it questions, attach permissions to it, run a payment through it, and trace it to its source. It stops being something that only plays and then waits in storage.
The engineers reading this are already objecting, and they are right to (side note, if I had a nickel for every time a video engineer told me I was wrong, I would have a lot of nickels). MXF is structured, IMF is componentized, and yes, timed metadata has existed for twenty years. The operations are not new. What is new is where they live. Today they are properties of the system holding the file, and they die at its walls. The metadata a facility writes often stays inside the facility that wrote it. The shift underway is those properties becoming durable attributes of the object itself, carried across organizational boundaries, readable by consumers that increasingly include machines.
Picture a single clip leaving a camera. As it moves through the edit, out to distribution, into an archive, and in front of an audience, it carries its own context, rights, and lineage with it. Wherever it lands, who is in it, who owns it, and what may be done with it arrive in the same package.
Today that is aspiration. Metadata gets stripped (and sometimes mangled) at every transcode, every major platform sheds it on upload, and somewhere an archivist is getting paid to reconstruct everything the pipeline threw away. Treat this idea as a design requirement rather than a description, the requirement the next decade of media infrastructure exists to satisfy.
Once video behaves that way, the strategic question stops being which model is best this quarter. It becomes what infrastructure a media object needs in order to be addressed, queried, permissioned, settled, and attributed. That infrastructure is the subject of this piece.
The surface: make it, move it, mean it
Strip media work down and it does three things. You are either making the thing, moving it, or establishing what the thing means. Who is in it, who made it, who it belongs to, what may be done with it, who gets paid when it is. Creating a story and getting it to an audience is all well and good, but without that meaning, the context of who is in it and who gets paid, there is not much point to the rest.
That is the surface, where the work happens and where the AI tooling lives. Most of the press attention sits here. So does most of the procurement budget, and most of the substitution risk. A new model arrives, vendors integrate it within a quarter, the headline price falls, and the buyer swaps one supplier for another without anything downstream noticing. The tools are the weather. They turn over monthly, and a strategy built on the weather inherits its shelf life.
The fight has migrated across the three over time. Moving it was the twentieth-century fight, when pipe ownership decided reach. Then discovery, when the algorithm decided what surfaced. I believe the fight is now on meaning. And meaning is where the surface stops being about tools.
The seam
Lay the three out as bands running across the top of the architecture. Make, move, mean. Two of them are cut from the same cloth. The third is not, and the line where they meet is what I call the seam, the place where two materials that behave differently get stitched into one garment.
Make and move commoditize. Mean is different. Rights, identity, lineage, and settlement are the point where the workflow hands off to something underneath it that runs by other rules.
That hand-off is the seam in the whole picture. Above it, tools turn over and prices fall. Below it, value accumulates and stays put. The mean band is the doorway between the two.
And if the seam sounds abstract, the studios just drew it themselves. MovieLabs, the studios’ own technology joint venture, published its Creative Vocabulary this month. A controlled vocabulary of creative terms, cinematography first, machine readable, openly licensed, built so creative intent survives translation across tools and AI models. Think about that implication, the industry is standardizing the semantics layer while the models commoditize. Language as infrastructure, shipped by the studios’ own joint venture.
Commoditize or accumulate
Foundation models are converging in capability across providers. Headline prices fall on a predictable curve, though the unit keeps getting redefined, from tokens to reasoning steps to context windows to agent invocations. Abstraction layers already let an operator swap one model for another without the calling system noticing.
The objection writes itself, and it is worth taking head on. The model layer is two or three labs and capital moats in the tens of billions, which hardly sounds like a commodity. Both things are true, because they describe different ends of the transaction. Supplier economics can concentrate while the input commoditizes at the point of use. What matters to an operator is substitutability, and a category with falling prices, drifting units, and suppliers you can swap without the calling system noticing is a commodity input no matter how few firms produce it.
The substrate underneath behaves differently. Codecs decide what can move through a network at a given quality. Transport decides how reliably it arrives. Storage decides what can be retrieved, and when. Metadata, rights, identity, lineage, and verification decide what can be found, used, attributed, and audited. None of these fall on the model curve. They are not all equal, either. Transport has been squeezed for a decade and mostly persists rather than accumulates. The accumulation is strongest where the primitives are hardest to swap, in rights, context, settlement, and identity.
And it is already visible if you know where to look. Gracenote is a metadata registry that has outlived every player generation built around it and changed hands for nine figures more than once. Content ID is a rights and settlement primitive that has routed billions of dollars, and no creator or label can route around it. Neither is a model. Both are substrate, and both got more valuable as the tools above them churned.
The surface is where capability commoditizes fastest. The substrate is where it accumulates. The architecture is a map of where the accumulation is going.
The substrate
The diagram puts that in space. The surface sits on top, the workflow plane where make, move, and mean play out. Everything below it is substrate.
Six layers carry the durable value, starting with rights and policy at the top, then context and asset memory, usage and settlement, and routing and distribution, with transport and representation and machine access at the base. They persist because they carry high switching cost and survive operational change, and the upper four are where the value pools.
Two spines run vertically through the whole stack. Identity, which covers talent IDs, digital doubles, authorship, and machine identity. And data, which covers vector stores, knowledge graphs, ontology, and training-data governance. They run across every layer rather than sitting in one of them, because identity and data are primitives the entire architecture depends on.
One band runs through the middle, and it carries the most important reading instruction in the diagram. The substrate travels with the media object. Context, rights, and lineage are not a database the object gets looked up in later. They ride with it. That is the design goal named earlier, drawn as a band, and the live fights below are the fight over how it gets built.
The full architecture runs to roughly seventy tiles, more than anyone holds in a single reading. Five clusters make it navigable, knowledge, trust and integrity, rights and compensation, identity and personhood, and standards and interoperability. The clusters are the way in. The stack is the thing. The Reader’s Guide walks it layer by layer and tile by tile, and it is free.
The fights are already underway
None of this is speculative. C2PA and MovieLabs are arguing provenance and trust. SAG-AFTRA is arguing identity, consent, and who gets paid when a likeness performs. The carriage tier is arguing transport and routing. The data layer is arguing what may be trained on and how attribution is proven. Each is a question about who controls a substrate primitive. Who writes the permission schema, who meters usage and settles it, who owns the memory graph, who defines the machine-readable format, who governs the training set.
There is a version of this decade where those questions get answered the way the last two rounds did, inside a platform’s walls. The largest platforms already run private versions of most of this stack, context, rights, settlement, and identity, working, at scale, and closed. These systems are messy, though. A decade of media sales and acquisitions has left most of them stitched together from whatever each deal brought in the door, and anyone who has lived inside one knows it. The mess is also the opening. The migration argument says the fight over meaning could be won the same way distribution and discovery were, by whoever owns the audience. Whether the substrate stays neutral or gets captured is not a footnote to the architecture. It is the fight, and it is why the standards column runs through every layer of the diagram.
The architecture does not resolve those fights. It gives you a clean instrument for naming them, and for seeing that they are the same kind of fight surfacing in different layers.
Reading it against your own operation
The lens changes by who is holding it. An operator reads it as a capability map, which tiles we touch, where we are exposed, where our fight is. A vendor reads it as a positioning map, which tiles we serve, which standards bodies sit between us and what comes next. A strategist reads it as a power map, where value accumulates, and what will be obvious in 2036 that is not obvious now.
The architecture does not change. The lens does, and the fight does. The next decade of media operations will be conducted in the substrate, and this is one way to see it before it is obvious.
The full architecture, all eight layers, both spines, and the five clusters, is in the Reader’s Guide, free at andybea.ch/reference-architecture. The companion Workflow Map covers the surface, sixteen stages and the use cases inside them. This piece is the substrate beneath it. Come argue with me about the tiles. That is what they are for. But be ready to buy the cocktails.