Faking Vision

Faking Vision is a method for representing what is visually present in a form that can be reconstructed, edited, tested, and compared.

Instead of treating an image or video as something that must simply be copied, Faking Vision breaks it into a structured representation of objects, instances, properties, spatial relationships, and—in video—temporal changes and persistent identities.

The resulting representation is an FV packet.

The packet becomes a closed description of what the reconstruction is supposed to contain. A renderer may interpret that description differently or fail to reproduce it precisely, but it is not allowed to silently add new semantic content to make the result seem plausible.

From Making Vision to Faking Vision

Faking Vision is downstream of Making Vision.

Making Vision is concerned with developing a working, scientifically defensible account of how visual information can be constructed and used.

Faking Vision takes relevant principles from that work and turns them into something operational:

visual source → structured representation → reconstruction → comparison

The purpose is not to claim that the packet is a perfect copy of nature's internal representation. It is to create an explicit representation that can be inspected, modified, reconstructed, and tested.

What is an FV packet?

An FV packet separates different kinds of information instead of collapsing them into a single description.

For an image, the core packet contains:

• OI — Object Inventory: what entities and instances are present

• MSRM — Map Spatial Relationship Manager: where those entities are and how they relate spatially

• PC — Project Constants: optional genuine project-wide constants

For video, the packet additionally contains:

• TD — Temporal Data: what changes, when it changes, and how persistent identities behave over time

TD makes the video representation temporally queryable rather than leaving time as an implicit property of the recording.

The closed-world principle

A central rule of Faking Vision is simple:

The packet is the reconstruction source of truth.

If an object is not in the closed inventory, the renderer does not get to invent it.

A background does not automatically authorize the objects normally associated with that background. A dark shape that merely resembles a bag does not become a bag unless the evidence supports promoting it to an object. An unresolved observation remains unresolved.

Likewise, moving an object does not create a second object. An object leaving the frame does not necessarily cease to exist. Persistent identities remain persistent unless the representation provides evidence otherwise.

This is important because a reconstruction can otherwise look perfectly coherent while being wrong.

Editing the representation

Once a scene has been represented as an FV packet, the representation itself can be changed.

Objects can be moved, removed, or added where appropriate. Their properties and components can be changed. Spatial relationships can be altered. In video, persistent objects can change position, state, or visibility over time.

The resulting packet describes a new reconstruction target.

The rendered result is therefore not the thing being edited. The structured representation is.

That distinction allows reconstruction experiments to ask much more precise questions:

What happens if this object moves?

What happens if this object is removed?

What happens if two objects exchange positions?

What happens if an identity persists while moving off-screen?

What happens if an uncertain observation is deliberately resolved one way or another?

The packet makes those changes explicit rather than burying them inside an image-generation prompt.

Reconstruction is a separate experiment

Faking Vision does not require a particular renderer or rendering technology.

The packet can be compiled into whatever representation a particular renderer requires. That compilation is itself an auditable step: it must preserve the packet's identities, spatial relationships, scene membership, temporal information, and uncertainty without adding outside knowledge.

This creates separate points at which a reconstruction can fail:

source analysis → packet → compilation → rendering → evaluation

A poor rendered result does not automatically mean the packet was wrong.

The packet may be correct and the renderer may simply have failed to reproduce it. Conversely, a renderer may produce a convincing image from a packet that was itself wrong. Faking Vision keeps those failures distinguishable.

Why “faking”?

Because the goal is not necessarily to reproduce the original pixels.

The goal is to see whether a structured representation can cause a reconstruction to contain the right things, in the right relationships, with the right changes over time.

A reconstruction can therefore be visually convincing while being semantically wrong.

That makes the name deliberate:

Faking Vision is about testing how far a visual representation can be pushed to produce a convincing reconstruction—and discovering where that representation, its compiler, or its renderer goes wrong.

The short version

Faking Vision turns visual observations into editable, reconstructable representations.

Images become structured packets of objects and spatial relationships.

Videos add persistent identities and temporal data.

Those packets can then be edited and compiled into reconstruction instructions without allowing the reconstruction system to quietly invent the missing world.

The representation is the experiment. The rendering is the test.

Repo link