Making Vision

Making Vision- Version 1

I was just looking how to deal with my personal unhappiness with AI image analysis. A bare basics project guidelines so that AI can do a better job analysing an image without prompt. My personal experiences, phone games (honourable mention to June's journey and Happy Colors-art section) and object oriented architecture created the base. Science verification was done much later when more than half the work was done. Gap testing using Claude, Gemini and Grok while main chat was using Chatgpt.

Vision is often described as though its primary task were to turn photons into recognized objects.

That is too narrow.

An organism does not encounter the world as an empty visual field waiting to be labelled. It already possesses persistent knowledge, spatial maps, object and material knowledge, expectations, learned behaviours, inherited information, social information, and current needs. Vision enters that already-functioning system and supplies current information to it.

But vision is not merely an API into a database either. A genuinely unfamiliar environment can still be navigated. A novel surface can still be matched. Moving neighbours can still be tracked. Free space can still be extracted. Current visual structure can be useful before the organism knows what the thing is.

Making Vision therefore treats biological vision as a high-bandwidth system for extracting structured information from current visual input and integrating that information with an organism's existing knowledge, spatial machinery, expectations, state, and action systems.

The central problem is not:

How does vision recognize everything?

It is:

How does an organism decide what current visual information matters, what it already knows about it, what it can infer directly from the current input, and what deserves further processing or persistence?

The organism does not start empty

Vision does not create the organism's world knowledge from scratch.

Before an organism encounters a particular visual scene, it may already know:

objects
materials
components
properties
characteristic behaviours
affordances
dangers
safe locations
routes
landmarks
social information
inherited responses
learned procedures
recurring environmental relationships

This knowledge can come from different sources.

Inherited knowledge

Some organisms arrive with executable knowledge already available. A newborn does not need to spend its first hours discovering every fundamental behaviour from scratch. Some knowledge is effectively pre-seeded.

The useful architectural distinction is:

know + executable set

rather than:

everything must first be learned through individual experience.

Individual learning

Encountering something repeatedly can turn:

unfamiliar thing

into:

known object/material + properties + behaviours + consequences.

Social learning

One organism can acquire information through another. A behaviour performed by one individual can therefore become information available to others.

This is particularly important for animals living in socially structured environments.

Distributed knowledge

Persistent knowledge need not reside entirely inside one individual.

It can exist across:

individuals
groups
cultures
environmental structures
physical artifacts
recurring social practices.

A sophisticated organism can therefore participate in a larger persistent information system than the contents of its own brain.

The object database is not a visual catalogue

An object database is not simply:

picture → name.

An object can accumulate information about:

appearance
material
texture
smell
taste
components
affordances
danger
safety
characteristic movement
characteristic interaction
methods
consequences
familiarity
location
social significance.

An object therefore has verbs as well as nouns.

A bird does not merely have:

wings.

It has characteristic ways of:

extending, flying, landing, turning, fleeing.

A predator does not merely know:

prey shape.

It may know:

prey movement, escape behaviour, preferred routes, likely reactions.

This matters when current sensory information disappears temporarily.

A predator following prey behind vegetation does not necessarily require a newly invented trajectory-physics module. Its existing knowledge about what that object tends to do can contribute to what it expects next.

The same principle appears in mundane human motor behaviour.

A person can know:

this switch controls that light
this cabinet opens this way
this route leads to the kitchen

and act without reconstructing the entire world from scratch every time.

Object knowledge is individual

Two members of the same species do not necessarily possess the same object/material database.

This became particularly visible through observations of Punch.

Punch kun, having substantial exposure to human environments, could be comfortable with objects that other macaques treated with caution.

A wiper is a good example.

For one monkey:

wiper → familiar material/object → manipulable → acceptable.

For another:

wiper → unfamiliar strange thing → uncertainty → caution.

The physical object is identical.

The persistent knowledge attached to it is not.

The same applies to human-raised animals interacting with phones, furniture, tools, containers and other human-world objects.

Thus:

same species + different developmental boards → different persistent object knowledge → different future promotion and behaviour.

Object familiarity is therefore not merely an objective property of the object.

It is partly a property of the relationship between object and organism.

The map is not the photograph

An organism can maintain persistent spatial knowledge without maintaining a detailed visual recording of every moment.

A spatial map can contain:

locations
layout
routes
landmarks
stable relationships
dangerous places
recurring destinations.

The map tells the organism:

where things generally are.

The current board tells it:

what is there now.

These are different.

A chair being moved does not necessarily require the organism to rewrite its entire room map. A new painting may eventually become part of the familiar room. A tiny trinket may remain a temporary board detail.

A dangerous change may be promoted much faster.

Maplets and the local inset

For immediate action, the organism does not need its entire map simultaneously.

A relevant portion can become active as a:

maplet / local inset.

The local inset combines:

persistent spatial structure
current board information
current body position
immediate action requirements.

It supports things like:

where to step
where to reach
where the door is
where the switch is
where the obstacle is
where the next route segment lies.

This explains how an organism can navigate a familiar place without continuously reconstructing the entire place in high resolution.

The persistent map provides the skeleton. The current sensory system supplies what has changed.

The board is current state

The board is the currently available environment.

It contains both stable structure and transient events.

Examples:

furniture
walls
pathways
people
objects
light
movement
temporary obstacles
temporary opportunities.

The board is not necessarily identical to physical reality.

It is the environment available to this organism through its sensory systems.

A bee and a human do not have identical visual boards because their sensory systems do not admit the same information. UV information can enter one organism and never enter another.

Temporal sampling can differ between species. Polarization information can exist physically without becoming part of a human visual board.

Thus:

the accessible board is constrained by the organism's sensory apparatus before attention and interpretation even begin.

Vision has direct access to current structure

This is one of the most important corrections made during the adversarial testing.

Vision is not merely:

current photons → database lookup.

A genuinely novel environment can still provide immediately useful information.

A human entering a room full of unfamiliar sculptures can still extract:

boundaries
free space
distances
relative positions
obstacles
movement
pathways.

The person does not need to know:

what each sculpture is.

Likewise, an animal can respond to current spatial relationships among moving objects without requiring durable landmarks for every element.

And a camouflage-capable animal can extract current surface properties without first creating a persistent database entry for every novel background.

Therefore vision has at least two complementary sources of usable information:

Current sensory computation

What is present in the current input?

Persistent knowledge

What do I already know that helps interpret or act on it?

Persistent knowledge can dramatically amplify vision.

But it is not a prerequisite for vision to produce useful structure.

Direct visual structure

Current visual input can provide structure such as:

boundaries
contrast
colour distribution
texture
spatial frequency
orientation
relative position
motion
free space
surface continuity
object relationships.

Some of these can be useful without identifying an object.

This matters enormously for camouflage.

A cuttlefish does not necessarily need to think:

“I recognize this exact novel object.”

It may need only enough current visual information to alter its skin appropriately.

The architecture therefore permits:

current visual input → direct feature extraction → immediate downstream use

without requiring:

object recognition → persistent storage → retrieval → action.

This is a core competence of vision, not an exotic exception.

Sensory information is not equally processed

The organism cannot deeply process everything.

There is a broad, cheap sampling process:

lazy scan

Most environmental information can remain cheap. A familiar room contains enormous amounts of information that simply does not deserve active processing every moment.

The organism can rely on:

existing map
existing expectations
existing object knowledge
existing routines

and let most of the board remain in the background.

This is one of the major efficiencies of biological perception.

Managed expectations

The organism does not merely see:

everything that is physically present.

It maintains expectations about the current board.

Examples:

a dining table has certain things on it
a doorway has open passage
a switchboard contains familiar switches
a road contains vehicles behaving within ordinary patterns.

Most expected information therefore requires little active attention. This creates an enormous efficiency gain.

A familiar chair in its familiar location does not repeatedly demand:

“WHAT IS THAT?”

The organism already knows.

Perturbation

A perturbation is information that escapes the cheap background treatment and becomes potentially worth further processing.

It can be:

unexpected movement
unexpected location
missing expected object
unusual object
unusual behaviour
unusual relationship
relevant sensory event
internally generated sensory anomaly.

But:

not every perturbation gets promoted.

This is important.

A change can be noticed without becoming important.

A look-see can happen without resulting in durable memory.

A board can contain thousands of tiny changes that never become meaningful.

Object promotion

The critical transition is:

background → candidate → promoted object/event

Object promotion means:

this is now worth processing.

It does not necessarily mean:

“I know exactly what this is.”

Promotion can precede classification. This distinction becomes especially clear in novel environments.

An animal can approach an unfamiliar object because it has judged:

worth investigating.

The object may not yet have a name, identity or complete classification.

Promotion is therefore an allocation of processing and behavioural resources, not necessarily a finished perceptual verdict.

Promotion is state-dependent

Novelty alone does not determine promotion. The same novel object can produce different behaviour depending on the organism's state.

A child may explore a novel toy when:

secure
regulated
comfortable
mentally available.

The same child may prefer a familiar object when:

uncertain
unsettled
seeking reassurance.

Thus:

novelty ≠ automatic promotion.

Promotion depends on something closer to:

current state × existing knowledge × current board × current needs/constraints

The current state matters because the organism's priorities change with time. A hungry predator and a satiated predator can encounter the same stimulus and allocate attention differently.

A safe, relaxed organism can afford exploration. An organism under threat may have no interest in exploratory information.

An organism that needs reassurance may actively choose the familiar over the novel.

Exploration is an expenditure

Exploration is not free.

To investigate something, an organism may have to spend:

energy
time
attention
exposure
motor effort
risk tolerance.

Therefore an organism's exploration behaviour reflects not merely:

“what is interesting?”

but:

“what is worth spending resources on right now?”

This makes the first approach to a novel object ecologically informative.

If an animal enters an unfamiliar environment and chooses to approach one object rather than another, that behaviour potentially reveals something about:

its existing danger knowledge
material familiarity
affordance knowledge
current need
current state
exploration threshold.

Novel board versus novel object

These are not the same perturbation.

An organism can encounter:

Familiar board + novel object

known room, strange toy.

Novel board + familiar object

known object in unfamiliar place.

Novel board + novel objects

almost everything is uncertain.

The behavioural response to these situations can differ. This distinction appeared clearly in developmental research and in animal exploration work.

It is also important for interpreting animal introductions to new environments.

The first explorer changes the experiment

When a new object enters a social environment, the first animal to interact with it can change what everyone else knows.

The sequence becomes:

new object → individual exploration → observed behaviour → social information → later exploration.

Therefore:

video zero matters.

If the first monkey touches, enters, climbs or plays with a new object before others see it, that is evidence about individual first-contact behaviour.

Later monkeys may be responding to:

the object plus the social information supplied by the first monkey.

This is why the introduction of a new shallow pool, new play material or new object is most informative when the earliest encounters can be reconstructed.

Persistence is selective

Promotion does not automatically create persistent knowledge.

An organism can:

notice → inspect → act → forget.

For persistence to be useful, there is a cost.

Persistent information requires:

storage
biological wiring
retrieval
updating
maintenance.

Species differ in how much information they can economically maintain.

Individuals also differ in what they have accumulated. Therefore the object database should not be imagined as infinitely expanding.

It is a managed persistent resource.

Persistence depends on relevance and attachment

Some information becomes durable because it is repeatedly useful. Some becomes durable because it is strongly attached to a location.

Some becomes durable because the consequences are enormous.

A tiny object with strong attachment can persist. A large temporary object can disappear from memory quickly.

Spatial layout and landmarks tend to have different persistence properties from temporary board objects.

A landmark can become:

location + object + relationship

rather than simply:

object.

Self-created perturbations

Humans can deliberately edit the board to exploit this machinery.

Suppose something must be soaked overnight.

Instead of relying entirely on:

“remember to soak this later”

a person can place the object beside the water bottle.

The intended chain becomes:

future water trip → predictable foot traffic → familiar landmark encountered → unexpected object beside it → object promotion → task association recovered.

The object is functioning as a temporary task landmark.

Its identity is not necessarily important. A shoe, bottle, box or piece of paper can work.

The important feature can simply be:

something is visibly interrupting an expected pathway.

The person has edited the board so their future self will encounter a perturbation at the right place and time.

Board editing can be cheaper than memory

This generalizes.

If a person has to remember:

“I must do X later”

they can instead arrange:

“Later, I will inevitably encounter something that reminds me.”

This is an extremely cheap form of externalized memory.

It also explains why humans deliberately maintain:

fixed places for keys
fixed places for tools
fixed locations for household supplies
visible task objects
route-linked reminders.

The environment becomes part of the retrieval system.

The wet-floor experiment

A spilled floor gives another simple example.

The immediate problem is not:

dry floor.

It is:

don't step there while it is slippery.

A conspicuous wet patch can act as a temporary landmark:

board change → visible perturbation → promotion → altered foot placement.

If evaporation can solve the underlying problem, the human does not necessarily need to spend additional energy mopping immediately.

The intervention can be:

edit the board enough to prevent the dangerous action.

This is a very small example of a much broader principle:

organisms do not have to solve every environmental problem directly if they can cheaply alter their interaction with it.

Camouflage: avoiding promotion

Camouflage provides an unusually clean demonstration of object promotion. The camouflaged organism is not necessarily trying to make the predator identify it as another animal.

Often the desired outcome is much simpler:

I am background.

The organism wants to remain:

scenery
surface
stationary structure
environmental texture
background noise.

In other words:

Don't see me.

Or more precisely:

Don't promote me.

The organism is trying to avoid becoming the perturbation that causes the observer to allocate further processing.

Camouflage is not necessarily “pretending to be another species”

This distinction matters.

A leaf insect does not need to convince a predator:

“I am definitely Species X.”

It can be enough to maintain:

“nothing here is an actionable animal.”

Appearance matters.

Behaviour matters.

Movement matters.

Expected background behaviour matters.

A creature that visually resembles a leaf but suddenly moves like an animal may generate exactly the perturbation that defeats the camouflage.

Thus camouflage exploits the observer's existing machinery:

background expectation → candidate detection → promotion

by trying to remain on the background side of the boundary.

This is the cleanest sense in which camouflage is faking vision:

“I need to project that I am part of the scenery.”

Camouflage is conditional

Camouflage-capable organisms do not necessarily maintain maximal camouflage constantly.

The behaviour is state- and need-dependent.

The organism can remain in an ordinary state until:

exposure risk / predator / environmental condition

makes camouflage worthwhile.

Then:

current sensory input → processing → skin/body state change.

This is another example of:

capability existing continuously while its expensive deployment is conditional.

Direct structure and persistent knowledge are complementary

The architecture therefore has two major sources of information.

Current visual computation

Provides:

what is physically structured in the current visual input?

Persistent knowledge

Provides:

what do I already know about what I'm seeing and where it belongs?

Neither replaces the other.

A genuinely novel room demonstrates direct structure.

A familiar bedroom demonstrates the power of persistent knowledge.

A novel toy demonstrates learning and persistence.

A familiar route demonstrates maplets and procedural behaviour.

A camouflaged animal demonstrates the interaction between current surface information and another organism's promotion machinery.

Vision does not have to classify everything

This is perhaps the most important conceptual correction.

A raw visual scene does not have to become:

object A
object B
object C
object D

before useful behaviour can happen.

Current visual computation can directly support:

free-space navigation
relative movement
surface matching
obstacle avoidance
tracking
body/environment relationships.

Meanwhile, object promotion selectively allocates additional processing.

Thus the system can be simultaneously:

high-bandwidth in acquisition

and:

highly selective in expensive interpretation.

There is no contradiction.

Behaviour is part of object knowledge

Objects are not static visual identities. They have expected behaviours.

This is why a bizarre behaviour can become a perturbation even when the object itself is familiar.

Examples:

familiar animal → unexpected movement
familiar bird → acoustically unexpected vocalization
familiar object → wrong interaction
familiar switch → unexpected outcome.

A seagull producing a cat-like meow is not merely:

“strange sound.”

It can be understood as:

acoustic mis-verb.

The object and the expected behaviour have come apart. This can be more informative than visual novelty.

Map staleness is not a catastrophic failure

Persistent maps inevitably become stale. A familiar switch arrangement can change.

Furniture can move.

A light can stop working.

A person can learn a workaround.

For example:

old switchboard → failure → wall lamp adopted → new motor routine.

Later:

original light restored → old routine becomes valid again.

The organism can retain both old and newer learned behaviours and fall back according to context.

This is ordinary interaction between:

persistent knowledge + current board + motor routine.

It does not require continuous total-map auditing.

Motor behaviour can bypass intensive interpretation

Well-practiced actions can operate largely from procedural knowledge.

Examples:

chores
familiar routes
repetitive household actions
practiced motor sequences.

This explains phenomena such as performing a familiar action while attention is elsewhere.

It also explains why changing the arrangement of a familiar environment can cause absurd mistakes:

learned motor expectation → current board mismatch → wrong action.

Again, the problem is not necessarily that vision failed.

The problem can be:

old expectation + new board.

Sensory systems can disagree

Different sensory systems can provide conflicting information.

An elevator can produce:

visual movement
vestibular information
proprioception
touch.

The organism integrates these inputs.

A person may experience:

dizziness
nausea
spatial uncertainty

because different sensory channels are supplying different information.

This does not require one universal sensory authority.

Likewise, a glass door can produce:

visual “open space”

followed by:

physical collision.

The resulting contradiction becomes an extremely strong board event.

The organism learns:

this apparent open region is physically occupied.

This is a normal correction process, not evidence that vision must possess absolute authority over spatial reality.

Internal sensory events

Not every sensory event comes from an external object.

Examples include:

phosphenes
illusions
hallucination-like percepts
sensory noise.

The architecture does not need to prevent these events from entering processing.

Instead:

classification is not guaranteed truth.

An internally generated visual event can still become:

“something is happening.”

The organism can investigate, revise or dismiss it.

Sleep is not sensory death

Sleep can be treated as an:

idle/deep-repair state

rather than a complete shutdown of environmental monitoring.

Relevant sensory events can still register.

Examples include:

touch → awakening
cold extremity → awakening
environmental movement → awakening
smell → possible awakening.

But sensory input does not necessarily wake the organism. This implies that some processing can occur below the awakening threshold.

Sleep is also a period in which persistent information can be:

consolidated
reorganized
refreshed
updated
maintained.

The organism can therefore be:

offline from ordinary interaction

without being:

offline from information processing.

Internal projection is broader than vision

Dreaming should not be turned into a special theory of vision.

The interesting observation is broader:

the brain can generate internally projected sensory/spatial content without current sensory input.

Echolocation already demonstrates something different:

nonvisual sensory input → spatial representation.

Dreaming demonstrates:

stored/internal material → internally generated scenes and events.

This suggests that internally generated projection is a broader brain capability than visual perception itself.

Making Vision only needs to acknowledge that current external visual input is not the only possible source of visual-like content.

What vision adds

Vision is unusually powerful because it can provide large amounts of structured information about the environment at a distance.

It can provide, in parallel:

spatial relationships
boundaries
motion
appearance
objects
surfaces
relative position
changes.

But its power should not be defined as:

“it owns spatial knowledge.”

Spatial knowledge can be acquired through other senses.

Vision's distinctive contribution is the bandwidth and structure of current visual acquisition.

The full working architecture

The current model can therefore be represented approximately as:

This is not intended as a literal neural wiring diagram.

It is an architectural description of functional relationships.

The central loop

The core loop is therefore not:

see → identify → remember.

It is closer to:

current input → cheap scan → candidate change → promotion when warranted → deeper processing → action and/or persistence

while persistent knowledge and current state continuously influence what counts as worth promoting. And direct current sensory computation can operate alongside this:

current input → useful structure → immediate action

without requiring a database lookup first.

Why familiar environments feel rich without constant detailed processing

A familiar room can feel immediately rich when the lights come on even though the organism does not maintain a full-resolution photograph of every object.

The persistent map supplies:

structure.

The local inset supplies:

relevant local organization.

Current vision supplies:

current details.

Expectations suppress the need to deeply inspect the majority of unchanged information.

Then, when something is wrong:

object moved
object missing
unexpected movement
unexpected obstacle

the perturbation can be promoted.

This gives the organism an economical compromise:

remember the skeleton, sample the present, investigate the exceptions.

The ecological significance of novel-board exploration

A genuinely unfamiliar board is therefore an unusually useful experimental condition.

The organism has less map support.

It must rely more heavily on:

direct current sensory structure

while still carrying its existing:

object database
danger knowledge
inherited defaults
material familiarity
state-dependent needs.

What it approaches first is therefore potentially informative.

A newly introduced object may be:

ignored
avoided
inspected
touched
approached
manipulated.

The first voluntary investigation is effectively a behavioural form of object promotion.

And the object chosen first may tell us something about what the organism's existing machinery considers:

safe enough
interesting enough
useful enough
familiar enough
worth the energy/risk.

This is why first-contact observations are more informative than merely watching animals after everyone has already explored the new object.

Social observation can change the database.

Novelty is not the same as danger

A genuinely novel object does not necessarily receive:

danger classification.

Nor does novelty necessarily produce:

exploration.

The organism's response depends on its existing database and current state. A species with a sparse object/danger database may behave differently from one with extensive accumulated knowledge.

An individual with extensive human-world exposure may treat an object as ordinary that another individual considers strange.

The same physical board can therefore produce different behaviour in different organisms.

Persistence is an energetic architectural choice

A persistent database is not free. Every durable distinction potentially requires:

storage
connectivity
retrieval
maintenance
updating.

Therefore organisms have reason to be selective about what becomes persistent.

This explains why:

location and landmark structure

can persist while:

transient decorative detail

does not necessarily survive.

It also explains why a dramatic event can be retained after one exposure while a trivial object disappears after repeated exposure.

Persistence is therefore not simply:

number of times seen.

It is:

what the organism has reason and capacity to retain.

Vision and the object database are reciprocal

The object database helps vision interpret current input.

But current visual encounters also update the object database.

This creates a loop:

existing knowledge → interpretation/promotion
current encounter → learning
learning → modified future promotion.

That means the system is not static.

Every sufficiently meaningful encounter can change what future vision does with the same object.

The animal that once investigated a wiper may later treat the wiper as ordinary.

The child who learns a toy's mechanism no longer needs to discover its affordances from scratch.

The predator that learns a prey pattern can subsequently detect it more efficiently.

The organism is continually changing the machinery that determines what it will notice next.

Making Vision is therefore not “vision creates a world”

The organism already has a world of:

memory
maps
objects
methods
expectations
constraints
social knowledge
inherited knowledge.

Vision continuously supplies current information to that system.

But it also does something equally important:

it extracts structure that has never been stored before.

That is why neither extreme is sufficient.

Too little:

Vision is just a camera feeding a database.

Fails on genuinely novel visual structure.

Too much:

Vision constructs the entire world independently every moment.

Fails to explain the enormous efficiency of maps, expectations, object knowledge and procedural behaviour.

The useful architecture is between them:

current visual computation + persistent knowledge + selective promotion + state-dependent allocation.

The simplest statement of Making Vision

A biological visual system does not need to deeply understand everything it sees.

It needs to:

Acquire current visual structure.
Use existing knowledge where available.
Maintain expectations about the current board.
Cheaply scan most of the environment.
Detect candidate perturbations.
Promote some candidates for deeper processing.
Use current state and needs to determine what deserves that promotion.
Extract useful structure even when no prior object knowledge exists.
Feed current information into spatial, motor and other sensory systems.
Persist sufficiently meaningful discoveries into existing knowledge and maps.
Allow future perception to be altered by what has been learned.

The result is neither a camera nor a database.

It is a selective, state-dependent interface between current visual structure and a persistent organismic system.

The particularly useful insight

The strongest insight to carry forward is perhaps this:

Being physically present is not the same as becoming perceptually important.

A chair can occupy the visual field without becoming an object of active processing.A camouflaged animal can be physically present without becoming an actionable object. A spilled patch of water can become highly important because it interrupts an expected walking path.

A familiar object can become a task reminder because a person deliberately places it where future foot traffic will promote it. A novel object can become worth exploring because the organism's current state makes investigation affordable.

The organism therefore does not merely perceive objects.

It continuously decides, through the interaction of sensory input, expectations, knowledge and current state:

what deserves to become an object of attention and action.

That is the core problem Making Vision is trying to explain.

V1 boundary

Making Vision does not attempt to explain all cognition.

It does not attempt to fully explain:

consciousness
dreams
all motor control
all decision theory
pathology
every sensory illusion
every evolutionary adaptation
social learning as a complete field
distributed intelligence as a complete field
all forms of memory.

Those systems intersect with vision.

They are not all the problem.

The project is deliberately narrower:

How does an organism turn current visual input into selectively useful information, given that it already possesses persistent knowledge, maps, expectations, current state, and action machinery?

And the current answer is:

By combining direct current visual computation with persistent knowledge and spatial structure, using lazy scanning and selective object/event promotion to allocate expensive processing according to expectation, perturbation, need, state, and consequence.

References and comparison to currently available science and research.

Page updated

Google Sites

Report abuse