# 🎥 Factorize Vision

> Walk the aisle with a phone — get product photos, names, prices, and filming coaching.

> **Suite identity:** icon 🎥 · accent `#d86bb1` · part of the [Factorize suite](../README-FACTORIZE.md)

## What it does

Drop a video of your inventory on the Vision Intake admin page (Factorize →
Vision Intake). The browser captures frames locally and sends them to **your
own Ollama server** — nothing leaves your network. A vision model finds the
product in each frame; the plugin crops it with the model's bounding box,
suggests a retail name and an industry-typical price, and scores the video
itself: too fast (blurry frames), too slow (lingering), gaps where products
were probably missed, and condition warnings ("box crushed", "label
obscured"). Each detection can be saved to the media library and turned into
a draft WooCommerce product in one click.

## Whole-video coverage & several products per frame

By default the plugin analyzes the **entire video**: one frame slot per
`frame_interval` seconds across the full duration (a 60s video at 2.5s is
~25 frames), each slot the sharpest of three candidates. Untick *Analyze the
whole video* to fall back to the fixed `max_frames` clip budget (first N×
interval seconds only) — the estimate line says which seconds are covered.

Each frame may contain **several products** (a shelf, a bin, a table). The
model returns a list per frame; every product gets its own bounding-box crop,
its own row, and its own vote across passes. Sightings of the same product
across frames are pooled by name, so a product seen in six frames is still
one review row (the sharpest, largest, most-agreed sighting is the photo).

## Count mode — every unit its own row

The **Mode** selector switches between *Catalog* (name products; sightings
of the same name across frames pool into one row — the right thing when you
want a product list) and **Count** (recall first: every physical unit is
its own row, names optional). Count mode changes three things:

- **Denser frames** — one slot per `count_interval` (1.25s) instead of 2.5s,
  so a fast pan cannot skip a unit.
- **Tiles** — each frame is analyzed as `count_tiles` overlapping strips
  along its long side (top/bottom halves of a portrait phone video). Fewer
  items per call keeps the 7B model enumerating instead of looping into an
  invented grid, and small items fill more of the picture. Calls scale with
  tiles; the estimate line says so.
- **Products and units** — sightings are pooled across the whole video by
  fuzzy name (same head noun, half the words shared, brands must agree —
  a Husky tool box never merges into a Milwaukee one), and each product's
  **units** is its highest count within a single frame: six Packout boxes
  listed side by side in one frame means six, seeing them again in the next
  frame does not add more. Brand-only or unnamed sightings are kept as
  "Item N — hint" rows instead of being quarantined, because here the crop
  and the count are the deliverable. (Per-box visual tracking across frames
  was tried first and dropped: the 7B model's boxes are largely invented, so
  it produced 316 "units" from a ~42-product video.)

Rows come out numbered with a units badge and a "seen in N frames" count;
the CSV export carries row number, units, and frames seen.

Measured on a 58-second bin-store walk the owner put at ~42 products:
**53 distinct products · 165 units** in 32 minutes (92 calls, 60k tokens),
40 of them confirmed in more than one frame; the overshoot is a few
over-splits (mis-read labels, two unnamed pools) that the review table is
there to catch. Details in `tests/VERIFICATION.md`.

## Factorize tokens — the monthly allowance

Every plugin in the suite that runs on your own server is free. The AI
feature — this one — is metered in **Factorize tokens: 1 token = 1 model
call** (one frame, tile, or pass sent to the vision model). Every store
gets **50 tokens per calendar month** included. The intake page shows the
meter (used / left) under the server line, the estimate line says how many
tokens a run will take and warns when it exceeds what is left, and when the
allowance runs out mid-run the run stops cleanly, keeps the rows it already
found, and shows who to contact. **Need more? Text or call 330-519-9231.**

Usage lives in the `factorize-vision_usage` option (`{ym, calls}`), which
resets when the month changes. **With a license key activated in the
Factorize hub**, the Factorize license server becomes the source of truth
instead: the allowance and the month's usage are synced from it, every
model call is recorded there (`empire_suite_license_tokens( 1 )`), and a
call the server denies stops the run — so the metered part cannot be edited
away on the store. If the server is unreachable the last synced figures
carry the site until it is back. `monthly_allowance`, `allowance_contact`,
and `allowance_unlimited` are config keys — a site with an arrangement for
more tokens gets `allowance_unlimited` set (or a higher allowance) via the
option or the `factorize-vision_config` filter.

Not to be confused with **model tokens** — the words the Ollama model reads
and writes per call, which the meter also reports for capacity planning
(~750 per call) but which are not billed.

## Intensity meter & token system

An **Intensity** selector next to the Analyze button controls how many
analysis passes each frame gets:

| level | passes/frame | how it works | accuracy target |
|---|---|---|---|
| 1 · Quick | 1 | single greedy pass (temperature 0.1) | ~80% |
| 2 · Cross-check | 2 | second warmer pass; disagreements flagged | ~90% |
| 3 · Consensus | 3 | three temperature-varied passes **vote**: majority on product/blur, name groups elect a leader, median price, largest agreeing bbox | 95–99% (with review) |

Measured on a 61-second, three-shelf video (25 frames, qwen2.5vl:7b on a
GPU — full write-up in `tests/VERIFICATION.md`):

| | intensity 1 | intensity 2 | intensity 3 |
|---|---|---|---|
| model calls / tokens / minutes | 25 / 18k / 8.5 | 50 / 36k / 17.6 | 75 / 52k / 25.3 |
| real products found | 13 | 15 | 15 |
| unanimous rows | — | 12 of 15 | 7 of 15 (all 98–100% catalog matches) |
| rows flagged for a human look | — | 8 | 5 |

Level 1 finds the inventory; level 2 sharpens names and labels confidence;
level 3 tells you which rows you can add without a second look (3/3) and
which are guesses (1/3, start unchecked).

Every model reply reports its token counts, so the page shows a **live token
meter** while analyzing, a pre-run estimate (≈750 reported tokens per call
with multi-product lists; ~20s per call GPU-resident, 45–60s on CPU), and a
per-run total in the feedback panel. The review table and CSV export carry a
per-product consensus column ("3/3 passes agree"); rows where passes
disagreed are called out in the feedback so they get a human look before
anything is added. Tokens are local compute (your Ollama box), not billed.

A **Run analytics** panel on the same page keeps the last 50 runs: intensity,
frames, model calls, tokens in/out, wall time, products found, consensus %,
blurry frames, and errors — so you can see exactly what an extra intensity
level buys on your hardware.

## Configuration

| key | default | effect |
|---|---|---|
| `ollama_host` | `192.168.1.6` | LAN address of the Ollama server |
| `ollama_port` | `11434` | Ollama port |
| `model` | `qwen2.5vl:7b` | preferred vision model (used when installed) |
| `fallback_model` | `moondream` | pulled on demand when the server has no vision model |
| `frame_interval` | `2.5` | seconds between captured frames |
| `max_frames` | `8` | clip budget when whole-video is unticked (one model call per frame per pass) |
| `whole_video` | `true` | analyze the full duration by default |
| `whole_video_cap` | `60` | safety cap on frame slots for very long videos |
| `frame_size` | `512` | longest frame side in px (smaller = faster) |
| `intensity` | `1` | default analysis passes per frame (1–3), selectable per run |
| `monthly_allowance` | `50` | Factorize tokens included per calendar month (1 token = 1 model call) |
| `allowance_contact` | `330-519-9231` | shown wherever the allowance is — who to ask for more |
| `allowance_unlimited` | `false` | lift the cap on a site with an arrangement for more |
| `mode` | `catalog` | default mode: `catalog` (one row per product) or `count` (one row per unit) |
| `count_interval` | `1.25` | seconds between frame slots in count mode |
| `count_tiles` | `2` | overlapping tiles per frame in count mode (1–4) |
| `count_max_items` | `15` | cap on units the model may list per tile |
| `num_ctx` | `4096` | context window per call — qwen2.5vl advertises 128k and Ollama would allocate tens of GB for it on CPU |
| `num_predict` | `700` | hard cap on reply tokens (a 6-product JSON list is well under it) |
| `keep_alive` | `15m` | how long Ollama keeps the model warm |
| `timeout` | `600` | per-call timeout, seconds (first call reloads the model) |

Override via the `factorize-vision_options` option or the
`factorize-vision_config` filter (the legacy `empire-vision-intake_options`
option is still honored).

**Why `num_ctx` matters (measured):** without it, Ollama sized qwen2.5vl:7b
for its advertised 128k context — a 51.9GB allocation that could not fit the
GPU, so the model ran on CPU (~40s/frame, 5–7 minute cold loads, constant
evictions). With `num_ctx 4096` the same model is 15.4GB, fully GPU-resident,
and generates ~24 tokens/s — roughly 15–20s per frame including the image.
If your frames are larger than 512px or you raise `num_predict`, raise
`num_ctx` proportionally (a 512px frame is ~330 image tokens).

## Works with

Standalone. The bundled Factorize hub puts the page at Factorize → Vision
Intake — this plugin alone stands up the hub, and its card registers under
the Insights tab. The screen requires the `manage_woocommerce` capability,
and draft-product creation needs WooCommerce.

## Hooks it registers

`admin_menu`, `admin_enqueue_scripts`, and AJAX: `et_vision_status`,
`et_vision_prepare` (model pull), `et_vision_frame` (analysis, pass-aware),
`et_vision_dupcheck` (inventory duplicate check), `et_vision_runlog`
(analytics), `et_vision_save` (media), `et_vision_draft` (draft product).

## Publishing and license

`readme.txt` (listing-format description) and `QUICKSTART.md` (5-minute
setup guide) live alongside this file. The plugin is distributed from the
Factorize site under the proprietary **Factorize License** (`LICENSE.txt`
in this folder and in the zip) — not through WordPress.org, whose directory
requires a GPL-compatible license. See `../README-FACTORIZE.md`.
