MediaShelf/docs/design.md

# MediaShelf — Software Design

**Status:** v1.4 — deployed · **Author:** Claude (spec) for Jess · **Date:** 2026-09-10

*Revision 1.1 — Tautulli (192.168.1.100:8181) confirmed present and promoted to the
primary watch-history source; Plex's own history demoted to fallback. Deployment path
settled: image built in CI/locally and pushed to Nox's Portainer.*

*Revision 1.4 — implemented and deployed. Running on Nox as Portainer stack 113
(`mediashelf`, host port 8086), built in place from a git stack rather than a registry
(§11.2). Not yet behind NPM.*

*Revision 1.3 — keep marks (§6.6) generalised from the earlier protected-libraries idea:
one `keep` concept applied at library, show, season or movie level, keyed on content guid
rather than Plex rating key so marks survive library rebuilds, kept items hidden from the
grid by default, and refused outright by v2 deletion.*

*Revision 1.2 — **validated against the live servers.** The measured library (§2.1) is
65.7 TB across 24 libraries with 60 users and 87,640 logged plays. Five corrections
followed: Tautulli's per-item table cannot supply TV sizes (§4.11), both Tautulli-derived
score weights recalibrated against real distributions (§6.1), cross-library duplicates
promoted to v1 (§6.5), keep marks added (§6.6), and confident-vs-uncertain
reclaim separated as distinct seed views (§7.2).*

---

## 1. Overview

MediaShelf is a self-hosted web application that reads a Plex Media Server library,
builds a local analytical snapshot of every item in it, and gives a human a fast way
to answer one question:

> *Which of these thousands of files are worth keeping, and which are burning disk
> for nothing?*

It does this by joining together facts Plex already knows but never shows side by
side — when a file was added, how large it is, where it lives on disk, which library
it belongs to, how many times it has been played, and when it was last played — and
exposing them as a sortable, filterable, chartable grid with a tunable "reclaim
score" that ranks deletion candidates.

**v1 is read-only.** MediaShelf will never delete, move, or modify a file or a Plex
record in this version. Its output is a report: an on-screen ranked list, saved rule
sets, and CSV export. Actual deletion is designed for in §14 but deliberately not
built yet, so the scanner and scoring model can be trusted before anything
destructive is wired up.

### 1.1 Goals

- Full-library ingest from Plex over its HTTP API, with no agent installed on the Plex host.
- Watch history from **Tautulli**, which has been logging every play since it was
  installed and knows *how much* of each item was actually watched — not just that a
  play started.
- A durable local snapshot so the UI is instant and Plex isn't hammered on every click.
- Movies at item level; TV rolled up to **season** level.
- Sort/filter/visualize on: date added, last watched, watch count, size on disk, file path, library.
- A weighted **reclaim score** with live sliders, so "what should I delete" is one sort click.
- **Saved views** — named, re-runnable rule sets like *unwatched, older than 2 years, over 10 GB*.
- CSV export of any view.
- A provider abstraction clean enough that Emby and Jellyfin drop in later without
  touching the scoring, storage, or UI layers.

### 1.2 Non-goals (v1)

- Deleting, moving, or trashing files. Not in v1. See §14.1.
- Writing anything back to Plex (no rating changes, no collection edits, no unmatch/refresh).
- Authentication or multi-user accounts — this is a LAN-only, single-admin tool. See §12.
- Music, photo, and "other video" libraries. Movie and Show sections only.
- Direct filesystem access. MediaShelf never mounts the media shares. All size and path
  data comes from the Plex API. See §4.4.
- Transcode/quality analysis, duplicate detection, subtitle management.

### 1.3 Decisions already made

| Question | Decision |
|---|---|
| Deletion in v1 | No — report only |
| Stack | Python + Flask, SQLite |
| Deployment | Portainer stack on **Nox** (192.168.1.77), published via Nginx Proxy Manager |
| Auth | None; LAN-only, single admin view, watch data aggregated across all Plex accounts |
| TV granularity | Season |
| Size/path source | Plex API only |
| Watch-history source | **Tautulli** primary, Plex session history as fallback |
| v1 output | Sortable grid + filters + charts, tunable reclaim score, saved rule sets |
| Image delivery | Built outside Nox, pushed to Nox's Portainer |
| Implementation | Design doc only for now; code to follow |

---

## 2. Context

The Plex server is **Loki** (192.168.1.10, Dell R710, Ubuntu 16.04, Plex 1.43.3) with
roughly 90 TB of RAID-6 storage across a Dell MD1000 vault. A second vault is powered
down awaiting drive upgrades, so reclaiming space on the live vault has real value —
this tool exists because buying disks is the more expensive alternative.

MediaShelf runs on **Nox**, not on Loki. Loki's job is serving media; MediaShelf is a
polite HTTP client that talks to it across the LAN and keeps its own database locally.
This also means MediaShelf survives Loki reboots, Plex upgrades, and the eventual
migration of services onto Enki.

### 2.1 The library, measured

Everything below was measured against the live servers on 2026-09-07 with
`tools/probe.py` and `tools/reclaim_preview.py` (§13.1). These are not estimates.

| | |
|---|---|
| Total size | **65.7 TB** |
| Libraries | **24** — 16 movie sections, 8 show sections |
| Movies | 2,930 across 16 sections |
| TV | 44,455 episodes → **2,807 seasons** across 664 shows |
| Grid rows at season granularity | **5,727** |
| Files (parts) | ~47,900 |
| Multi-file items | 640, concentrated in Documentary (338 of 2,511 parts) |
| Storage roots | `/mnt/titan4` 40.6 TB · `/mnt/titan3` 24.3 TB · `/mnt/titan1` 849 GB |
| Tautulli history | 87,640 plays since **2025-03-08**, 24,204 distinct items, 60 users |
| Never played | **27.4 TB (41.7%)** — of which 6.5 TB confident, 20.9 TB uncertain |

Four things here materially changed the design.

**This is not a household server.** 60 distinct users appear in the history. Every
component calibrated for a family of three or four had to be recalibrated (§6.1), and
"who else watches this" becomes a real signal rather than a rounding error.

**80% of the library predates Tautulli.** Coverage begins 2025-03-08, but content goes
back to 2000. So of the 27.4 TB never played, only 6.5 TB is *confidently* unwatched —
added while Tautulli was watching and never touched since. The other 20.9 TB might have
been watched in 2019 and nobody can tell. The `pre_history` flag (§4.11) is therefore not
an edge case, it is the majority state, and the UI must make the distinction impossible
to miss.

**24 libraries, not two.** Several are genre splits of the same kind of content (Horror,
Comedy, War, Superhero, Classic), and *Movies* / *4K Movies* frequently hold the same
film twice. Cross-library duplicate detection moves from "later candidate" to a real v1
concern (§6.5).

**Some content must never be a deletion candidate, and no data can tell you which.**
*Family Videos* is 63 items and 38 GB of what looks like home video, 86% of it "never
played" and therefore ranked highly by a naive reclaim score. The same applies for reasons
the tool cannot see at all: content kept deliberately in case someone wants it later, or
held at 4K because re-acquiring it is expensive. Keep marks are a first-class concept
(§6.6) — and because the tool cannot infer any of this, it does not try: nothing is kept
until a person says so.

---

## 3. Architecture

```
┌──────────────────────────── Nox (192.168.1.77) ────────────────────────────┐
│                                                                            │
│   ┌─────────────────── mediashelf container ───────────────────┐           │
│   │                                                            │           │
│   │   Flask app (gunicorn, 2 workers)                          │           │
│   │     ├── /            → UI (server-rendered shell + JS)     │           │
│   │     ├── /api/*       → JSON API                            │           │
│   │     └── /healthz                                           │           │
│   │                                                            │           │
│   │   APScheduler (in-process, 1 worker only)                  │           │
│   │     └── nightly scan job                                   │           │
│   │                                                            │           │
│   │   Ingestion pipeline                                       │           │
│   │     PlexProvider ──▶ normalize ──▶ upsert ──▶ rollup       │           │
│   │                                                            │           │
│   │   SQLite  /data/mediashelf.db  (WAL)                       │           │
│   └────────────────────────────┬───────────────────────────────┘           │
│                                │ volume: /data                             │
└──────────┬─────────────────────┴───────────────────────────────────────────┘
           │                                        │
           │ HTTP :32400 + X-Plex-Token             │ HTTP :8181 + apikey
           ▼                                        ▼
  Loki (192.168.1.10)                     Isis (192.168.1.100:8181)
  Plex Media Server                       Tautulli
  · libraries, items, parts               · every play since install
  · size, path, added_at                  · per-user, per-item, % complete
  · session history (fallback)
```

### 3.1 Component responsibilities

**`mediashelf.providers`** — the pluggable source layer. Defines two protocols:
`MediaProvider` (libraries, items, parts) implemented by `PlexProvider`, and
`HistoryProvider` (watch events) implemented by both `TautulliHistoryProvider` and
`PlexHistoryProvider`. Everything above this layer speaks only in MediaShelf's own
normalized dataclasses. Splitting the two protocols is what lets watch history come
from a different box than the library itself — which, given Tautulli lives on Isis and
Plex on Loki, is not hypothetical. It is also the seam Emby/Jellyfin slot into (§14.2).

**`mediashelf.ingest`** — orchestrates a scan: pull sections, pull items, pull history,
normalize, upsert into SQLite inside a transaction, recompute season rollups, mark
vanished items, write a `scan` row.

**`mediashelf.scoring`** — pure functions. Takes a row of metrics plus a weight profile,
returns a 0–100 reclaim score and the component breakdown. No I/O, fully unit-testable.

**`mediashelf.rules`** — evaluates saved-view rule sets into SQL `WHERE` fragments via a
whitelisted field/operator grammar. Never string-interpolates user input into SQL.

**`mediashelf.api`** — Flask blueprints returning JSON.

**`mediashelf.web`** — one server-rendered HTML shell; the grid, charts, and sliders are
client-side JS against `/api`.

### 3.2 Why Flask + SQLite

The dataset is small in database terms. A library of ~5,000 movies and ~2,000 seasons
across ~60,000 episodes is a few tens of megabytes. SQLite in WAL mode handles this with
sub-10 ms queries on indexed columns, needs no second container, and backs up by copying
one file. Flask matches the pattern already running for the Mythica portal on Nox, so
deployment, logging, and reverse-proxy conventions carry over unchanged.

The one real constraint SQLite imposes is a single writer. Since the only writer is the
scan job, and scans run one at a time behind a lock, this is not a limitation. It does
mean **gunicorn must run the scheduler in exactly one worker** — see §11.3.

---

## 4. Data sources

MediaShelf reads from two servers. **Plex on Loki** is the authority on what exists —
libraries, items, files, sizes, paths, added dates. **Tautulli on Isis** is the authority
on what has been watched. Sections 4.1–4.7 cover Plex; 4.8–4.11 cover Tautulli.

### 4.1 Plex connection

MediaShelf authenticates with a Plex server token (`X-Plex-Token`) supplied via
environment variable. It talks directly to `http://192.168.1.10:32400` on the LAN — no
plex.tv round trip, no account login, no dependency on Plex's cloud being up.

All requests carry standard client identification headers so the connection is
identifiable in Plex's own logs:

```
X-Plex-Token: <token>
X-Plex-Client-Identifier: mediashelf-<stable-uuid>
X-Plex-Product: MediaShelf
X-Plex-Version: <app version>
Accept: application/json
```

Plex returns JSON when `Accept: application/json` is set, which avoids XML parsing
entirely. Every response body is wrapped in a `MediaContainer` object.

### 4.2 Endpoints used

| Purpose | Endpoint |
|---|---|
| Server identity / version | `GET /identity` |
| List libraries | `GET /library/sections` |
| Library detail + on-disk locations | `GET /library/sections/{id}` |
| All movies in a section | `GET /library/sections/{id}/all?type=1` |
| All episodes in a show section | `GET /library/sections/{id}/all?type=4` |
| All seasons in a show section | `GET /library/sections/{id}/all?type=3` |
| Global watch history | `GET /status/sessions/history/all` |
| Accounts (for history attribution) | `GET /accounts` |

Pagination on every list endpoint uses `X-Plex-Container-Start` and
`X-Plex-Container-Size` headers (or the equivalent query params). MediaShelf pages at
**500 items** per request; the `MediaContainer.totalSize` field on the first response
gives the loop its bound.

### 4.3 The fields that matter

From each `Video` element on `/library/sections/{id}/all`:

| Plex attribute | Meaning | MediaShelf column |
|---|---|---|
| `ratingKey` | stable per-server item id | `provider_item_id` |
| `guid` | cross-server content identity | `guid` |
| `type` | `movie` / `episode` / `season` / `show` | `kind` |
| `title`, `year` | display | `title`, `year` |
| `addedAt` | epoch seconds — **follows the FILE, not the library entry** (§4.12) | `provider_added_at` |
| `updatedAt` | epoch seconds, last metadata change | `updated_at` |
| `duration` | milliseconds | `duration_ms` |
| `viewCount` | plays — **token-account scoped**, see §4.5 | (advisory only) |
| `lastViewedAt` | epoch seconds — token-account scoped | (advisory only) |
| `librarySectionID` | owning library | `library_id` |
| `grandparentRatingKey`, `grandparentTitle` | (episodes) the show | `show_id`, `show_title` |
| `parentRatingKey`, `parentIndex` | (episodes) the season and its number | `season_id`, `season_number` |

From the nested `Media[].Part[]` elements:

| Plex attribute | MediaShelf column |
|---|---|
| `Part.file` | `file_path` — absolute path on the Plex host |
| `Part.size` | `size_bytes` |
| `Part.container` | `container` |
| `Media.videoResolution`, `videoCodec`, `audioCodec`, `bitrate` | `resolution`, `video_codec`, `audio_codec`, `bitrate` |

An item can have several `Media` entries (multiple versions) each with several `Part`
entries (split files). MediaShelf stores **every part as its own row** in `media_part`
and treats an item's size as the sum of its parts. This matters: a movie with a 4K and a
1080p version is one item occupying two files, and a reclaim report that reported only
the first part would understate the win.

### 4.4 Why Plex-only, and what it costs

MediaShelf does not mount the media shares. That keeps the container trivially portable,
removes any possibility of it touching a file by accident, and means it works from
anywhere that can reach port 32400.

The accepted trade-offs:

- **Size is Plex's number, not the filesystem's.** Plex records `Part.size` at scan
  time. If a file is replaced outside Plex and Plex hasn't rescanned, the number is
  stale. In practice Plex's own scanner keeps this current; the design accepts drift.
- **Orphans are invisible.** A file on disk that Plex has never indexed will not appear
  in MediaShelf at all. Finding those needs a filesystem walk, which is explicitly out
  of scope (see §14.3 for the optional-mount future).
- **Paths are Loki-relative.** `file_path` is the path as the Plex server sees it. It is
  displayed and exported for a human's benefit, not resolved by MediaShelf.

### 4.5 Why Plex is not the watch-data source

Two problems make Plex's own watch data unfit to build a reclaim tool on.

**It is scoped to one account.** `viewCount` and `lastViewedAt` on a library item reflect
only the account that owns the token. Plays by home users, managed users, and shared users
are invisible. For a household library this understates watches badly, and a tool that
recommends deleting "unwatched" content someone else watches weekly is worse than useless.

**It only knows that a play started.** `/status/sessions/history/all` records a playback
event. It does not record whether the viewer watched the whole thing or bailed after four
minutes. Those two events mean opposite things for a deletion decision, and Plex reports
them identically.

`/status/sessions/history/all` solves the first problem — it is server-wide across all
accounts — so it remains the **fallback** source, and §4.11 covers when it is used. But
Tautulli solves both, so Tautulli is the primary.

### 4.6 Plex scan strategy

**Full scan** — walks every section, every item, every part, and the complete history.
Run on first launch and on demand. Expect a few minutes for a large library.

**Incremental scan** — the nightly default. Two cheap tricks:

1. Items: request the section sorted by `addedAt:desc` and stop paging once the page's
   oldest `addedAt` predates the last successful scan. This catches additions. To catch
   *removals* and metadata edits, a full item sweep still runs weekly (configurable) —
   an item silently vanishing from Plex is exactly the event that must not be missed.
2. History: request `/status/sessions/history/all` sorted by `viewedAt:desc` and stop at
   the last-seen `viewedAt` watermark.

**Concurrency** — one scan at a time, enforced by a row in a `scan_lock` table plus an
in-process lock. A scan that dies leaves a stale lock; locks older than
`SCAN_LOCK_TIMEOUT` (default 2 h) are broken automatically and the orphaned scan row is
marked `failed`.

**Politeness** — requests are issued serially with a small configurable delay
(`PLEX_REQUEST_DELAY_MS`, default 0) and a hard timeout. Loki is a 2010-vintage R710;
the ingest is designed to be slow and steady rather than parallel and aggressive.

### 4.7 Failure handling

| Failure | Behaviour |
|---|---|
| Plex unreachable | Scan marked `failed` with the error; UI keeps serving the last good snapshot with a staleness banner |
| Token rejected (401) | Scan fails fast, prominent UI banner, no retry loop |
| Section disappears | Its items are marked `missing` rather than deleted (§5.4) |
| Malformed / partial item | Item skipped, logged, counted in `scan.warning_count`; the scan still succeeds |
| Timeout mid-page | Up to 3 retries with exponential backoff, then the scan fails |

### 4.12 Date Added is not the date it was added

Plex's `addedAt` is the field its UI calls Date Added, and it is the obvious
source for "when did this arrive". It is also, on this library, wrong 19% of the
time — because it tracks the **file**, not the library entry. Replace or
re-encode a file and Plex resets Date Added while keeping the item, its ratingKey
and its entire watch history.

Measured on the live server (2026-09-10), comparing `addedAt` against
`lastViewedAt` on items that have both:

| Library | items with both dates | watched BEFORE "added" |
|---|---|---|
| Movies | 509 | 55 (10.8%) |
| TV Show Archive | 1,393 | 306 (22.0%) |
| **overall** | **1,902** | **361 (19.0%)** |

*2001: A Space Odyssey* reports added 2026-07-31 and last watched 2017-08-26.
Plex's API exposes no better field; the true insert time exists only in Plex's
own SQLite `metadata_items.created_at`, which needs filesystem access to Loki.

This is not cosmetic. `pre_history` is derived from `added_at`, so an old item
whose file was replaced looks post-coverage and is promoted into the
**confident** reclaim pool — the one pool that is supposed to be trustworthy. The
new-arrival grace is affected the same way in reverse.

**What MediaShelf does about it.** A view is proof the item already existed, so
the earliest view is a lower bound on the true add date. Two independent
witnesses, and both are needed:

```
added_at = MIN(provider_added_at, first_watched_at, provider_last_viewed_at)
```

`first_watched_at` is MediaShelf's first completed Tautulli play — precise, but
only inside the history window. `provider_last_viewed_at` is Plex's own
`lastViewedAt`, which reaches back as far as the server does.

The first version of this correction used Tautulli alone and **fired zero times
on the live library**, because the wrong dates are overwhelmingly older than the
2025-03 history window. Plex's `lastViewedAt` is the field that actually carries
the evidence, and it was being parsed and discarded.

`provider_added_at` keeps Plex's raw value, `added_at_source` records which
applied, and the UI explains the substitution on any corrected row rather than
silently showing a different date than Plex does. Items nobody ever watched keep
Plex's value — nothing contradicts it.

Separately, `first_seen_at` records when MediaShelf itself first saw a row. That
is authoritative for everything added from now on, whatever Plex does to its own
field, and needs no correction.

The exact fix, if it is ever worth the access: read `metadata_items.created_at`
from Plex's database on Loki and backfill it.

### 4.8 Tautulli — the watch-history source

Tautulli runs at `http://192.168.1.100:8181` (on Isis) and has been logging every
playback on Loki since the day it was installed. **It stays LAN-only and is not published
externally** — which costs nothing here, since MediaShelf runs on Nox on the same network.
The practical consequence is that design validation happens on the LAN: see
`tools/probe.py` (§13.1). Its database is independent of Plex's:
clearing Plex's history, or Plex pruning its own, does not touch it. That makes it a
deeper and more durable record than anything Plex exposes.

Its API is a single endpoint with a `cmd` parameter:

```
GET http://192.168.1.100:8181/api/v2?apikey=<key>&cmd=<command>&<params>
```

Every response is `{"response": {"result": "success"|"error", "message": ..., "data": ...}}`.
A transport-level 200 with `result: "error"` in the body is the normal failure mode, so
**the result field must be checked on every call** — treating HTTP 200 as success is the
mistake this API invites.

| Purpose | Command |
|---|---|
| Reachability / version | `cmd=get_server_info`, `cmd=arnold` (returns a quote; a cheap liveness ping) |
| Which Plex server it watches | `cmd=get_server_info` → `pms_identifier` |
| Users | `cmd=get_users` |
| Libraries | `cmd=get_libraries` |
| **Watch history** | `cmd=get_history` |
| Per-item size / play count (cross-check) | `cmd=get_library_media_info` |

### 4.9 Ingesting history

```
cmd=get_history
  &grouping=0          # do NOT collapse successive plays — we want raw events
  &order_column=date
  &order_dir=desc
  &start=<offset>
  &length=1000
  &after=<YYYY-MM-DD>  # incremental watermark
```

`grouping=0` matters. Tautulli's *Group Successive Play History* setting merges a paused-
and-resumed play into one row for display purposes. For counting distinct viewings that
is what we want — but the grouping should be MediaShelf's decision, applied consistently,
not a mirror of however that setting happens to be configured on Isis. So MediaShelf pulls
ungrouped events and does its own session collapsing: **events for the same item by the
same user whose start times fall within `SESSION_MERGE_WINDOW` (default 6 hours) count as
one viewing.**

The fields MediaShelf keeps from each history row:

| Tautulli field | Use |
|---|---|
| `row_id` | idempotency key for upsert |
| `reference_id` | Tautulli's own grouping id, kept for cross-checking |
| `date` / `started` / `stopped` | epoch seconds |
| `rating_key` | the leaf item — movie or episode |
| `parent_rating_key`, `grandparent_rating_key` | season and show, for rollup |
| `user_id`, `user`, `friendly_name` | who watched |
| `media_type` | `movie` / `episode` |
| `percent_complete` | **how much was actually watched** |
| `watched_status` | Tautulli's own verdict against its configured threshold |
| `play_duration`, `paused_counter` | real time spent, minus pauses |
| `platform`, `player` | stored, unused in v1 |

### 4.10 What "watched" means

This is the part that most changes the quality of the output, so MediaShelf defines it
itself rather than inheriting Tautulli's threshold setting:

| Derived value | Rule |
|---|---|
| **completed view** | `percent_complete >= COMPLETION_THRESHOLD` (default 85) |
| **partial view** | `percent_complete` between `ABANDON_CEILING` (default 15) and the completion threshold |
| **abandoned view** | `percent_complete < ABANDON_CEILING` — started and bailed |

MediaShelf uses `percent_complete` rather than `watched_status` as the primary signal
because `watched_status` depends on how the threshold is configured on Isis, and that
configuration can change without warning. `watched_status` is stored anyway and shown in
the UI as a cross-check; a systematic disagreement between the two is worth seeing.

The rollups per item become:

- `watch_count` — **completed** views only
- `partial_count`, `abandoned_count`
- `last_watched_at` — most recent completed view
- `last_touched_at` — most recent view of any kind, including abandoned
- `first_watched_at`, `distinct_watcher_count`
- `avg_percent_complete`

**An abandoned view is evidence for deletion, not against it.** Three people started a
film and all three quit twenty minutes in: that is a stronger signal to delete than a film
nobody ever opened, because the household has now actively rejected it. Plex's history
cannot express this at all, and it is the single biggest reason to prefer Tautulli. The
scoring model uses it (§6.1, `rejection` component).

### 4.11 Fallback, coverage, and cross-checks

**When Tautulli is unavailable**, MediaShelf falls back to `PlexHistoryProvider` and
`/status/sessions/history/all`. Events ingested that way are marked `source = 'plex'` and
carry no `percent_complete`, so they count as completed views (the only assumption
available) and are excluded from the `rejection` component. The UI shows a banner
explaining the score is running in degraded mode, because a silently degraded score is a
score that gets trusted when it shouldn't be.

**Tautulli's coverage horizon.** Tautulli only knows about plays since it was installed.
Anything watched before that is invisible to it — and those items will look never-watched.
On first ingest MediaShelf records `history_coverage_since` = the oldest event Tautulli
holds, and:

- Items with `added_at` earlier than that date get a `pre_history` flag.
- The UI shows a badge on those rows and a one-line explanation on the dashboard.
- The `staleness` score component is **capped** for `pre_history` items, so a 2009 movie
  that was watched in 2015 and never since doesn't get scored as though it had never been
  watched at all.

**Cross-check with `get_library_media_info` — movie libraries only.** Tautulli maintains
its own per-item table with `file_size`, `added_at`, `play_count`, and `last_played`.
For **movie** sections this is a useful second opinion: MediaShelf pulls it once per scan
per library and records size disagreements against Plex as scan warnings, not errors.

For **show** sections it is unusable and must not be called. Measured against the live
server (2026-09-07): `get_library_media_info` on a show section returns one row per
*show*, with `file_size: 0` on every row, and passing `section_type=episode`, `season`,
or `show` changes nothing — all four return the same 169 show-level rows with zero sizes.
A `refresh=true` rebuild might populate them, but that triggers an expensive scan on
Tautulli and was deliberately not attempted.

The consequence is architectural and worth stating plainly: **Plex is the only size
authority for TV**, which is 49 of the library's 66 TB. Tautulli contributes watch data
and nothing else for show libraries. Any future code tempted to use Tautulli's table as a
convenient size source will silently produce zeroes for three-quarters of the library.

**Joining to Plex.** Both systems use the same Plex `rating_key`, so the join is direct —
no title or path matching, no fuzzy logic. If `get_server_info.pms_identifier` does not
match the `machineIdentifier` from Plex's `/identity`, Tautulli is watching a *different*
Plex server and MediaShelf refuses to use it rather than joining nonsense data together.
That check runs at the start of every scan.

---

## 5. Data model

SQLite, WAL mode, foreign keys on. Migrations are plain numbered SQL files applied at
startup, tracked in a `schema_version` table — no ORM migration framework.

### 5.1 Tables

```sql
-- One row per configured media server. v1 has exactly one, but the column exists
-- so Emby/Jellyfin can be added without a migration.
CREATE TABLE provider (
    id              INTEGER PRIMARY KEY,
    kind            TEXT NOT NULL,          -- 'plex' | 'emby' | 'jellyfin'
    name            TEXT NOT NULL,
    base_url        TEXT NOT NULL,
    server_id       TEXT,                   -- Plex machineIdentifier
    version         TEXT,
    last_scan_id    INTEGER,
    created_at      INTEGER NOT NULL
);

CREATE TABLE library (
    id              INTEGER PRIMARY KEY,
    provider_id     INTEGER NOT NULL REFERENCES provider(id),
    provider_key    TEXT NOT NULL,          -- Plex section id
    title           TEXT NOT NULL,
    kind            TEXT NOT NULL,          -- 'movie' | 'show'
    locations       TEXT,                   -- JSON array of on-disk roots
    keep_all        INTEGER NOT NULL DEFAULT 0,  -- library-level keep rule (§6.6)
    scanned_at      INTEGER,
    UNIQUE (provider_id, provider_key)
);

-- Explicit human judgements that the reclaim score must not override (§6.6).
-- Keyed on CONTENT identity, never on rating_key, which Plex reassigns.
CREATE TABLE keep_mark (
    id                  INTEGER PRIMARY KEY,
    scope               TEXT NOT NULL,      -- 'show' | 'season' | 'movie'
    mode                TEXT NOT NULL,      -- 'keep' | 'exclude'
    library_id          INTEGER NOT NULL REFERENCES library(id),
    guid                TEXT NOT NULL,      -- movie guid, or the SHOW's guid
    season_number       INTEGER,            -- set only for scope='season'
    -- convenience only; allowed to go stale, never matched on
    provider_item_id    TEXT,
    label               TEXT NOT NULL,      -- human-readable title at marking time
    note                TEXT,               -- "might watch someday", "expensive to re-rip"
    created_at          INTEGER NOT NULL,
    updated_at          INTEGER NOT NULL,
    last_matched_scan_id INTEGER,           -- NULL after a scan => orphaned
    UNIQUE (scope, library_id, guid, season_number)
);

-- The unit of analysis. One row per movie, and one row per SEASON.
-- Shows themselves get a row too (kind='show') for grouping and display,
-- but carry no size of their own.
CREATE TABLE media_item (
    id                  INTEGER PRIMARY KEY,
    provider_id         INTEGER NOT NULL REFERENCES provider(id),
    library_id          INTEGER NOT NULL REFERENCES library(id),
    provider_item_id    TEXT NOT NULL,      -- Plex ratingKey
    kind                TEXT NOT NULL,      -- 'movie' | 'show' | 'season'
    guid                TEXT,
    title               TEXT NOT NULL,
    sort_title          TEXT,
    year                INTEGER,
    -- season/show linkage
    parent_id           INTEGER REFERENCES media_item(id),   -- season -> show
    season_number       INTEGER,
    -- provenance
    added_at            INTEGER,            -- epoch s; for a season, MIN over episodes
    updated_at          INTEGER,
    -- rollups (denormalized, recomputed each scan)
    episode_count       INTEGER NOT NULL DEFAULT 0,
    size_bytes          INTEGER NOT NULL DEFAULT 0,
    duration_ms         INTEGER NOT NULL DEFAULT 0,
    part_count          INTEGER NOT NULL DEFAULT 0,
    primary_path        TEXT,               -- representative file path for display
    resolution          TEXT,
    video_codec         TEXT,
    -- watch rollups (from watch_event)
    watch_count             INTEGER NOT NULL DEFAULT 0,  -- COMPLETED views only
    partial_count           INTEGER NOT NULL DEFAULT 0,
    abandoned_count         INTEGER NOT NULL DEFAULT 0,
    last_watched_at         INTEGER,                     -- last completed view
    last_touched_at         INTEGER,                     -- last view of any kind
    first_watched_at        INTEGER,
    distinct_watcher_count  INTEGER NOT NULL DEFAULT 0,
    avg_percent_complete    REAL,
    pre_history             INTEGER NOT NULL DEFAULT 0,  -- added before history coverage
    -- keep state, recomputed each scan and on every keep change (§6.6)
    kept                    INTEGER NOT NULL DEFAULT 0,
    kept_via                TEXT,           -- 'library' | 'show' | 'season' | 'movie'
    kept_mark_id            INTEGER REFERENCES keep_mark(id),
    provider_view_count     INTEGER NOT NULL DEFAULT 0,  -- advisory, token-scoped
    -- lifecycle
    status              TEXT NOT NULL DEFAULT 'present',  -- 'present' | 'missing'
    first_seen_scan_id  INTEGER,
    last_seen_scan_id   INTEGER,
    UNIQUE (provider_id, provider_item_id)
);

-- Episodes are ingested (they carry the size and the history), but they are NOT
-- the unit of analysis. They exist so seasons can be rolled up accurately and so
-- a season can be expanded in the UI.
CREATE TABLE episode (
    id                  INTEGER PRIMARY KEY,
    season_item_id      INTEGER NOT NULL REFERENCES media_item(id) ON DELETE CASCADE,
    provider_item_id    TEXT NOT NULL,
    episode_number      INTEGER,
    title               TEXT,
    added_at            INTEGER,
    duration_ms         INTEGER,
    size_bytes          INTEGER NOT NULL DEFAULT 0,
    watch_count         INTEGER NOT NULL DEFAULT 0,
    last_watched_at     INTEGER,
    status              TEXT NOT NULL DEFAULT 'present',
    UNIQUE (provider_item_id)
);

-- Every physical file. A movie with two versions has two rows.
CREATE TABLE media_part (
    id                  INTEGER PRIMARY KEY,
    media_item_id       INTEGER REFERENCES media_item(id) ON DELETE CASCADE,
    episode_id          INTEGER REFERENCES episode(id) ON DELETE CASCADE,
    provider_part_id    TEXT,
    file_path           TEXT NOT NULL,
    size_bytes          INTEGER NOT NULL DEFAULT 0,
    container           TEXT,
    resolution          TEXT,
    video_codec         TEXT,
    audio_codec         TEXT,
    bitrate             INTEGER,
    CHECK ((media_item_id IS NULL) != (episode_id IS NULL))
);

-- One row per playback event, server-wide, all accounts.
-- Primary source is Tautulli; 'plex' rows are degraded fallback events (§4.11).
CREATE TABLE watch_event (
    id                  INTEGER PRIMARY KEY,
    provider_id         INTEGER NOT NULL REFERENCES provider(id),
    source              TEXT NOT NULL,      -- 'tautulli' | 'plex'
    source_row_id       TEXT NOT NULL,      -- Tautulli row_id, or Plex historyKey
    reference_id        TEXT,               -- Tautulli's own grouping id
    provider_item_id    TEXT NOT NULL,      -- the leaf: movie or episode ratingKey
    account_id          TEXT,
    viewed_at           INTEGER NOT NULL,   -- 'date' / 'started'
    stopped_at          INTEGER,
    play_duration_s     INTEGER,
    paused_counter_s    INTEGER,
    percent_complete    INTEGER,            -- NULL for source='plex'
    watched_status      REAL,               -- Tautulli's verdict, advisory
    disposition         TEXT NOT NULL,      -- 'completed' | 'partial' | 'abandoned'
    session_id          TEXT,               -- MediaShelf's own merge key (§4.9)
    media_type          TEXT,
    platform            TEXT,
    UNIQUE (provider_id, source, source_row_id)
);

CREATE TABLE account (
    id              INTEGER PRIMARY KEY,
    provider_id     INTEGER NOT NULL REFERENCES provider(id),
    account_id      TEXT NOT NULL,          -- Tautulli user_id / Plex accountID
    name            TEXT,
    friendly_name   TEXT,
    UNIQUE (provider_id, account_id)
);

-- What the history record actually covers, so 'never watched' can be
-- distinguished from 'watched before Tautulli existed' (§4.11).
CREATE TABLE history_coverage (
    id                  INTEGER PRIMARY KEY,
    provider_id         INTEGER NOT NULL REFERENCES provider(id),
    source              TEXT NOT NULL,
    earliest_event_at   INTEGER,
    latest_event_at     INTEGER,
    event_count         INTEGER NOT NULL DEFAULT 0,
    updated_at          INTEGER NOT NULL,
    UNIQUE (provider_id, source)
);

CREATE TABLE scan (
    id                  INTEGER PRIMARY KEY,
    provider_id         INTEGER NOT NULL REFERENCES provider(id),
    mode                TEXT NOT NULL,      -- 'full' | 'incremental'
    trigger             TEXT NOT NULL,      -- 'manual' | 'schedule' | 'startup'
    status              TEXT NOT NULL,      -- 'running' | 'succeeded' | 'failed'
    started_at          INTEGER NOT NULL,
    finished_at         INTEGER,
    items_seen          INTEGER DEFAULT 0,
    items_added         INTEGER DEFAULT 0,
    items_updated       INTEGER DEFAULT 0,
    items_missing       INTEGER DEFAULT 0,
    events_added        INTEGER DEFAULT 0,
    warning_count       INTEGER DEFAULT 0,
    error               TEXT
);

CREATE TABLE saved_view (
    id              INTEGER PRIMARY KEY,
    name            TEXT NOT NULL UNIQUE,
    description     TEXT,
    rules           TEXT NOT NULL,          -- JSON rule group, see §7
    sort            TEXT,                   -- JSON: [{field, dir}]
    columns         TEXT,                   -- JSON array of column keys
    weights         TEXT,                   -- JSON weight profile, see §6
    created_at      INTEGER NOT NULL,
    updated_at      INTEGER NOT NULL
);

CREATE TABLE setting (
    key             TEXT PRIMARY KEY,
    value           TEXT NOT NULL
);
```

### 5.2 Indexes

```sql
CREATE INDEX ix_item_library     ON media_item(library_id, kind, status);
CREATE INDEX ix_item_added       ON media_item(added_at);
CREATE INDEX ix_item_lastwatch   ON media_item(last_watched_at);
CREATE INDEX ix_item_size        ON media_item(size_bytes);
CREATE INDEX ix_item_watchcount  ON media_item(watch_count);
CREATE INDEX ix_item_parent      ON media_item(parent_id);
CREATE INDEX ix_item_guid        ON media_item(guid);   -- duplicate grouping, §6.5
CREATE INDEX ix_item_kept        ON media_item(kept, library_id);
CREATE INDEX ix_keep_lookup      ON keep_mark(library_id, guid, season_number);
CREATE INDEX ix_episode_season   ON episode(season_item_id);
CREATE INDEX ix_part_item        ON media_part(media_item_id);
CREATE INDEX ix_part_episode     ON media_part(episode_id);
CREATE INDEX ix_event_item       ON watch_event(provider_item_id);
CREATE INDEX ix_event_viewed     ON watch_event(viewed_at);
CREATE INDEX ix_event_disp       ON watch_event(provider_item_id, disposition);
CREATE INDEX ix_event_account    ON watch_event(account_id, viewed_at);
```

A full-text index over `title` uses SQLite FTS5 (`media_item_fts`) kept in sync by
triggers, so the grid's search box is instant even at 100k rows.

### 5.3 Season rollup rules

After items and history are ingested, a rollup pass recomputes each season row:

| Season field | Rule |
|---|---|
| `size_bytes` | `SUM(episode.size_bytes)` |
| `episode_count` | `COUNT(episode)` |
| `duration_ms` | `SUM(episode.duration_ms)` |
| `added_at` | `MIN(episode.added_at)` — when the season first landed |
| `watch_count` | `SUM(episode.watch_count)` — completed episode views |
| `abandoned_count` | `SUM(episode.abandoned_count)` |
| `last_watched_at` | `MAX(episode.last_watched_at)` |
| `last_touched_at` | `MAX(episode.last_touched_at)` |
| `primary_path` | the common directory prefix of its episodes' paths |

A show row rolls up the same way from its seasons, for display and for the "delete the
whole show" case.

The `watch_count` choice deserves a note: summing episode plays means a 24-episode season
watched once has `watch_count = 24`, while a movie watched once has `watch_count = 1`.
Comparing those raw numbers across kinds is meaningless. So the scoring model
(§6) uses a **normalized completion figure** for TV — `watch_count / episode_count` —
rather than the raw sum, and the grid displays both (`24 plays · 1.0× season`).

### 5.4 Missing items

Nothing is ever hard-deleted by a scan. When a full sweep completes and an item that was
present is no longer returned by Plex, its `status` flips to `missing` and
`last_seen_scan_id` freezes. Missing items are hidden from the default grid but remain
queryable — they're the record of what left the library and when, which is exactly what
you want the first time something disappears unexpectedly.

---

## 6. The reclaim score

The score answers: *how much do I gain by deleting this, weighed against how likely I am
to want it?* It is a weighted sum of normalized components, each in `[0, 1]`, scaled to
0–100. Higher means "better candidate for deletion".

### 6.1 Components

| Component | Definition | Rationale |
|---|---|---|
| **`size`** | `log10(1 + size_bytes) / log10(1 + max_size_in_library)` | Log scale — the difference between 2 GB and 4 GB matters more than between 60 GB and 62 GB |
| **`staleness`** | `min(days_since_last_watch / STALE_HORIZON, 1)`, where a never-watched item scores `1.0` | How long since anyone cared |
| **`unpopularity`** | `1 - min(normalized_watches / POPULAR_AT, 1)` where `normalized_watches` = `watch_count` for movies, `watch_count / episode_count` for seasons | Rewatched things are keepers |
| **`age`** | `min(days_since_added / AGE_HORIZON, 1)` | Something added last week deserves a chance |
| **`solitude`** | `1 - min(distinct_watcher_count / SOLITUDE_AT, 1)` | Content almost nobody watches is safer to cut |
| **`rejection`** | `min(abandoned_count / REJECTED_AT, 1)`, zero when `watch_count > 0` | Started and quit is an active verdict — see below |

`STALE_HORIZON` defaults to 730 days, `AGE_HORIZON` to 1095 days, `POPULAR_AT` to 3,
`REJECTED_AT` to 2, `SOLITUDE_AT` to 3.

Both of the Tautulli-dependent components were calibrated against real measurements
rather than guessed, and both moved:

**`solitude` — divisor 3 confirmed, weight raised.** The original divisor assumed a
household. With 60 users the concern was that it would saturate. It does not: of 24,204
watched items, **61.7% were watched by exactly one user and 20.0% by two**, with a long
thin tail. Three distinct watchers already puts an item in the top ~8% of the library, so
a divisor of 3 is the right shape. What changed is that on a 60-user server "only one
person ever watched this" is a genuinely informative statement rather than noise, so the
weight goes up from 0.04 to 0.10.

**`rejection` — real but smaller than hoped, weight cut.** Across all 87,640 plays the
split is **76.3% finished · 18.8% partial · 4.9% abandoned**. Abandonment is a real
signal but a thin one, so its weight drops from 0.12 to 0.06. It stays in the model
because when it does fire it is unusually decisive — an item several people started and
none finished is a much stronger delete case than the score's other components can
express — but it should not be pushing the ranking around on 5% of the evidence.

The larger surprise is the **18.8% partial** band. That is four times the abandoned share
and it is genuinely ambiguous: someone 60% through an episode may have been interrupted
and intending to return, or may have quietly given up. v1 deliberately does not score it
in either direction; it is stored, displayed, and filterable, and the first month of
looking at real rows should settle whether it deserves a component of its own.

The `rejection` component only exists because Tautulli supplies `percent_complete`
(§4.10). It is deliberately zeroed the moment anyone completes a view, so a film that was
abandoned twice and then watched through is not penalised. When history runs in
Plex-fallback mode this component is unavailable and drops out of the weighting
entirely (§4.11).

### 6.2 The formula

```
score = 100 × Σ(wᵢ · componentᵢ) / Σ(wᵢ)
```

Default weight profile:

```json
{ "size": 0.28, "staleness": 0.24, "unpopularity": 0.22,
  "solitude": 0.10, "age": 0.10, "rejection": 0.06 }
```

The denominator sums only the weights of components that are **available** for that row.
When history is in Plex-fallback mode, `rejection` is dropped and the remaining weights
renormalize, rather than every item silently scoring lower because one component is
pinned at zero. The same applies per-row: a `pre_history` item has its `staleness`
capped (§4.11), and the UI shows which components contributed.

### 6.3 The grace rule

Two hard overrides, applied after scoring, because a purely numeric model will
eventually recommend something obviously wrong:

- **New-arrival grace** — an item with `added_at` inside `GRACE_DAYS` (default 30) is
  clamped to score 0 and flagged `grace: new`. You haven't had a chance to watch it yet.
- **Recent-watch grace** — an item watched inside `RECENT_DAYS` (default 90) is clamped
  to a maximum of 25 and flagged `grace: recent`.

Both are toggleable per view, and the UI shows why an item was clamped rather than
silently hiding it.

### 6.4 Where it's computed

In SQL, as a generated expression in the query, not as a stored column. Weights change
every time a slider moves; storing the score would mean recomputing 100k rows per drag.
The normalization constants (`max_size_in_library`, horizons) are computed once per
request in a CTE. At this row count the whole thing runs in single-digit milliseconds.

The same function is implemented in Python in `mediashelf.scoring` for unit tests and CSV
export, with a property test asserting the SQL and Python implementations agree to within
0.01 on a generated corpus. Two implementations of the same formula is a real risk, and
that test is the mitigation.

### 6.5 Cross-library duplicates

With 16 movie sections, the same film is often held more than once — most obviously
*Movies* vs *4K Movies*, but genre sections overlap too. Two copies of a film are 100%
redundant storage, and neither copy looks like a deletion candidate on its own: each has
its own watch history, and plays are split between them, so both score as moderately
watched.

MediaShelf detects this using the Plex `guid`, which identifies the *content* rather than
the file, and is the same across sections. Items sharing a `guid` are grouped, and the
group is surfaced with:

- total redundant bytes (group size minus its largest member)
- which copy is played and which is not — the common case is that everyone watches the
  4K copy and the 1080p copy has not been touched in a year
- resolution and bitrate of each copy, so "keep the best one" is a visible decision

This is a **report**, not an automatic recommendation. Deciding that a 4K copy supersedes
a 1080p one depends on what plays back well on the household's devices, which MediaShelf
has no way to know. It surfaces the pairs and their sizes and lets a human choose.

A `duplicates` flag on the item and a *Duplicate groups* view are the v1 surface. This was
originally deferred to §14.3; the measured library shape moved it forward.

### 6.6 Keep marks

**A reclaim score measures what is unwatched, not what is valuable.** Those two things
correlate for ordinary purchased or downloaded content and are close to unrelated
everywhere else. *Family Videos* is 38 GB of irreplaceable home video that is 86% never
played and scores as a perfect deletion target. *4K Movies* is 72.9% never played and is
the most expensive content in the library to re-acquire. And of the 1,337 never-played
seasons, an unknown number are simply things kept deliberately, on purpose, in case
someone wants them later.

No amount of watch data can infer any of that. It has to be stated by a human, and once
stated the score must not be able to override it. That is what a **keep mark** is.

#### Scopes

One kind of mark, applied at four levels:

| Scope | Granularity |
|---|---|
| `library` | a whole section — *keep everything in 4K Movies* |
| `show` | a series and all its seasons, present and future |
| `season` | one season of one series |
| `movie` | one film |

Libraries are kept by a **rule** (`library.keep_all`), so anything added to that library
later is kept automatically. Everything below the library level is marked **explicitly**,
per item — deliberate acts on specific things, each with a timestamp and an optional note.

#### Precedence

Most specific wins, and an explicit mark can point in either direction:

```
movie/season explicit 'exclude'  →  NOT kept   (overrides an inherited library keep)
movie/season explicit 'keep'     →  kept
show 'keep'                      →  kept
library keep_all                 →  kept
otherwise                        →  not kept
```

The `exclude` direction exists so that *"keep all of 4K Movies except this one"* is
expressible. Without it a library rule is a trap: the only way out would be to turn the
whole rule off and re-mark 313 films by hand.

Each item carries a computed `kept` flag and a `kept_via` field naming which mark applied,
recomputed at the end of every scan and immediately on any keep change. `kept_via` matters
in the UI — "kept because the whole library is kept" and "kept because you marked this
season in March" are different facts and lead to different actions.

#### Identity, and why it is not the rating key

Plex `ratingKey` values are **not stable**. A library rebuild, an unmatch-and-rematch, or
a section removed and re-added can reassign them. A keep list stored against rating keys
would silently detach from the content it was protecting, and in v2 the consequence of
that is deleting something explicitly marked keep. That is the worst failure this
application could have.

So keep marks are stored against **content identity** instead:

| Scope | Key |
|---|---|
| `library` | the library row |
| `movie` | `(library_id, guid)` |
| `show` | `(library_id, guid)` |
| `season` | `(library_id, show_guid, season_number)` |

Plex's `guid` identifies the *content*, not the file, and survives rebuilds. Seasons are
keyed by their show's guid plus the season number rather than a season guid, so the scheme
does not depend on Plex exposing season-level guids at all.

**Marks are scoped per library on purpose.** Because *Movies* and *4K Movies* hold the
same films under the same `guid` (§6.5), an unscoped mark would keep both copies at once —
so keeping the 4K version would silently protect the 1080p one you were trying to remove.
Scoping by `(library_id, guid)` keeps the two copies independently markable, which is
exactly what the duplicate-groups view needs.

`provider_item_id` is stored alongside as a convenience for linking, and is allowed to go
stale. It is never the thing a mark is matched on.

#### Orphans

If a scan completes and a keep mark matched nothing, it is flagged **orphaned** rather
than deleted. The content may have been removed from Plex, renamed, rematched to a
different guid, or moved to another library. The Kept view lists orphans separately with
their label and note as recorded at marking time, so a keep that has come adrift is
visible rather than silent. Nothing removes a keep mark except an explicit human action.

#### Effect

- Kept items are **hidden from the grid by default**, with a *Show kept* toggle.
- They are excluded from every candidate view, from CSV exports of those views, and from
  the reclaim totals — which is why those totals are always reported three ways (§9.1).
- Their score is still computed and shown when kept items are displayed, so *"this scores
  91 and I'm keeping it anyway"* stays a visible, revisitable decision.
- In v2, a kept item **cannot be quarantined**. The deletion path refuses it at the
  guardrail, before any confirmation dialog. Removing the keep mark is a separate,
  deliberate act — deletion never offers to do it for you as part of the same click.

**Nothing ships kept.** No library is pre-marked, in seed data or in configuration — every
keep in the system got there because a person put it there. The `KEEP_ALL_LIBRARIES`
environment variable exists so a library *can* be marked from configuration when that is
convenient, but its default is empty and it stays empty unless someone sets it.

That is a deliberate choice rather than an omission. A keep mark's value is that it
records a human judgement; one the application invented on the user's behalf would be
indistinguishable in the database from one they actually made, and would undermine the
audit trail that makes the Kept view worth reading.

#### The number that keeps this honest

A keep list that grows quietly will eventually swallow the library and leave the reclaim
report meaningless without anyone noticing. So the dashboard always shows the split:

```
27.4 TB never played  ·  8.1 TB kept  ·  19.3 TB available
```

If the middle number climbs toward the first, the tool is telling you it has stopped
being useful — which is worth knowing, and is invisible if kept items are simply
subtracted out.

---

## 7. Saved views and rules

A saved view bundles four things: a rule set, a sort, a column selection, and a weight
profile. Naming one — *"Big unwatched movies"* — makes it a one-click, re-runnable
report that re-evaluates against fresh data every time it's opened.

### 7.1 Rule grammar

Rules are a nested JSON group, evaluated into parameterized SQL:

```json
{
  "op": "and",
  "rules": [
    { "field": "kind",            "op": "in",     "value": ["movie"] },
    { "field": "size_bytes",      "op": "gte",    "value": 10737418240 },
    { "field": "watch_count",     "op": "eq",     "value": 0 },
    { "field": "added_at",        "op": "older_than_days", "value": 730 },
    { "op": "or", "rules": [
        { "field": "library_id",  "op": "in",     "value": [1, 4] },
        { "field": "resolution",  "op": "eq",     "value": "4k" }
    ]}
  ]
}
```

Fields are drawn from a whitelist mapping field name → column and type. Operators are a
fixed set: `eq`, `ne`, `lt`, `lte`, `gt`, `gte`, `in`, `not_in`, `contains`,
`starts_with`, `is_null`, `is_not_null`, `older_than_days`, `newer_than_days`,
`never` (for watch fields). The whitelisted fields include the completion metrics —
`watch_count`, `partial_count`, `abandoned_count`, `avg_percent_complete`,
`last_touched_at`, `pre_history` — plus `kept` and `kept_via`. Anything outside the
whitelist is a 400. Values are always
bound parameters. There is no SQL string interpolation anywhere in this path.

### 7.2 Seed views shipped by default

| Name | Rule |
|---|---|
| Never watched, large | `watch_count = 0 AND size_bytes >= 8 GB` |
| Cold storage | `last_watched_at older than 3 years` |
| One-and-done movies | `kind = movie AND watch_count = 1 AND last_watched_at older than 2 years` |
| Abandoned seasons | `kind = season AND watch_count/episode_count < 0.3 AND added_at older than 1 year` |
| Tried and rejected | `abandoned_count >= 2 AND watch_count = 0` — the household said no |
| Bailed in the first 15 min | `avg_percent_complete < 15 AND watch_count = 0 AND size_bytes >= 4 GB` |
| Biggest 100 | sorted by `size_bytes desc`, no filter |
| Recently added | `added_at newer than 30 days` — the sanity check, not a delete list |
| **Confident reclaim** | `watch_count = 0 AND pre_history = false` — added while Tautulli was watching and never played. On the measured library this is 6.5 TB and is the list to act on first |
| **Uncertain reclaim** | `watch_count = 0 AND pre_history = true` — 20.9 TB that *might* have been watched before 2025-03-08. Needs human judgement, not a bulk action |
| Duplicate groups | items sharing a `guid` across libraries (§6.5) |
| Kept | `kept = true` — what you have decided to keep, and what it costs (§6.6) |

Every view except *Kept* excludes kept items implicitly; `kept` is a whitelisted rule
field, so a view can opt back in explicitly when that is the point.

---

## 8. HTTP API

All JSON, all under `/api/v1`. No auth in v1 (§12).

### 8.1 Library data

```
GET /api/v1/items
```

Query parameters:

| Param | Meaning |
|---|---|
| `library_id` | repeatable filter |
| `kind` | `movie` / `season` / `show` |
| `q` | FTS search over title |
| `rules` | URL-encoded JSON rule group (§7.1) |
| `view_id` | apply a saved view instead of ad-hoc rules |
| `sort` | e.g. `reclaim_score:desc`, `size_bytes:desc` |
| `weights` | URL-encoded JSON weight profile |
| `include_missing` | default `false` |
| `include_kept` | default `false` — kept items are hidden unless asked for (§6.6) |
| `page`, `page_size` | page_size max 500 |

Response:

```json
{
  "total": 4821,
  "page": 1,
  "page_size": 100,
  "aggregate": {
    "total_size_bytes": 41234567890123,
    "item_count": 4821,
    "kept_bytes": 8906044184985,
    "available_bytes": 21228456789012
  },
  "items": [
    {
      "id": 1183,
      "kind": "movie",
      "title": "Example Film",
      "year": 2011,
      "library": { "id": 1, "title": "Movies" },
      "size_bytes": 32212254720,
      "added_at": 1490000000,
      "last_watched_at": null,
      "last_touched_at": 1712000000,
      "watch_count": 0,
      "partial_count": 0,
      "abandoned_count": 3,
      "avg_percent_complete": 8.4,
      "distinct_watcher_count": 3,
      "pre_history": false,
      "episode_count": null,
      "primary_path": "/mnt/vault2/movies/Example Film (2011)/Example Film (2011).mkv",
      "part_count": 1,
      "resolution": "4k",
      "kept": false,
      "kept_via": null,
      "reclaim_score": 91.4,
      "reclaim_components": {
        "size": 0.97, "staleness": 1.0, "unpopularity": 1.0,
        "rejection": 1.0, "age": 1.0, "solitude": 0.0
      },
      "grace": null,
      "flags": ["rejected"]
    }
  ]
}
```

Other endpoints:

```
GET  /api/v1/items/{id}            full detail incl. every part and, for seasons, episodes
GET  /api/v1/libraries             list with item counts and total size
GET  /api/v1/accounts              Plex accounts seen in history
GET  /api/v1/stats/overview        headline numbers for the dashboard
GET  /api/v1/stats/size-by-library
GET  /api/v1/stats/added-over-time?bucket=month
GET  /api/v1/stats/watch-distribution
GET  /api/v1/stats/size-vs-lastwatched   scatter data, downsampled server-side
GET  /api/v1/stats/completion            finished / abandoned / never-opened, by size
GET  /api/v1/sources                     per-source status: reachable?, coverage window,
                                         event count, active history source
```

### 8.2 Keep marks

```
GET    /api/v1/keeps                    all marks, with resolved item counts and bytes
GET    /api/v1/keeps/orphans            marks that matched nothing on the last scan
POST   /api/v1/keeps                    { scope, library_id, item_id | guid,
                                          season_number?, mode, note? }
PATCH  /api/v1/keeps/{id}               edit the note
DELETE /api/v1/keeps/{id}               un-keep (the only way a mark is ever removed)
POST   /api/v1/keeps/bulk               { item_ids: [...], mode, note? }
PUT    /api/v1/libraries/{id}/keep_all  { keep_all: true|false }
```

`POST /keeps` accepts either an `item_id` (the UI's normal path — the server resolves it
to the durable `(library_id, guid, season_number)` key) or those key parts directly.
`POST /keeps/bulk` is what makes marking tractable: with 1,337 never-played seasons,
keeping a batch has to be one action on a multi-selection, not 1,337 clicks.

Every keep response includes `resolved_items` and `resolved_bytes`, so the cost of a mark
is visible at the moment it is made rather than discovered later in a total.

### 8.3 Saved views

```
GET    /api/v1/views
POST   /api/v1/views
GET    /api/v1/views/{id}
PUT    /api/v1/views/{id}
DELETE /api/v1/views/{id}
```

### 8.4 Scans

```
GET  /api/v1/scans                 history, newest first
GET  /api/v1/scans/current         null or the running scan with progress
POST /api/v1/scans                 { "mode": "full" | "incremental" } → 202 or 409 if one is running
```

Scan progress is polled at `/api/v1/scans/current` every 2 s while one is running. No
websockets — polling one row is cheaper than the complexity.

### 8.5 Export

```
GET /api/v1/export.csv?<same params as /items>
```

Streams the full result set — not just the current page — as CSV with the currently
selected columns plus the reclaim score and its components. This is the v1 deliverable:
the artifact you take away and act on.

### 8.6 Health

```
GET /healthz    → { "status": "ok", "db": "ok",
                    "plex": "ok"|"unreachable",
                    "tautulli": "ok"|"unreachable"|"disabled",
                    "history_source": "tautulli"|"plex",
                    "last_scan_at": 1757000000, "stale": false }
```

---

## 9. Web UI

Server-rendered Flask shell, vanilla JS plus two small libraries: a virtualized table
(TanStack Table core or Tabulator) and a charting library. No build step, no npm in the
container — assets are vendored. This keeps the container small and the deploy identical
to the Mythica pattern.

### 9.1 Screens

**Dashboard** — the landing page. Headline tiles (total items, total size, size of
never-watched content, size of content untouched for 2+ years, last scan time), plus the
three-way reclaim split that keeps the keep list honest (§6.6):

```
NEVER PLAYED  27.4 TB     ·     KEPT  8.1 TB     ·     AVAILABLE  19.3 TB
```

and the charts:

- *Size by library* — horizontal bar.
- *Added over time* — monthly bars, stacked by library. Shows acquisition habits.
- *Watched vs unwatched by size* — the "how much of this shelf has anyone ever touched"
  chart, which is usually the one that motivates the whole exercise. Split three ways
  now that completion data exists: **finished · started and abandoned · never opened**.
  The middle band is the interesting one and is invisible without Tautulli.
- *Size vs. last watched* — scatter, log-size on Y, days-since-watch on X. Points in the
  upper right are the reclaim targets, and clicking a region filters the grid to it.

**Library grid** — the main workspace. A virtualized table, every column sortable, with:

- A filter rail: library checkboxes, kind, size range, added range, last-watched range,
  watch-count range, resolution, and a free-text title search.
- A weight panel with five sliders and a live-updating `reclaim_score` column.
- Multi-select via checkboxes, shift-click ranges, and select-all-matching-filter.
  In v1 selection drives the aggregate footer ("**312 items selected · 4.1 TB**"), the
  CSV export, and **bulk keep**. The buttons that will eventually delete are present but
  disabled, with a tooltip explaining v1 is report-only — so the workflow is proven
  before it's armed.
- A **keep toggle** on every row, and *Keep selected* on a multi-selection with an
  optional shared note. Kept rows are hidden by default; a *Show kept* switch in the
  filter rail brings them back, badged with why they are kept (`library` / `show` /
  `season` / `movie`) and still showing their score.
- Row expansion: a season expands to its episodes; any row expands to its file parts
  with full paths.

**Kept items** — the audit surface for §6.6. Every mark with its scope, label, note, when
it was made, and how many items and bytes it currently resolves to, sorted by size so the
expensive keeps are obvious. Library-level rules are listed separately from explicit
marks, and **orphaned marks get their own section** — a keep that no longer matches
anything is a signal that content was removed, renamed, or rematched, and it should be
seen rather than silently carried forever. Un-keeping is done from here or from the grid;
nothing else ever removes a mark.

**Saved views** — list, create, edit, duplicate, run. Creating one captures the current
filter, sort, columns, and weights, so the flow is *explore, then name what you found*.

**Item detail** — everything known: all parts with paths and sizes, a per-account watch
timeline showing each play with how far it got (a row of 8%, 12%, 6% tells the story at a
glance), and the score breakdown showing each component's contribution.

**Scans** — history table, per-scan counts and duration, a "Scan now" button, live
progress.

**Settings** — connection status, scan schedule, score defaults, horizons, grace periods.

### 9.2 Interaction principles

- Every number that represents bytes is displayed in both human units and, on hover, the
  exact byte count. Reclaim decisions turn on real numbers.
- The reclaim score column always shows its components on hover. A score you can't
  interrogate is a score you won't trust, and won't act on.
- Filters live in the URL query string, so any view is linkable and back/forward works.
- Nothing in v1 is destructive, and the UI says so plainly in the header rather than
  leaving the user to wonder.
- Keeping something is one click and un-keeping is one click, but neither ever happens as
  a side effect of another action. A keep mark is a stated human judgement, so only a
  human statement creates or removes it.

---

## 10. Configuration

All via environment variables, so the Portainer stack is the single source of truth.

| Variable | Default | Notes |
|---|---|---|
| `MEDIASHELF_SECRET_KEY` | — | required, Flask session signing |
| `PLEX_BASE_URL` | — | required, e.g. `http://192.168.1.10:32400` |
| `PLEX_TOKEN` | — | required, secret |
| `PLEX_VERIFY_SSL` | `true` | |
| `PLEX_TIMEOUT_S` | `30` | |
| `PLEX_PAGE_SIZE` | `500` | |
| `PLEX_REQUEST_DELAY_MS` | `0` | throttle for Loki's sake |
| `TAUTULLI_BASE_URL` | — | e.g. `http://192.168.1.100:8181` — unset disables Tautulli |
| `TAUTULLI_API_KEY` | — | secret |
| `TAUTULLI_TIMEOUT_S` | `30` | |
| `TAUTULLI_PAGE_SIZE` | `1000` | `length` on `get_history` |
| `HISTORY_SOURCE` | `auto` | `auto` \| `tautulli` \| `plex` — `auto` prefers Tautulli, falls back |
| `SESSION_MERGE_WINDOW_H` | `6` | events merged into one viewing (§4.9) |
| `COMPLETION_THRESHOLD` | `85` | percent complete counting as a watch |
| `ABANDON_CEILING` | `15` | percent complete below which a play is "abandoned" |
| `DATABASE_PATH` | `/data/mediashelf.db` | on the mounted volume |
| `SCAN_SCHEDULE_CRON` | `0 4 * * *` | nightly incremental at 04:00 |
| `SCAN_FULL_SWEEP_CRON` | `0 3 * * 0` | weekly full sweep, Sunday 03:00 |
| `SCAN_ON_STARTUP` | `false` | |
| `SCAN_LOCK_TIMEOUT_S` | `7200` | |
| `SCORE_STALE_HORIZON_DAYS` | `730` | |
| `SCORE_AGE_HORIZON_DAYS` | `1095` | |
| `SCORE_POPULAR_AT` | `3` | |
| `SCORE_REJECTED_AT` | `2` | abandoned views for a full `rejection` score |
| `SCORE_SOLITUDE_AT` | `3` | distinct watchers before `solitude` reaches zero |
| `KEEP_ALL_LIBRARIES` | *(empty)* | optional comma-separated library names to mark `keep_all` on first run. **Empty by default** — no library is ever kept unless someone says so (§6.6) |
| `SCORE_GRACE_DAYS` | `30` | |
| `SCORE_RECENT_DAYS` | `90` | |
| `TZ` | `America/Regina` | so cron times mean what they look like |
| `LOG_LEVEL` | `INFO` | |

`PLEX_TOKEN` and `TAUTULLI_API_KEY` are secrets and must not be committed. The repo ships
`.env.example` with placeholders, and `.env` is gitignored. (Worth noting given the
SOAP-password-in-history problem on the Mythica repo — this one starts clean and stays
clean.) Neither credential is ever written to the database, the logs, or an API response;
`/api/v1/settings` redacts both.

---

## 11. Deployment

### 11.1 Container

Single image, `python:3.12-slim` base, multi-stage so build deps don't ship. Runs as a
non-root user. One volume: `/data`. One port: 8080.

### 11.2 Getting the image onto Nox

**As deployed (2026-09-10): a Portainer git stack.** No registry exists on this network
and there is no SSH account on Nox, so Portainer clones the repo onto Nox itself and
`docker compose up` builds the image there. The compose file carries both `build: .` and a
pinned `image:` tag, so the built image is still versioned rather than an anonymous
`latest`, and a redeploy replaces it predictably.

Redeploying is a "Pull and redeploy" in Portainer, or `POST /api/stacks/{id}/git/redeploy`
against the API. Nothing has to be pushed anywhere first.

The two alternatives, if the build ever needs to move off Nox:

1. **Registry** — build, tag `registry.hallsworth.ca/mediashelf:<version>` (or Forgejo's
   own container registry, which this Forgejo version supports), push, and have the stack
   pull it. Worth setting up if builds get slow or Nox gets busy.
2. **Image upload** — `docker save | gzip` and load via Portainer's *Images → Import*.
   Fine once or twice; makes rollback manual and versions easy to lose track of.

Either way the image is **tagged with a real version**, never deployed as bare `latest`.
The Mythica stack had to be torn down and recreated because a `PUT` update kept serving
old code; immutable version tags are how that failure mode is avoided rather than
worked around.

### 11.3 Portainer stack on Nox

```yaml
services:
  mediashelf:
    image: ${MEDIASHELF_IMAGE:-mediashelf:0.1.0}   # never bare :latest, see 11.2
    build: .                                       # built on Nox by the git stack
    container_name: mediashelf
    restart: unless-stopped
    ports:
      - "${MEDIASHELF_PORT:-8086}:8080"
    volumes:
      - mediashelf_data:/data
    environment:
      - MEDIASHELF_SECRET_KEY=${MEDIASHELF_SECRET_KEY}
      - PLEX_BASE_URL=${PLEX_BASE_URL}
      - PLEX_TOKEN=${PLEX_TOKEN}
      - TAUTULLI_BASE_URL=${TAUTULLI_BASE_URL}
      - TAUTULLI_API_KEY=${TAUTULLI_API_KEY}
      - HISTORY_SOURCE=auto
      - DATABASE_PATH=/data/mediashelf.db
      - SCAN_SCHEDULE_CRON=${SCAN_SCHEDULE_CRON}
      - TZ=America/Regina
      - LOG_LEVEL=INFO
    healthcheck:
      test: ["CMD", "python", "-c", "import urllib.request;urllib.request.urlopen('http://localhost:8080/healthz')"]
      interval: 60s
      timeout: 10s
      retries: 3

volumes:
  mediashelf_data:
```

**Every variable used in the stack must be listed under `environment:` here, not only set
in Portainer's UI.** A variable set in Portainer that isn't declared in the compose file
silently does nothing — the same trap that bit `PLAYERMAP_SHOW_GMS` on the Mythica stack.

Host port **8086** (8085 and most of the 808x range were already taken on Nox). Not yet
published through Nginx Proxy Manager — reachable at `http://192.168.1.77:8086` on the
LAN. If an NPM host is added later its upstream must point at the **host** port, 8086,
not the container's 8080.

### 11.4 The gunicorn / scheduler constraint

APScheduler runs in-process. With `--workers 2`, both workers would start a scheduler and
two scans would race for the SQLite write lock. Mitigation: the scheduler starts only in
the worker that successfully acquires an exclusive advisory lock on a lockfile in `/data`
at boot. The `scan_lock` table is the second line of defence. Both are needed; either
alone has a race window.

### 11.5 Backup

The database is one file. A weekly `sqlite3 .backup` to the volume, keeping four, is
enough for the bulk of it — items, parts and watch events are all reconstructible from
Plex and Tautulli, so backing those up only saves a re-ingest.

Two things in the database are **not** reconstructible from anything: the saved-view
definitions and, far more importantly, the **keep marks**. Those encode human judgements
that exist nowhere else, and losing them means either re-marking hundreds of items by
hand or — worse, in v2 — deleting something that was explicitly protected.

So keeps and views get their own treatment beyond the database backup: a nightly export
to `/data/keeps.json` and `/data/views.json`, human-readable, containing the durable keys
(`library`, `guid`, `season_number`) rather than rating keys, plus labels and notes.
These are small, diffable, and worth committing to the repo. Restoring them into an empty
database is an import of one file, and because they key on content identity rather than
rating keys, they restore correctly even into a rebuilt Plex library.

---

## 12. Security

MediaShelf is LAN-only with no authentication, by decision. That is a reasonable posture
for a read-only tool behind a firewall, and it is a bad posture the moment v2 adds
deletion. This section records what that means now and what changes later.

**v1 posture:**

- No auth, no sessions, no user records.
- Bind to the LAN; if published through NPM, restrict by source IP at the proxy or keep
  the hostname internal-only. It should not be reachable from the internet.
- `PLEX_TOKEN` lives in the environment only, never in the database, never in logs, never
  in an API response. The `/api/v1/settings` endpoint redacts it.
- No user input reaches SQL except through the whitelisted rule grammar (§7.1) with bound
  parameters.
- All rendered strings are escaped; file paths from Plex are treated as untrusted display
  data.
- CSRF is not a concern in v1 because nothing mutating exists beyond scan triggering and
  saved views. Both use `POST` with a `SameSite=Strict` token anyway, because retrofitting
  CSRF protection after deletion exists is exactly the kind of thing that gets skipped.

**Required before v2 ships deletion:** authentication is not optional. A single admin
password with a signed session, rate-limited login, and an audit log is the minimum. An
unauthenticated endpoint that deletes files on a 90 TB array is an accident waiting for
a misconfigured proxy rule.

---

## 13. Testing

| Layer | Approach |
|---|---|
| Provider | Recorded JSON fixtures (`responses`/`vcr.py`) for both sources: Plex covering a movie section, a show section with multi-season shows, multi-version movies, split parts; Tautulli covering `get_history` pages, `get_users`, `get_library_media_info`, and an error-shaped 200 response. No live server needed in CI. |
| History semantics | Disposition classification at threshold boundaries; session merging across the window; a `pre_history` item scoring differently from a truly-never-watched one; Plex-fallback rows renormalizing the weights correctly. |
| Keep marks | Every precedence combination in §6.6, especially `exclude` overriding a library rule. **Rating-key churn: a mark must still resolve after every `ratingKey` in the library is reassigned** — the single most important test in the suite, since its failure mode in v2 is deleting protected content. Same-guid items in two libraries mark independently. Orphan detection fires when content disappears and does not fire on a rescan that merely renumbers. Export/import round-trips into an empty database. |
| Normalization | Table-driven tests: odd cases like an item with no `Part`, a season with a missing episode, unicode titles, zero-byte parts. |
| Ingest | In-memory SQLite; assert idempotency (running the same scan twice changes nothing), assert `missing` transitions, assert history watermarking doesn't skip or duplicate events. |
| Scoring | Unit tests on known inputs; **property test asserting the SQL and Python implementations agree** (§6.4); tests that grace rules clamp correctly. |
| Rules | Every operator round-trips to correct SQL; injection attempts in field names and values are rejected with 400. |
| API | Flask test client, schema assertions on every response shape. |
| Smoke | A `make smoke` target that runs a real scan against a live Plex and prints the top 20 reclaim candidates — the human sanity check that no unit test replaces. |

The single most important test is idempotency: a scanner that double-counts sizes or
duplicates history events produces a report that looks plausible and is wrong, which is
worse than one that crashes.

### 13.1 Validating the design before writing the app

Both Plex and Tautulli are LAN-only, so nothing outside the network can check this design
against real data. `tools/probe.py` closes that gap: a dependency-free, **read-only**
script run from anywhere on the LAN that issues GET requests only and answers the open
questions in §15 directly —

- Tautulli's coverage horizon, and therefore how much of the library gets `pre_history`
- the finished / partial / abandoned split across a sample of real plays, which is the
  assumption the `rejection` component rests on
- whether successive-play grouping is actually happening, so the session-merge logic
  (§4.9) can be checked against reality
- real library shapes: movie counts, season counts, total sizes, multi-version items
- path roots and their sizes, which decides whether grouping by physical vault is worth
  building

It also does the `pms_identifier` cross-check from §4.11 and reports whether it passes.
Its output is safe to share — credentials are redacted from every message, including
error messages, which is verified by test. `--dump` writes a full item inventory to JSON
for offline analysis, and prints the twenty largest never-watched items as a first taste
of what the real report will look like.

`tools/reclaim_preview.py` goes further: it joins Plex sizes to Tautulli plays exactly as
MediaShelf will, and reports never-played bytes per library split into the confident and
uncertain pools, TV broken out at season granularity, and the watcher-count distribution.
This is what produced the calibration in §2.1 and §6.1.

`tools/mockserver.py` mocks both APIs so the probe can be exercised without touching a
live server; it is a test fixture, not part of the application.

Both scripts were run against the live servers on 2026-09-07 from a LAN host, and their
findings are folded into this document. Re-running them after any significant library
change is the cheapest way to check whether the score's constants still fit.

---

## 14. Roadmap

### 14.1 v2 — Deletion

The reason v1 is report-only is that the scoring model needs to earn trust before it
gets to remove anything. When it does, the design is **two-stage quarantine, not
delete**:

1. **Select** items in the UI, review the manifest (every file path, every byte, total).
2. **Quarantine** — MediaShelf records an intent and moves the files to a trash directory
   on the same filesystem (a rename, so it's instant and atomic regardless of size),
   then triggers a Plex library refresh so the entries disappear from Plex.
3. **Purge** — after a configurable hold (default 14 days) the quarantined files are
   permanently removed, either automatically or by an explicit click. Until then,
   restore is a rename back.

This requires the one thing v1 deliberately avoids: filesystem access to the media
shares. The v2 container mounts them read-write and gains a path-mapping layer
translating Plex's paths (`/mnt/titan4/...` as Loki sees them) into container paths.
Guardrails, in the order they fire:

1. **Keep marks are refused first** (§6.6). A kept item cannot be quarantined at all —
   not with a confirmation, not with a force flag. Removing the mark is a separate,
   deliberate act taken beforehand, and the deletion flow never offers to do it inline.
   This is the guardrail that matters most, because it is the one protecting decisions a
   human actually made.
2. A configured **allowlist of path prefixes** MediaShelf may touch, and a refusal to act
   on any path outside them.
3. A **dry run** on every destructive operation, listing exactly what would happen, with
   explicit confirmation required.
4. An **immutable audit log** of every quarantine and purge — what, when, how big, what
   the score was that recommended it, and whether a keep mark was removed in the days
   before. That last field is worth having: "un-kept on Tuesday, deleted on Wednesday" is
   the pattern you want to be able to find after the fact.

A scan that runs between marking and deleting must not be able to change a keep
resolution underneath an in-flight operation, so quarantine re-resolves the keep state at
the moment of execution rather than trusting what the UI displayed.

Deletion also makes authentication mandatory (§12) and makes a dry-run mode
non-negotiable: every destructive operation gets a preview that lists exactly what would
happen and requires explicit confirmation.

### 14.2 v3 — Emby and Jellyfin

The `MediaProvider` protocol is the whole preparation:

```python
class MediaProvider(Protocol):
    def server_info(self) -> ServerInfo: ...
    def libraries(self) -> list[Library]: ...
    def items(self, library: Library) -> Iterator[Item]: ...     # yields normalized items
    def refresh_library(self, library: Library) -> None: ...     # v2 only

class HistoryProvider(Protocol):
    def server_info(self) -> ServerInfo: ...      # for the pms_identifier match, §4.11
    def accounts(self) -> list[Account]: ...
    def watch_events(self, since: int | None) -> Iterator[WatchEvent]: ...
    def coverage(self) -> Coverage: ...           # earliest/latest event, count
    @property
    def has_completion_data(self) -> bool: ...    # False for PlexHistoryProvider
```

Jellyfin and Emby share an API lineage, so one `JellyfinProvider` will likely cover both
with a capability flag or two. The mapping is close: `Items` with
`Fields=Path,MediaSources,DateCreated`, `UserData.PlayCount` and `LastPlayedDate` for
watch data, `/Users` for accounts. Two differences matter. Their watch data is genuinely
per-user and must be fetched per user and merged, rather than read from one global history
endpoint. And there is no Tautulli equivalent, so `has_completion_data` is `False` and the
`rejection` component drops out — the same degraded path the Plex fallback already
exercises, which is a good reason for that path to exist and be tested from day one
rather than bolted on later.

The `provider` table already carries `kind` and the schema already keys everything on
`(provider_id, provider_item_id)`, so multiple servers can coexist in one database — the
door is open to a combined view across Plex and Jellyfin without a migration.

### 14.3 Later candidates

- **Optional filesystem stat** — if the shares happen to be mounted, verify Plex's sizes
  against reality and surface orphans Plex never indexed.
- **Duplicate and version analysis** — the same title held at 4K and 1080p is a common
  and invisible source of waste.
- **Trend tracking** — MediaShelf already snapshots on a schedule; a chart of library
  size over time and a projection of when the vault fills up is nearly free.
- **Direct Tautulli database read** — Tautulli's SQLite file could be read directly
  instead of paging its API. Measured: 87,640 rows page in about 40 seconds at 10,000 per
  request, which is fine for a nightly job, so this stays unnecessary. It would also
  couple MediaShelf to Tautulli's internal schema.
- **Per-user reclaim views** — the data to answer "what does only one person watch"
  is already stored; it just has no UI in v1.
- **Notifications** — a monthly "here's what's gone cold" summary.

---

## 15. Open questions

### Resolved

- ~~Is Tautulli running anywhere?~~ **Yes** — 192.168.1.100:8181, on Isis. Promoted to
  the primary watch-history source; see §4.8–4.11.
- ~~Plex history retention on Loki~~ — no longer on the critical path, since Tautulli
  keeps its own independent record.
- ~~Where does the image get built?~~ Built off-box, delivered to Nox's Portainer (§11.2).

### Still open

### Resolved by measurement (2026-09-07)

Questions 1–4 were answered by running `tools/probe.py` and `tools/reclaim_preview.py`
against the live servers. The findings are in §2.1 and changed the design in five places:
the Tautulli size cross-check (§4.11), both Tautulli-dependent score weights (§6.1),
cross-library duplicates (§6.5), keep marks (§6.6), and two new seed views (§7.2).

- ~~When was Tautulli installed?~~ **2025-03-08**, 547 days of coverage — but the library
  goes back to 2000, so 80% of it predates that. `pre_history` is the majority state.
- ~~Is successive-play grouping on?~~ Immaterial — MediaShelf requests `grouping=0` and
  merges itself. Confirmed working against 87,640 real rows.
- ~~Multiple libraries of the same kind?~~ **24 libraries**, 16 of them movie sections.
- ~~Path structure?~~ **titan4 / titan3 / titan1**, and grouping by root is worth building.

### Still open

1. **Registry or image upload?** (§11.2) A registry is the better long-term answer but is
   infrastructure that doesn't exist yet. A preference, not a measurement.
2. **Which libraries, if any, should be kept wholesale?** Deliberately left unanswered —
   nothing ships kept (§6.6), and this is a judgement to make against a real report rather
   than in advance. The candidates the measurements suggest are *Family Videos*
   (irreplaceable home video), *4K Movies* (72.9% never played and expensive to
   re-acquire, though marking it removes 5 TB from consideration permanently), and
   *Sports*, which looks like personal capture. All three are equally well served by
   explicit marks on the items that actually matter.
3. **What should the 18.8% partial-view band mean?** Four times the abandoned share and
   genuinely ambiguous (§6.1). v1 stores and displays it without scoring it; a month of
   looking at real rows should settle whether it earns a component.
4. **Is the 20.9 TB uncertain pool worth attacking at all?** It is three times the size of
   the confident pool, and the honest answer is that nobody can tell from data whether
   those items were watched in 2019. Options are to leave it to human judgement, to
   import older history if any exists elsewhere, or to accept that content untouched for
   18+ months is stale enough regardless of what happened before. Worth deciding
   deliberately rather than by default.