Jess Hallsworth
6a557bcdd9
The application the design describes: Flask + SQLite, Plex for library data, Tautulli for watch history, report-only. Structure follows the design's seams. providers/ splits MediaProvider from HistoryProvider, because on this network library data and watch data live on different machines and Jellyfin later will have no Tautulli equivalent. scoring.py implements the reclaim score twice - as a SQL expression for the live grid (weights change on every slider drag, so storing it would mean rewriting thousands of rows per drag) and in Python for CSV export and tests, with a property test over 500 generated rows asserting the two agree. rules.py compiles saved views to parameterized SQL through a field/operator whitelist; nothing user-supplied is ever interpolated. Three properties are enforced by test rather than asserted in prose: - Ingest is idempotent. Three consecutive full scans leave every count and every byte total unchanged. A scanner that double-counts produces a report that looks plausible and is wrong. - Keep marks survive Plex reassigning every rating key in the library. They are keyed on content GUID, scoped per library so the Movies and 4K Movies copies of the same film mark independently. - Every config variable the app reads is declared in docker-compose.yml, so a variable set in Portainer can never silently do nothing. Also found and fixed while verifying against a fake Plex+Tautulli pair: executescript() commits the pending transaction, so migrations needed their BEGIN/COMMIT inside the script; replaceChildren() renders null as the literal text "null"; a hash-only URL change does not reload the document, so deep links needed a hashchange listener; and SQLite ROUND rounds half away from zero where Python rounds half to even. 73 tests, no live server required. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GVbG48GAXfCZatcmX123Ra
120 lines
5.7 KiB
Markdown
120 lines
5.7 KiB
Markdown
# MediaShelf
|
|
|
|
A self-hosted web app for figuring out which of the thousands of files in a Plex library
|
|
are actually worth keeping.
|
|
|
|
MediaShelf scans a Plex Media Server over its HTTP API and pulls watch history from
|
|
Tautulli, builds a local snapshot of every movie and TV season, and joins together facts
|
|
that are never shown side by side — date added, size on disk, file path, owning library,
|
|
watch count, last watched, and *how much of it anyone actually finished* — into a
|
|
sortable, filterable, chartable grid with a tunable **reclaim score** that ranks deletion
|
|
candidates.
|
|
|
|
## Status
|
|
|
|
**v1 is report-only.** MediaShelf does not delete, move, or modify anything — not a file,
|
|
not a Plex record. It produces a ranked list, saved rule sets, and CSV export. Deletion is
|
|
designed for in the roadmap but deliberately not built, so the scanner and the scoring
|
|
model can be trusted before anything destructive is wired up.
|
|
|
|
The application is implemented and tested; it has not yet been deployed. The design was
|
|
validated against the live servers first: 65.7 TB across 24 libraries, 2,930 movies and
|
|
2,807 TV seasons, 87,640 logged plays from 60 users, and 27.4 TB never played. See
|
|
`docs/design.md` §2.1.
|
|
|
|
## Running it
|
|
|
|
```bash
|
|
pip install -r requirements-dev.txt
|
|
cp .env.example .env # fill in PLEX_TOKEN and TAUTULLI_API_KEY
|
|
|
|
python3 -m mediashelf.cli scan --full # first ingest
|
|
python3 -m mediashelf.cli serve # http://127.0.0.1:8080
|
|
make test # 73 tests, no live server needed
|
|
```
|
|
|
|
Deployment is a single container behind Nginx Proxy Manager — see `docs/design.md` §11.
|
|
Build off-box and push the image to Nox's Portainer; `docker-compose.yml` is the stack.
|
|
|
|
## Layout
|
|
|
|
```
|
|
mediashelf/
|
|
config.py every setting, read once from the environment
|
|
db.py SQLite (WAL) + numbered SQL migrations
|
|
providers/ Plex (library) and Tautulli (history) behind two protocols
|
|
ingest.py scan orchestration, rollups, keep resolution
|
|
scoring.py the reclaim score, in SQL and in Python
|
|
rules.py saved-view grammar -> parameterized SQL, whitelist only
|
|
keeps.py GUID-keyed keep marks
|
|
queries.py the item query, live score, dashboard aggregates
|
|
api.py / web.py JSON API and the page shell
|
|
tools/ read-only probe, reclaim preview, fake Plex+Tautulli
|
|
tests/ 73 tests, run entirely against the fake server
|
|
```
|
|
|
|
## What it does
|
|
|
|
- Full-library ingest from Plex, no agent on the Plex host, no filesystem mounts
|
|
- Movies at item level, TV rolled up to **season** level
|
|
- Watch data from **Tautulli**, so plays by *every* account are counted and a play that
|
|
was abandoned after five minutes is distinguished from one that was finished — Plex's
|
|
own history reports those identically. Falls back to Plex session history if Tautulli
|
|
is unavailable, and says so rather than degrading silently
|
|
- Sort and filter on every metric; charts for size by library, additions over time,
|
|
finished vs. abandoned vs. never-opened by size, and size vs. last-watched
|
|
- A weighted reclaim score with live sliders, and grace rules so it never recommends
|
|
something you added last week
|
|
- Named, re-runnable saved views — *"unwatched, older than 2 years, over 10 GB"*,
|
|
*"two people started it and nobody finished it"*
|
|
- **Keep marks** at library, series, season or movie level, for the things you are holding
|
|
on purpose — home video, 4K copies, shows you might want someday. Kept items are hidden
|
|
from the working grid and refused outright by deletion. Marks are keyed on Plex content
|
|
GUIDs rather than rating keys, so they survive a library rebuild
|
|
- CSV export of any view
|
|
|
|
## Stack
|
|
|
|
Python + Flask, SQLite (WAL), vanilla JS front-end (no build step), single container
|
|
deployed as a Portainer stack behind Nginx Proxy Manager.
|
|
|
|
## Roadmap
|
|
|
|
- **v2** — two-stage quarantine-then-purge deletion, with authentication, a path
|
|
allowlist, and an audit log
|
|
- **v3** — Emby and Jellyfin support behind the existing `MediaProvider` abstraction
|
|
|
|
## Read-only LAN tools
|
|
|
|
Plex and Tautulli are both LAN-only, so these exist to check the design and the numbers
|
|
against real data from inside the network. It is **read-only** — GET requests only,
|
|
nothing is modified — and has no dependencies beyond the standard library.
|
|
|
|
```bash
|
|
export PLEX_BASE_URL=http://192.168.1.10:32400
|
|
export PLEX_TOKEN=...
|
|
export TAUTULLI_BASE_URL=http://192.168.1.100:8181
|
|
export TAUTULLI_API_KEY=...
|
|
|
|
python3 tools/probe.py # library shapes, sizes, coverage horizon
|
|
python3 tools/probe.py --dump inventory.json # plus a full item inventory
|
|
python3 tools/reclaim_preview.py # the actual reclaim numbers
|
|
```
|
|
|
|
`reclaim_preview.py` joins Plex sizes to Tautulli plays the way the application will, and
|
|
reports never-played bytes per library — split into the *confident* pool (added while
|
|
Tautulli was watching, never played) and the *uncertain* pool (predates Tautulli, might
|
|
have been watched). It also prints the largest never-played TV seasons and the
|
|
watcher-count distribution used to calibrate the score.
|
|
|
|
It reports Tautulli's coverage horizon, the finished/abandoned/never-opened split across
|
|
real plays, library shapes and sizes, path roots, multi-version items, and whether
|
|
Tautulli is actually watching the Plex server you think it is. Credentials are redacted
|
|
from all output including error messages, so the result is safe to paste anywhere.
|
|
|
|
## Documentation
|
|
|
|
- [`docs/design.md`](docs/design.md) — the full software design
|
|
- [`tools/probe.py`](tools/probe.py) — read-only reconnaissance script
|
|
- [`tools/reclaim_preview.py`](tools/reclaim_preview.py) — read-only reclaim numbers
|
|
- [`tools/mockserver.py`](tools/mockserver.py) — mock Plex/Tautulli for testing the probe
|