offlineu/README.md

# OfflineU

Self-hosted course viewer for a local video/audio/text training library. Runs
as a single Flask app in Docker, tracks watch progress and notes per lesson,
and gives the whole library a dashboard-style home page instead of a bare
file browser.

This is a personal deployment, not a public project — this doc is internal
reference for running/maintaining it, not a pitch.

![Lesson view](images/lesson-0-8-2025-08-04-04_58_17.png)
*Screenshot predates the current theming/icon work — kept for a rough idea
of layout, not pixel-accurate.*

---

## Deployment

Deployed via `docker-compose.yml`, built directly from the `Dockerfile` in
this repo (`pull_policy: build`, not a registry pull):

```yaml
services:
  offlineu:
    build: .
    pull_policy: build
    container_name: offlineu
    network_mode: host
    ports:
      - "5000:5000"
    environment:
      - PUID=1000
      - PGID=10
      - TZ=America/New_York
      - FLASK_ENV=production
    volumes:
      - /volume1/files/training:/app/courses   # course library
      - /volume2/docker/offlineu/data:/app/data # settings/progress/notes
    restart: unless-stopped
```

**Auto-deploy:** the Gitea repo has a webhook to Dockhand — a push to `main`
triggers a full image **rebuild** from the Dockerfile (confirmed, not just a
container restart), so Dockerfile changes (e.g. adding `ffmpeg`) take effect
on the very next push without any manual step.

**Confirming a deploy landed:** a plain `VERSION` file at the repo root
(a timestamp + short description, updated by hand alongside each commit)
ships into the image via the normal `COPY . .` and is shown at the bottom
of the Settings page and in `/health`'s JSON response, so after a push you
can check the running container actually picked it up instead of
guessing. Deliberately not derived from `git rev-parse` at Docker build
time - Dockhand's build context doesn't reliably have `.git` available,
which silently produced "unknown" instead of an actual commit. Reads as
`dev` outside Docker (no `VERSION` file to read, e.g. before the first
commit that adds one).

---

## Local development

```bash
pip install -r requirements.txt
python offlineu_core.py --library-path /path/to/courses --debug
```

Opens on `http://127.0.0.1:5000`. `ffprobe` (from ffmpeg) needs to be on
`PATH` for video-duration lookups to work locally; without it those just
silently stay blank instead of erroring.

### CLI options

| Option                | Description                                          |
| ---------------------- | ----------------------------------------------------- |
| `--host`               | Bind host (default `0.0.0.0`)                          |
| `--port`               | Bind port (default `5000`)                             |
| `--debug`               | Enable Flask debug mode                                |
| `--create-templates`    | Regenerate default templates if missing                |
| `--library-path`        | Course library root (overrides `COURSES_LIBRARY_PATH`) |
| `course_path` (positional) | Load a specific course directly at startup           |

### Environment variables

| Variable              | Default        | Purpose                                   |
| ---------------------- | -------------- | ------------------------------------------ |
| `COURSES_LIBRARY_PATH`  | `/app/courses` | Course library root                        |
| `OFFLINEU_DATA_DIR`     | `/app/data`    | Where settings/progress/notes data lives   |
| `AUTO_LOAD_COURSE`      | —              | Course path to load automatically on start |

---

## Features

**Library browsing**
- Lazy-loading folder browser (grid or list view) with search, including an
  opt-in transcript search across subtitle files
- Course cards show media file count, total video/audio runtime, and
  completion % at a glance
- Cover art: uses a manually-placed `cover`/`folder`/`thumbnail`/`thumb`/
  `poster` image if a course has one, otherwise auto-generates one via
  `ffmpeg` from a frame of the course's first video
- **File Management** ([/unsorted](templates/unsorted.html), its own item
  ("Files") in the bottom tab bar alongside Home/Notes/Help/Settings): a
  dedicated page for everything that touches files on disk, since it needs
  more room than a Settings card and warrants review before anything
  actually changes.
  - *Sort Unsorted*: drop new/incoming courses into an `Unsorted` folder at
  the library root, then scan proposes a destination for each one by keyword
  overlap against the *existing* category tree - read fresh from disk every
  scan, so it adapts to whatever folders you actually have rather than any
  hardcoded subject list. Three outcomes per item: a confident match to an
  existing folder, a suggested new subfolder under a broader category match,
  or "needs review" (unchecked by default) when nothing overlaps at all.
  Every row has a folder-picker dropdown (listing every real category in the
  library, plus "+ Create new folder…") so a wrong or low-confidence guess
  is one click to correct, and a name field to rename the course's folder
  in the same move. Creating a new folder means picking its parent from a
  dropdown and typing just the new folder's own name, with a live "Will
  create: X/Y" preview so the resulting path is confirmed before applying -
  the same picker Manage Library's Move action uses. The picker excludes
  actual course/item folders, not just organizational ones: a leaf folder
  holding a single non-video/audio file (an ebook, an audiobook in a
  format this app doesn't play, etc.); a folder whose whole subtree has no
  video/audio anywhere in it at all (a course's bundled source code, a
  Python virtualenv, project assets, ...) even though it has plenty of
  subfolders; and a release-bundle folder that wraps a single real course
  one level down under its own "course display name" folder, alongside
  unrelated (possibly empty) junk siblings at the same level - all three
  cases Library browsing's own course detection wouldn't catch either,
  since it only checks one level of nesting for video/audio. Nothing on
  disk moves until you review and hit Apply. Matching
  ignores common noise (e-learning
  platform names, release/distribution-group tags, dates) via a stopword
  list, and beyond that treats a match against a folder's own deliberate
  name as always stronger evidence than a word only borrowed from a
  sibling course's title - and among those borrowed words, downweights
  ones that recur across many categories (a prolific creator's name, etc.)
  so they can't outvote a genuinely specific word just by sharing more of
  them.
  - *Refresh Library*: manually bypasses the 5-minute filesystem-scan cache,
  for when files were added/removed directly on disk.
  - *Bulk Rename*: find & replace across every course/folder name in the
  library at once, with a per-match preview and the ability to drop
  individual matches before applying. Three match modes: plain text
  (default, literal substring), wildcard (shell-style `*`/`?`, e.g.
  `.BOOKWARE*` catches `.BOOKWARE-GETH`, `.BOOKWARE-BOOKTIME`,
  `.BOOKWARE-BLZiSO`, etc. in one pass), or full regex (with
  backreferences in the replacement, e.g. `\1`).
  - *Manage Library*: hide courses/folders from the browser without
  touching anything on disk (hiding a folder hides everything inside it;
  bulk-select elsewhere to hide or queue several at once), rename a
  course/folder in place, or **move** one anywhere else in the library via
  the same folder-picker/create-new-folder UI Sort Unsorted uses - handy
  for correcting a bad auto-sort later or just reorganizing. A search box
  finds any course/folder by name anywhere in the tree instead of
  expanding levels one by one (results get the same Move/Rename/Hide
  actions, just without an expand arrow, since a hit is somewhere specific
  rather than a level to browse into); every row also has a checkbox, so
  several items - found via search or expanded across different tree
  levels - can be hidden, shown, or moved to the same destination together
  in one batch instead of one at a time.
  - *Undo*: a "Last action: ... [Undo]" bar appears after any move or
  rename (Sort Unsorted apply, Bulk Rename apply, a Manage Library
  move/rename, a bulk move) and reverses the whole batch in one click.
  Only ever covers move/rename - hiding is already a one-click toggle with
  nothing to undo, and Delete is permanent by design, so it's deliberately
  never in this history no matter how "undo" gets framed. Re-checks each
  item before reversing it (the original spot may have been reused since),
  so a partial batch failure reports exactly what did and didn't reverse
  rather than silently doing nothing.
  - *Duplicate Courses*: scans every course in the library (Unsorted
  included) for names that look like the same thing filed twice, using the
  same tokenizer as Sort Unsorted so a platform-name or release-tag
  difference doesn't hide a real duplicate. Groups by similarity (union of
  any two courses over the threshold, transitively) rather than showing
  raw pairs; each course in a group gets a one-click Hide, or a **Delete**
  that permanently removes it from disk (the one destructive action in the
  whole app - confirmed with the full path before it runs). A group can
  also be marked **"Not a duplicate"** if the match is wrong, which
  excludes that specific pair from future scans without touching anything
  else that happens to match one of those courses; ignored pairs are
  listed (and reversible) under "Ignored matches," and persist through
  backup/restore alongside hidden paths and Next Up.
  - *Clean Up Stale References*: finds entries in the hidden-paths list,
  Next Up queue, or Recently Viewed history that point at a path no longer
  on disk - normally from renaming/moving/deleting a course directly on
  the NAS instead of through the app (doing it through the app already
  keeps these in sync). Review individually or clear them all at once;
  course files themselves are never touched.
  - *Storage Usage*: disk usage per top-level library folder, largest
  first, with a simple proportional bar per entry - manually triggered
  (it reads every file's size) rather than run automatically.

**Dashboard**
- Library-wide stats (courses, lessons completed, time watched, time
  remaining, day streak) and a 90-day activity heatmap
- Next Up queue (manually curated, reorderable)
- Pick Back Up (courses with progress that have gone stale)
- Recently Added / Recently Viewed
- Surprise Me — dice icon in the header, random pick weighted toward
  incomplete courses

**Playback & progress**
- Video/audio player with resize, playback-speed presets, and resume-from-
  last-position
- Auto-tracks watch progress and completion per lesson
- Video/audio durations read via `ffprobe` and cached persistently per
  course (`.offlineu_duration_cache.json`), so total runtime and per-lesson
  length show up before you've ever pressed play — not just after; the
  same cache backs the library-wide "time remaining" stat
- Settings → "Precompute Lengths & Cover Art" walks the whole library in
  the background to populate durations and thumbnails up front, with live
  progress

**Notes**
- Timestamped notes per lesson, capturable via a keyboard shortcut without
  leaving the player
- Notes Hub: every note across the whole library, searchable, in one list
- Per-course study guide export (markdown, organized by section)

**Personalization** (Settings)
- 46 built-in themes (grouped in the dropdown: Base, Editor Themes, and 5
  categories of website-inspired schemes sourced from Figma's "53 Unique
  Website Color Schemes" - 34 of the 53 kept, all as dark/colored
  backgrounds rather than the source page's mostly-white mockups, after
  cutting a couple of neon-bright accents and pruning near-duplicates by
  actual color distance), or a fully custom accent color, corner radius,
  and card style
- Font, text size, page width, and spacing density
- Default library path override (Refresh Library and Bulk Rename live on the
  File Management page - see above)

**Backup & integrations**
- One-click backup export and restore: settings, hidden-path choices, the
  Next Up queue, recent-view history, ignored-duplicate pairs, Outline
  config, and every course's progress/notes as a zip (not the course files
  themselves). Restore
  overwrites current data and needs a matching course folder to already
  exist for each course's progress to land - it's a "put my data back"
  action, not a merge
- Optional [Outline](https://www.getoutline.com/) integration — push a
  lesson's notes to an Outline document/collection

---

## Data files

Everything under `OFFLINEU_DATA_DIR` (app-wide, not tied to a course):

| File                  | Contents                                  |
| ---------------------- | ------------------------------------------ |
| `settings.json`        | Theme, layout, library path, etc.          |
| `hidden_paths.json`    | Courses/folders curated out of the browser |
| `next_up.json`         | The Next Up queue, in order                |
| `recent_views.json`    | Cross-course "Recently Viewed" history     |
| `ignored_duplicates.json` | Course-path pairs confirmed not duplicates |
| `outline_config.json`  | Outline API token/collection mapping       |

Per-course, written inside the course's own folder on the library volume:

| File                              | Contents                                              |
| ----------------------------------- | -------------------------------------------------------- |
| `.offlineu_progress.json`           | Per-lesson completed/progress/duration/notes             |
| `.offlineu_duration_cache.json`     | ffprobe duration cache, keyed by (path, size, mtime)     |
| `.offlineu_thumbnail.jpg`           | Auto-generated cover art (only if no manual cover exists) |

---

## Folder structure example

```
MyCourse/
├── Section 1/
│   ├── 01 - Intro.mp4
│   ├── 02 - Setup Guide.pdf
│   └── 03 - Quiz.html
├── Section 2/
│   ├── 04 - Advanced Tips.mp4
│   └── resources/
│       └── extras.md
├── .offlineu_progress.json        ← created automatically
├── .offlineu_duration_cache.json  ← created automatically
└── .offlineu_thumbnail.jpg        ← created automatically, only if no manual cover exists
```

No metadata files needed — course/section/lesson names come straight from
folder and file names.

An `Unsorted/` folder at the library root (alongside the real category
folders) is the drop point for new/incoming courses — see Sort Unsorted
above.

## Supported file types

| Type      | Extensions                                                        |
| --------- | ------------------------------------------------------------------ |
| Video     | `.mp4` `.mkv` `.avi` `.mov` `.webm` `.m4v` `.flv` `.wmv`           |
| Audio     | `.mp3` `.wav` `.m4a` `.aac` `.ogg` `.flac`                          |
| Documents | `.txt` `.md` `.html` `.htm` `.pdf` `.docx` `.doc` `.rtf`            |
| Subtitles | `.srt` `.vtt` `.ass` `.sub` `.sbv`                                  |
| Quizzes   | Any doc file whose name contains `quiz`, `exam`, `test`, `assessment`, `exercise`, `assignment`, or `homework` |

---

## Tools

`tools/generate_website_scheme_themes.py` is the one-off script that
produced the 34 "website scheme" theme entries in `THEME_PALETTES` (of
53 candidates - all 24 light-mode ones were converted to dark/colored
backgrounds per request, then 19 were cut across two review passes: 2
for a neon-bright accent, 17 for landing too close to another kept
scheme once mapped into actual UI colors, since forcing everything dark
made several collide that hadn't before) - not part of the running app,
kept for reference if those palettes ever need regenerating or
extending. `build_theme()` takes a `force_mood='dark'|'light'` override
for exactly this kind of bulk remapping. See its module docstring for
how it turned Figma's color-scheme screenshots into actual UI palettes.