offlineu/README.md

# OfflineU

Self-hosted course viewer for a local video/audio/text training library. Runs
as a single Flask app in Docker, tracks watch progress and notes per lesson,
and gives the whole library a dashboard-style home page instead of a bare
file browser.

This is a personal deployment, not a public project — this doc is internal
reference for running/maintaining it, not a pitch.

![Lesson view](images/lesson-0-8-2025-08-04-04_58_17.png)
*Screenshot predates the current theming/icon work — kept for a rough idea
of layout, not pixel-accurate.*

---

## Deployment

Deployed via `docker-compose.yml`, built directly from the `Dockerfile` in
this repo (`pull_policy: build`, not a registry pull):

```yaml
services:
  offlineu:
    build: .
    pull_policy: build
    container_name: offlineu
    network_mode: host
    ports:
      - "5000:5000"
    environment:
      - PUID=1000
      - PGID=10
      - TZ=America/New_York
      - FLASK_ENV=production
    volumes:
      - /volume1/files/training:/app/courses   # course library
      - /volume2/docker/offlineu/data:/app/data # settings/progress/notes
    restart: unless-stopped
```

**Auto-deploy:** the Gitea repo has a webhook to Dockhand — a push to `main`
triggers a full image **rebuild** from the Dockerfile (confirmed, not just a
container restart), so Dockerfile changes (e.g. adding `ffmpeg`) take effect
on the very next push without any manual step.

**Confirming a deploy landed:** a plain `VERSION` file at the repo root
(a timestamp + short description, updated by hand alongside each commit)
ships into the image via the normal `COPY . .` and is shown at the bottom
of the Settings page and in `/health`'s JSON response, so after a push you
can check the running container actually picked it up instead of
guessing. Deliberately not derived from `git rev-parse` at Docker build
time - Dockhand's build context doesn't reliably have `.git` available,
which silently produced "unknown" instead of an actual commit. Reads as
`dev` outside Docker (no `VERSION` file to read, e.g. before the first
commit that adds one). `CHANGELOG.md` at the repo root accumulates that
same line on every commit (newest first) instead of overwriting it, and
shows as a "What's New" list under the version line in Settings - so you
can see recent history, not just the current build.

---

## Local development

```bash
pip install -r requirements.txt
python offlineu_core.py --library-path /path/to/courses --debug
```

Opens on `http://127.0.0.1:5000`. `ffprobe` (from ffmpeg) needs to be on
`PATH` for video-duration lookups to work locally; without it those just
silently stay blank instead of erroring.

### CLI options

| Option                | Description                                          |
| ---------------------- | ----------------------------------------------------- |
| `--host`               | Bind host (default `0.0.0.0`)                          |
| `--port`               | Bind port (default `5000`)                             |
| `--debug`               | Enable Flask debug mode                                |
| `--create-templates`    | Regenerate default templates if missing                |
| `--library-path`        | Course library root (overrides `COURSES_LIBRARY_PATH`) |
| `course_path` (positional) | Load a specific course directly at startup           |

### Environment variables

| Variable              | Default        | Purpose                                   |
| ---------------------- | -------------- | ------------------------------------------ |
| `COURSES_LIBRARY_PATH`  | `/app/courses` | Course library root                        |
| `OFFLINEU_DATA_DIR`     | `/app/data`    | Where settings/progress/notes data lives   |
| `AUTO_LOAD_COURSE`      | —              | Course path to load automatically on start |

---

## Features

**Library browsing**
- Lazy-loading folder browser (grid or list view) with search, including an
  opt-in transcript search across subtitle files
- Course cards show media file count, total video/audio runtime, and
  completion % at a glance
- Cover art: uses a manually-placed `cover`/`folder`/`thumbnail`/`thumb`/
  `poster` image if a course has one, otherwise auto-generates one via
  `ffmpeg` - samples a few candidate frames from the first several seconds
  of the course's first video (where an intro title card typically lives)
  and keeps the one that compresses to the largest JPEG, a cheap proxy for
  "has the most going on" that favors a title card/logo over a blank
  fade-in or a plain frame of the presenter
- **File Management** ([/unsorted](templates/unsorted.html), its own item
  ("Files") in the bottom tab bar alongside Home/Notes/Help/Settings): a
  dedicated page for everything that touches files on disk, since it needs
  more room than a Settings card and warrants review before anything
  actually changes.
  - *Sort Unsorted*: drop new/incoming courses into an `Unsorted` folder at
  the library root, then scan proposes a destination for each one by keyword
  overlap against the *existing* category tree - read fresh from disk every
  scan, so it adapts to whatever folders you actually have rather than any
  hardcoded subject list. Three outcomes per item: a confident match to an
  existing folder, a suggested new subfolder under a broader category match,
  or "needs review" (unchecked by default) when nothing overlaps at all.
  Every row has a folder-picker dropdown (listing every real category in the
  library, plus "+ Create new folder…") so a wrong or low-confidence guess
  is one click to correct, and a name field to rename the course's folder
  in the same move. Creating a new folder means picking its parent from a
  dropdown and typing just the new folder's own name, with a live "Will
  create: X/Y" preview so the resulting path is confirmed before applying -
  the same picker Manage Library's Move action uses. The picker excludes
  actual course/item folders, not just organizational ones: a leaf folder
  holding a single non-video/audio file (an ebook, an audiobook in a
  format this app doesn't play, etc.); a folder whose whole subtree has no
  video/audio anywhere in it at all (a course's bundled source code, a
  Python virtualenv, project assets, ...) even though it has plenty of
  subfolders; and a release-bundle folder that wraps a single real course
  one level down under its own "course display name" folder, alongside
  unrelated (possibly empty) junk siblings at the same level - all three
  cases Library browsing's own course detection wouldn't catch either,
  since it only checks one level of nesting for video/audio. Nothing on
  disk moves until you review and hit Apply. Matching
  ignores common noise (e-learning
  platform names, release/distribution-group tags, dates) via a stopword
  list, and beyond that treats a match against a folder's own deliberate
  name as always stronger evidence than a word only borrowed from a
  sibling course's title - and among those borrowed words, downweights
  ones that recur across many categories (a prolific creator's name, etc.)
  so they can't outvote a genuinely specific word just by sharing more of
  them.
  - *Refresh Library*: manually bypasses the 5-minute filesystem-scan cache
  (this now includes the category tree Sort Unsorted/Manage Library's
  picker builds - previously rebuilt on every page load, a full,
  uncached walk of the whole library that got noticeably slow over a
  NAS-mounted (SMB) library as the course count grew), for when files
  were added/removed directly on disk.
  - *Bulk Rename*: find & replace across every course/folder name in the
  library at once, with a per-match preview and the ability to drop
  individual matches before applying. Three match modes: plain text
  (default, literal substring), wildcard (shell-style `*`/`?`, e.g.
  `.BOOKWARE*` catches `.BOOKWARE-GETH`, `.BOOKWARE-BOOKTIME`,
  `.BOOKWARE-BLZiSO`, etc. in one pass), or full regex (with
  backreferences in the replacement, e.g. `\1`).
  - *Manage Library*: hide courses/folders from the browser without
  touching anything on disk (hiding a folder hides everything inside it;
  bulk-select elsewhere to hide or queue several at once), rename a
  course/folder in place, or **move** one anywhere else in the library via
  the same folder-picker/create-new-folder UI Sort Unsorted uses - handy
  for correcting a bad auto-sort later or just reorganizing. A search box
  finds any course/folder by name anywhere in the tree instead of
  expanding levels one by one (results get the same Move/Rename/Hide
  actions, just without an expand arrow, since a hit is somewhere specific
  rather than a level to browse into); every row also has a checkbox, so
  several items - found via search or expanded across different tree
  levels - can be hidden, shown, or moved to the same destination together
  in one batch instead of one at a time.
  - *Undo*: a "Last action: ... [Undo]" bar appears after any move or
  rename (Sort Unsorted apply, Bulk Rename apply, a Manage Library
  move/rename, a bulk move) and reverses the whole batch in one click.
  Only ever covers move/rename - hiding is already a one-click toggle with
  nothing to undo, and Delete is permanent by design, so it's deliberately
  never in this history no matter how "undo" gets framed. Re-checks each
  item before reversing it (the original spot may have been reused since),
  so a partial batch failure reports exactly what did and didn't reverse
  rather than silently doing nothing.
  - *Duplicate Courses*: scans every course in the library (Unsorted
  included) for names that look like the same thing filed twice, using the
  same tokenizer as Sort Unsorted so a platform-name or release-tag
  difference doesn't hide a real duplicate. Groups by similarity (union of
  any two courses over the threshold, transitively) rather than showing
  raw pairs; each course in a group gets a one-click Hide, or a **Delete**
  that permanently removes it from disk (confirmed with the full path
  before it runs). A group can also be marked **"Not a duplicate"** if the
  match is wrong, which excludes that specific pair from future scans
  without touching anything else that happens to match one of those
  courses; ignored pairs are listed (and reversible) under "Ignored
  matches," and persist through backup/restore alongside hidden paths and
  Next Up.
  - *Duplicate Lesson Files*: the same idea one level down - media files
  sitting in the same folder that look like the same lesson downloaded
  twice (e.g. a re-download that landed alongside the original instead of
  replacing it). Compares only within a folder, at a higher match
  threshold than Duplicate Courses, so two different lessons on a similar
  topic don't get flagged; shares its ignore list with Duplicate Courses.
  Delete here removes a single file, not a whole course.
  - *Clean Up Stale References*: finds entries in the hidden-paths list,
  Next Up queue, Favorites, or Recently Viewed history that point at a path
  no longer on disk - normally from renaming/moving/deleting a course
  directly on the NAS instead of through the app (doing it through the app
  already keeps these in sync). Review individually or clear them all at
  once; course files themselves are never touched.
  - *Storage Usage*: disk usage per top-level library folder, largest
  first, with a simple proportional bar per entry - manually triggered
  (it reads every file's size) rather than run automatically. Click any
  folder row to drill into its own contents one level at a time, with a
  Back button to climb back out.

**Dashboard**
- Library-wide stats (courses, lessons completed, time watched, time
  remaining, day streak) and a 90-day activity heatmap
- Favorites — star any course from its list row or its own page for quick
  access from the dashboard, independent of recency or curated queues
- Next Up queue (manually curated, reorderable)
- Pick Back Up (courses with progress that have gone stale)
- Recently Added / Recently Viewed
- Surprise Me — dice icon in the header, random pick weighted toward
  incomplete courses
- Grid view shows a small progress ring in the corner of each course's
  thumbnail (a checkmark once complete) instead of a separate bar, so
  completion reads at a glance without switching to list view

**Playback & progress**
- Video/audio player with resize, playback-speed presets, and resume-from-
  last-position
- Auto-play next lesson when one ends, with a cancelable few-second
  countdown - on by default, toggle it off in Settings → Video Player
- Keyboard shortcuts on the lesson page: Space (play/pause), ←/→ (seek
  10s), ↑/↓ (volume), `,`/`.` (step playback speed), `[`/`]` (previous/next
  lesson), `F` (fullscreen), `N` (quick-capture a note) - see Help for the
  full list
- Auto-tracks watch progress and completion per lesson
- Video/audio durations read via `ffprobe` and cached persistently per
  course (`.offlineu_duration_cache.json`), so total runtime and per-lesson
  length show up before you've ever pressed play — not just after; the
  same cache backs the library-wide "time remaining" stat
- Settings → "Precompute Lengths & Cover Art" walks the whole library in
  the background to populate durations and thumbnails up front, with live
  progress
- Settings → "Regenerate Thumbnails" deletes and re-runs auto-generated
  cover art for every course that has one (manually-placed covers are
  never touched) - the only way to pick up an improvement to how
  thumbnails are picked on courses whose thumbnail was already cached
  under the old logic, since it's otherwise served from disk forever

**Notes**
- Timestamped notes per lesson, capturable via a keyboard shortcut without
  leaving the player
- Notes Hub: every note across the whole library, searchable, in one list
- Per-course study guide export (markdown, organized by section)

**Personalization** (Settings)
- 46 built-in themes (grouped in the dropdown: Base, Editor Themes, and 5
  categories of website-inspired schemes sourced from Figma's "53 Unique
  Website Color Schemes" - 34 of the 53 kept, all as dark/colored
  backgrounds rather than the source page's mostly-white mockups, after
  cutting a couple of neon-bright accents and pruning near-duplicates by
  actual color distance), or a fully custom accent color, corner radius,
  and card style
- Font, text size, page width, and spacing density
- Default library path override (Refresh Library and Bulk Rename live on the
  File Management page - see above)

**Backup & integrations**
- One-click backup export and restore: settings, hidden-path choices, the
  Next Up queue, Favorites, recent-view history, ignored-duplicate pairs,
  Outline config, and every course's progress/notes as a zip (not the
  course files themselves). Restore
  overwrites current data and needs a matching course folder to already
  exist for each course's progress to land - it's a "put my data back"
  action, not a merge
- Optional [Outline](https://www.getoutline.com/) integration — push a
  lesson's notes to an Outline document/collection

---

## Data files

Everything under `OFFLINEU_DATA_DIR` (app-wide, not tied to a course):

| File                  | Contents                                  |
| ---------------------- | ------------------------------------------ |
| `settings.json`        | Theme, layout, library path, etc.          |
| `hidden_paths.json`    | Courses/folders curated out of the browser |
| `next_up.json`         | The Next Up queue, in order                |
| `favorites.json`       | Favorited course paths                     |
| `recent_views.json`    | Cross-course "Recently Viewed" history     |
| `ignored_duplicates.json` | Course-path pairs confirmed not duplicates |
| `outline_config.json`  | Outline API token/collection mapping       |

Per-course, written inside the course's own folder on the library volume:

| File                              | Contents                                              |
| ----------------------------------- | -------------------------------------------------------- |
| `.offlineu_progress.json`           | Per-lesson completed/progress/duration/notes             |
| `.offlineu_duration_cache.json`     | ffprobe duration cache, keyed by (path, size, mtime)     |
| `.offlineu_thumbnail.jpg`           | Auto-generated cover art (only if no manual cover exists) |

---

## Folder structure example

```
MyCourse/
├── Section 1/
│   ├── 01 - Intro.mp4
│   ├── 02 - Setup Guide.pdf
│   └── 03 - Quiz.html
├── Section 2/
│   ├── 04 - Advanced Tips.mp4
│   └── resources/
│       └── extras.md
├── .offlineu_progress.json        ← created automatically
├── .offlineu_duration_cache.json  ← created automatically
└── .offlineu_thumbnail.jpg        ← created automatically, only if no manual cover exists
```

No metadata files needed — course/section/lesson names come straight from
folder and file names.

An `Unsorted/` folder at the library root (alongside the real category
folders) is the drop point for new/incoming courses — see Sort Unsorted
above.

## Supported file types

| Type      | Extensions                                                        |
| --------- | ------------------------------------------------------------------ |
| Video     | `.mp4` `.mkv` `.avi` `.mov` `.webm` `.m4v` `.flv` `.wmv`           |
| Audio     | `.mp3` `.wav` `.m4a` `.aac` `.ogg` `.flac`                          |
| Documents | `.txt` `.md` `.html` `.htm` `.pdf` `.docx` `.doc` `.rtf`            |
| Subtitles | `.srt` `.vtt` `.ass` `.sub` `.sbv`                                  |
| Quizzes   | Any doc file whose name contains `quiz`, `exam`, `test`, `assessment`, `exercise`, `assignment`, or `homework` |

---

## Tools

`tools/generate_website_scheme_themes.py` is the one-off script that
produced the 34 "website scheme" theme entries in `THEME_PALETTES` (of
53 candidates - all 24 light-mode ones were converted to dark/colored
backgrounds per request, then 19 were cut across two review passes: 2
for a neon-bright accent, 17 for landing too close to another kept
scheme once mapped into actual UI colors, since forcing everything dark
made several collide that hadn't before) - not part of the running app,
kept for reference if those palettes ever need regenerating or
extending. `build_theme()` takes a `force_mood='dark'|'light'` override
for exactly this kind of bulk remapping. See its module docstring for
how it turned Figma's color-scheme screenshots into actual UI palettes.