From acabb8998190cb0e66798ded9ebca7edd79f75dd Mon Sep 17 00:00:00 2001 From: rmsitz Date: Sun, 23 Aug 2026 22:50:48 -0400 Subject: [PATCH] Add auto-thumbnails, library-wide remaining time, backup restore Auto-generated course thumbnails: find_course_thumbnail() falls back to grabbing a frame from a course's first video via ffmpeg when there's no manual cover image, cached to .offlineu_thumbnail.jpg so it only ever runs once. Folded into the "Precompute" prewarm button, now "Precompute Lengths & Cover Art". Library-wide "Remaining" stat: sums every course's already-cached duration (no ffprobe, just a JSON read) into the dashboard's Library Stats card, alongside the existing "Watched" figure. Backup restore: new /api/backup/restore endpoint and a "Choose Backup File..." control in Settings, with zip-slip and missing-course guards. Also fixed a gap in the export itself - next_up.json and outline_config.json weren't being backed up before, so restore wouldn't have been a true round trip. Move Surprise Me from a full-width button above the course list to a dice-icon button in the dashboard header, swapping with the course-name badge depending on whether a course is loaded. Update README to cover all of the above. Co-Authored-By: Claude Sonnet 5 --- README.md | 39 ++++--- offlineu_core.py | 179 +++++++++++++++++++++++++++++--- templates/course_dashboard.html | 41 +++++--- templates/help.html | 2 +- templates/settings.html | 49 ++++++++- 5 files changed, 261 insertions(+), 49 deletions(-) diff --git a/README.md b/README.md index b0b8e52..46f5519 100644 --- a/README.md +++ b/README.md @@ -85,17 +85,21 @@ silently stay blank instead of erroring. opt-in transcript search across subtitle files - Course cards show media file count, total video/audio runtime, and completion % at a glance +- Cover art: uses a manually-placed `cover`/`folder`/`thumbnail`/`thumb`/ + `poster` image if a course has one, otherwise auto-generates one via + `ffmpeg` from a frame of the course's first video - Hide courses/folders from the browser without touching anything on disk; bulk-select to hide or queue several at once - Bulk rename across course/folder names (find & replace) **Dashboard** -- Library-wide stats (courses, lessons completed, time watched, day streak) - and a 90-day activity heatmap +- Library-wide stats (courses, lessons completed, time watched, time + remaining, day streak) and a 90-day activity heatmap - Next Up queue (manually curated, reorderable) - Pick Back Up (courses with progress that have gone stale) - Recently Added / Recently Viewed -- Surprise Me — random pick, weighted toward incomplete courses +- Surprise Me — dice icon in the header, random pick weighted toward + incomplete courses **Playback & progress** - Video/audio player with resize, playback-speed presets, and resume-from- @@ -103,9 +107,11 @@ silently stay blank instead of erroring. - Auto-tracks watch progress and completion per lesson - Video/audio durations read via `ffprobe` and cached persistently per course (`.offlineu_duration_cache.json`), so total runtime and per-lesson - length show up before you've ever pressed play — not just after -- Settings → "Precompute Video Lengths" walks the whole library in the - background to populate that cache up front, with live progress + length show up before you've ever pressed play — not just after; the + same cache backs the library-wide "time remaining" stat +- Settings → "Precompute Lengths & Cover Art" walks the whole library in + the background to populate durations and thumbnails up front, with live + progress **Notes** - Timestamped notes per lesson, capturable via a keyboard shortcut without @@ -120,9 +126,12 @@ silently stay blank instead of erroring. 5-minute filesystem-scan cache) **Backup & integrations** -- One-click backup export: settings, hidden-path choices, recent-view - history, and every course's progress/notes as a zip (not the course files - themselves) +- One-click backup export and restore: settings, hidden-path choices, the + Next Up queue, recent-view history, Outline config, and every course's + progress/notes as a zip (not the course files themselves). Restore + overwrites current data and needs a matching course folder to already + exist for each course's progress to land - it's a "put my data back" + action, not a merge - Optional [Outline](https://www.getoutline.com/) integration — push a lesson's notes to an Outline document/collection @@ -142,10 +151,11 @@ Everything under `OFFLINEU_DATA_DIR` (app-wide, not tied to a course): Per-course, written inside the course's own folder on the library volume: -| File | Contents | -| ----------------------------------- | ---------------------------------------------------- | -| `.offlineu_progress.json` | Per-lesson completed/progress/duration/notes | -| `.offlineu_duration_cache.json` | ffprobe duration cache, keyed by (path, size, mtime) | +| File | Contents | +| ----------------------------------- | -------------------------------------------------------- | +| `.offlineu_progress.json` | Per-lesson completed/progress/duration/notes | +| `.offlineu_duration_cache.json` | ffprobe duration cache, keyed by (path, size, mtime) | +| `.offlineu_thumbnail.jpg` | Auto-generated cover art (only if no manual cover exists) | --- @@ -162,7 +172,8 @@ MyCourse/ │ └── resources/ │ └── extras.md ├── .offlineu_progress.json ← created automatically -└── .offlineu_duration_cache.json ← created automatically +├── .offlineu_duration_cache.json ← created automatically +└── .offlineu_thumbnail.jpg ← created automatically, only if no manual cover exists ``` No metadata files needed — course/section/lesson names come straight from diff --git a/offlineu_core.py b/offlineu_core.py index a279211..b5151b3 100644 --- a/offlineu_core.py +++ b/offlineu_core.py @@ -36,6 +36,7 @@ TEXT_EXTENSIONS = {'.txt', '.md', '.html', '.htm', '.pdf', '.docx', '.doc', '.rt QUIZ_INDICATORS = {'quiz', 'exam', 'test', 'assessment', 'exercise', 'assignment', 'homework'} IMAGE_EXTENSIONS = {'.jpg', '.jpeg', '.png', '.webp'} THUMBNAIL_BASENAMES = ('cover', 'folder', 'thumbnail', 'thumb', 'poster') +AUTO_THUMBNAIL_FILENAME = '.offlineu_thumbnail.jpg' # Base directory the "Library" browser scans for courses, so users don't have # to type a full filesystem path. Matches the ./courses volume mount in @@ -560,20 +561,73 @@ def _has_direct_media(directory: Path) -> bool: def find_course_thumbnail(course_path: str) -> Optional[str]: """ - Look for a cover image directly inside a course folder (not recursive - - this only needs to catch the common 'cover.jpg next to the sections' - layout, not go hunting through every subfolder). + A course's cover image: a manually-placed one directly inside the + folder (cover/folder/thumbnail/thumb/poster.*, checked non-recursively + - just the common 'cover.jpg next to the sections' layout, not every + subfolder) if there is one, otherwise an auto-generated frame grabbed + from the course's first video. Almost none of this library's courses + ship with real cover art, so without the fallback the grid view would + be mostly bare initials. """ + course_dir = Path(course_path) try: - names = {f.name.lower(): f for f in Path(course_path).iterdir() if f.is_file()} + names = {f.name.lower(): f for f in course_dir.iterdir() if f.is_file()} except (PermissionError, OSError): return None + for base in THUMBNAIL_BASENAMES: for ext in IMAGE_EXTENSIONS: match = names.get(f'{base}{ext}') if match: return str(match) - return None + + auto_thumb = names.get(AUTO_THUMBNAIL_FILENAME) + if auto_thumb: + return str(auto_thumb) + + # Generating (or determining there's nothing to generate from) is the + # one part of this that isn't a cheap directory listing - cache the + # outcome briefly so repeat image requests for a doc-only course don't + # re-walk its files on every load. + return cache_get_or_compute( + f'auto_thumbnail:{course_dir}', + lambda: _generate_course_thumbnail(course_dir) + ) + + +def _generate_course_thumbnail(course_dir: Path) -> Optional[str]: + """ + Grab a single frame from the course's first video as stand-in cover + art, via ffmpeg, cached to .offlineu_thumbnail.jpg so this only ever + runs once per course - the next call finds that file directly through + the fast path above instead of hitting this function again. Returns + None (nothing cached to disk) if the course has no video at all, so a + video added later is still picked up on the next request past the + short in-memory cache above. + """ + video_files = sorted( + f for f in course_dir.rglob('*') + if f.is_file() and f.suffix.lower() in VIDEO_EXTENSIONS + ) + if not video_files: + return None + source = video_files[0] + + duration = _probe_media_duration_seconds(source) or 0 + offset = min(30.0, duration * 0.1) if duration else 5.0 + + output_path = course_dir / AUTO_THUMBNAIL_FILENAME + try: + result = subprocess.run( + ['ffmpeg', '-y', '-ss', str(offset), '-i', str(source), + '-frames:v', '1', '-vf', 'scale=480:-1', '-q:v', '3', str(output_path)], + capture_output=True, timeout=20 + ) + except (OSError, subprocess.SubprocessError): + return None + if result.returncode != 0 or not output_path.exists(): + return None + return str(output_path) _SECTION_NAME_RE = re.compile( @@ -985,6 +1039,24 @@ def _course_total_duration_seconds(course_dir: Path, media_files: List[Path]) -> return total if have_any else None +def _cached_course_duration_seconds(course_dir: Path) -> float: + """ + Sum of whatever's already in a course's persistent duration cache file, + without walking its files or ever touching ffprobe - for the + library-wide "time remaining" stat, which needs to stay cheap enough to + compute on every dashboard load (it runs inside the same per-course + loop as _scan_library_activity). A course that's never been viewed, + searched, or prewarmed yet just contributes 0 here; run "Precompute + Lengths & Cover Art" in Settings for a complete total. + """ + try: + with open(course_dir / DURATION_CACHE_FILENAME, 'r') as f: + cache = json.load(f) + except (FileNotFoundError, json.JSONDecodeError, OSError): + return 0.0 + return sum(entry.get('duration_seconds') or 0 for entry in cache.values()) + + def _course_summary(course_dir: Path) -> Dict[str, Any]: """Build the {type, name, path, media_files, hidden, has_thumbnail, duration_display} shape shared by the Library browser, search results, @@ -1185,6 +1257,7 @@ def _scan_library_activity() -> Dict[str, Any]: lessons_tracked = 0 completed_lessons = 0 watched_seconds = 0 + total_duration_seconds = 0.0 active_dates = set() activity_by_date: Dict[str, int] = {} courses: List[Dict[str, Any]] = [] @@ -1192,6 +1265,7 @@ def _scan_library_activity() -> Dict[str, Any]: for course_dir in get_all_course_dirs(): total_courses += 1 + total_duration_seconds += _cached_course_duration_seconds(course_dir) try: with open(course_dir / '.offlineu_progress.json', 'r') as f: progress = json.load(f) @@ -1248,6 +1322,7 @@ def _scan_library_activity() -> Dict[str, Any]: 'lessons_tracked': lessons_tracked, 'completed_lessons': completed_lessons, 'watched_seconds': watched_seconds, + 'total_duration_seconds': total_duration_seconds, 'streak_days': streak_days, 'activity_by_date': activity_by_date, 'courses': courses, @@ -1269,11 +1344,17 @@ def get_random_incomplete_course() -> Optional[Path]: def format_library_stats(scan: Dict[str, Any]) -> Dict[str, Any]: """Library-wide overview for the dashboard's stats card, from a _scan_library_activity() result.""" + remaining_seconds = max(0, scan['total_duration_seconds'] - scan['watched_seconds']) return { 'total_courses': scan['total_courses'], 'lessons_tracked': scan['lessons_tracked'], 'completed_lessons': scan['completed_lessons'], 'watched_display': format_duration(scan['watched_seconds']), + # None until at least some course's duration cache has been + # populated (via viewing/searching the library or running + # "Precompute Lengths & Cover Art") - a remaining estimate from a + # library that's mostly uncached would just be misleadingly low. + 'remaining_display': format_duration(remaining_seconds) if scan['total_duration_seconds'] else None, 'streak_days': scan['streak_days'] } @@ -2180,9 +2261,10 @@ _duration_prewarm_state: Dict[str, Any] = { def _run_duration_prewarm(): """ Walk every course in the library and populate its persistent duration - cache (see _course_total_duration_seconds) up front, so opening a - course for the first time doesn't pay the ffprobe cost right then - - it was already paid here, once, in the background. + cache (see _course_total_duration_seconds) and cover-art thumbnail + (see _generate_course_thumbnail) up front, so opening a course or + browsing the library for the first time doesn't pay either cost right + then - both were already paid here, once, in the background. """ global _duration_prewarm_state course_dirs = get_all_course_dirs() @@ -2196,6 +2278,7 @@ def _run_duration_prewarm(): if f.is_file() and f.suffix.lower() in VIDEO_EXTENSIONS | AUDIO_EXTENSIONS ] _course_total_duration_seconds(course_dir, media_files) + find_course_thumbnail(str(course_dir)) _duration_prewarm_state['done'] += 1 except Exception as e: _duration_prewarm_state['error'] = str(e) @@ -2210,7 +2293,7 @@ def _run_duration_prewarm(): @app.route('/api/library/prewarm-durations', methods=['POST']) def prewarm_durations_api(): - """Kick off (or report on an already-running) background scan of every course's video/audio duration.""" + """Kick off (or report on an already-running) background scan of every course's video/audio duration and cover art.""" if not _duration_prewarm_state['running']: threading.Thread(target=_run_duration_prewarm, daemon=True).start() return jsonify(_duration_prewarm_state) @@ -2489,21 +2572,30 @@ def reset_settings_api(): }) +# Shared between download_backup and restore_backup so the two can never +# drift out of sync with each other about what belongs at the archive root. +BACKUP_ROOT_FILES = { + 'settings.json': SETTINGS_FILE, + 'hidden_paths.json': HIDDEN_PATHS_FILE, + 'next_up.json': NEXT_UP_FILE, + 'recent_views.json': RECENT_VIEWS_FILE, + 'outline_config.json': OUTLINE_CONFIG_FILE, +} + + @app.route('/api/backup') def download_backup(): """ Bundle everything that isn't recoverable from the course files - themselves - settings, hidden-path curation, recently-viewed history, - and every course's progress/notes file - into a single downloadable - zip. Cheap insurance before a NAS migration or a docker volume mistake. + themselves - settings, hidden-path curation, the Next Up queue, + recently-viewed history, Outline config, and every course's + progress/notes file - into a single downloadable zip. Cheap insurance + before a NAS migration or a docker volume mistake; see restore_backup + for the other half of this round trip. """ buffer = io.BytesIO() with zipfile.ZipFile(buffer, 'w', zipfile.ZIP_DEFLATED) as zf: - for name, path in ( - ('settings.json', SETTINGS_FILE), - ('hidden_paths.json', HIDDEN_PATHS_FILE), - ('recent_views.json', RECENT_VIEWS_FILE), - ): + for name, path in BACKUP_ROOT_FILES.items(): if os.path.exists(path): zf.write(path, name) @@ -2519,6 +2611,59 @@ def download_backup(): return send_file(buffer, mimetype='application/zip', as_attachment=True, download_name=filename) +@app.route('/api/backup/restore', methods=['POST']) +def restore_backup(): + """ + Restore a zip produced by download_backup: the root-level app files in + BACKUP_ROOT_FILES, plus each course's progress file from + progress//.offlineu_progress.json. Overwrites + whatever's currently there - this is a deliberate "put my data back" + action, not a merge. + """ + uploaded = request.files.get('backup') + if not uploaded or not uploaded.filename: + return jsonify({'success': False, 'error': 'No backup file selected'}), 400 + + try: + zf = zipfile.ZipFile(io.BytesIO(uploaded.read())) + except zipfile.BadZipFile: + return jsonify({'success': False, 'error': 'That file is not a valid zip archive'}), 400 + + os.makedirs(DATA_DIR, exist_ok=True) + names = set(zf.namelist()) + restored = [] + skipped = [] + + for archive_name, dest_path in BACKUP_ROOT_FILES.items(): + if archive_name not in names: + continue + with zf.open(archive_name) as src, open(dest_path, 'wb') as dst: + dst.write(src.read()) + restored.append(archive_name) + + library_root = os.path.abspath(get_library_root()) + for name in names: + if not (name.startswith('progress/') and name.endswith('/.offlineu_progress.json')): + continue + relative = name[len('progress/'):-len('/.offlineu_progress.json')] + dest_dir = os.path.abspath(os.path.join(library_root, relative)) + # Guard against a zip entry trying to write outside the library + # root (zip-slip) - restore is a trusted, deliberate admin action, + # but the archive's paths themselves shouldn't be trusted blindly. + if not (dest_dir == library_root or dest_dir.startswith(library_root + os.sep)): + skipped.append(relative) + continue + if not os.path.isdir(dest_dir): + skipped.append(relative) + continue + with zf.open(name) as src, open(os.path.join(dest_dir, '.offlineu_progress.json'), 'wb') as dst: + dst.write(src.read()) + restored.append(name) + + invalidate_cache() + return jsonify({'success': True, 'restored': len(restored), 'skipped': skipped}) + + @app.route('/load_course', methods=['POST']) def load_course(): """Load course from selected directory""" diff --git a/templates/course_dashboard.html b/templates/course_dashboard.html index 32ebdba..461cf6a 100644 --- a/templates/course_dashboard.html +++ b/templates/course_dashboard.html @@ -136,6 +136,25 @@ white-space: nowrap; } + .header-icon-btn { + display: flex; + align-items: center; + justify-content: center; + width: 44px; + height: 44px; + flex-shrink: 0; + border-radius: 50%; + background: var(--bg-primary); + border: 1px solid var(--accent); + color: var(--accent); + text-decoration: none; + transition: background 0.2s, color 0.2s; + } + .header-icon-btn:hover { + background: var(--accent); + color: white; + } + .course-name-heading { /* course.name is the raw folder name, unlike lesson titles (cleaned up via _clean_lesson_name) - real course folders are @@ -288,14 +307,6 @@ box-sizing: border-box; } - .surprise-me-btn { - display: block; - width: 100%; - text-align: center; - font-size: 1.05em; - font-weight: 600; - box-sizing: border-box; - } .progress-bar { background: #444; @@ -982,6 +993,8 @@ {% if course %}
{{ course.name }}
+ {% elif library_stats and library_stats.total_courses %} + {{ icons.icon('dice', 20) }} {% endif %} @@ -1153,12 +1166,6 @@ {% endmacro %} - {% if library_stats and library_stats.total_courses %} -
- {{ icons.icon('dice', 16) }} Surprise Me -
- {% endif %} -

{{ icons.icon('grid') }}Your Courses

@@ -1212,6 +1219,12 @@
{{ library_stats.watched_display }}
Watched
+ {% if library_stats.remaining_display %} +
+
{{ library_stats.remaining_display }}
+
Remaining
+
+ {% endif %}
{{ library_stats.streak_days }}
Day streak
diff --git a/templates/help.html b/templates/help.html index 253ee91..f83ad65 100644 --- a/templates/help.html +++ b/templates/help.html @@ -163,7 +163,7 @@
  • Browse your library from the main page, or enter a path manually in Settings
  • Click a course to load it - folders drill down, courses open directly
  • Start learning! Progress, notes, and completion are all saved automatically
    • -
  • Not sure what to watch? {{ icons.icon('dice', 14) }} Surprise Me at the top of the dashboard picks a random course you haven't finished
    • +
  • Not sure what to watch? The dice icon {{ icons.icon('dice', 14) }} in the top-right of the dashboard header picks a random course you haven't finished
    • diff --git a/templates/settings.html b/templates/settings.html index 226db6e..2b91a13 100644 --- a/templates/settings.html +++ b/templates/settings.html @@ -451,10 +451,10 @@ Courses are cached briefly for speed - use this after adding or removing files directly on disk if you don't want to wait a few minutes for it to notice.
    - +
    - Reads every video/audio file's length up front and caches it, so course cards in the Library show their total runtime immediately instead of computing it the first time each course is viewed. + Reads every video/audio file's length and generates cover art for any course without one, up front - so course cards in the Library show their runtime and artwork immediately instead of computing it the first time each course is viewed.
    @@ -540,10 +540,18 @@

    Backup & Export

    Download Backup
    +
    + + + +
    +
    @@ -1068,6 +1076,41 @@ .catch(() => {}); })(); + function handleRestoreFileChosen(input) { + const file = input.files && input.files[0]; + input.value = ''; // allow re-choosing the same file later + if (!file) return; + + if (!confirm(`Restore from "${file.name}"? This overwrites your current settings and every course's progress/notes with what's in the backup.`)) { + return; + } + + const status = document.getElementById('restore-backup-status'); + status.style.color = 'var(--text-muted)'; + status.textContent = 'Restoring…'; + + const formData = new FormData(); + formData.append('backup', file); + fetch('/api/backup/restore', { method: 'POST', body: formData }) + .then(r => r.json()) + .then(data => { + if (data.success) { + status.style.color = 'var(--success)'; + const skippedNote = data.skipped && data.skipped.length + ? ` (${data.skipped.length} course${data.skipped.length === 1 ? '' : 's'} skipped - not found in the current library)` + : ''; + status.innerHTML = `${ICON_SVGS.check} Restored ${data.restored} file${data.restored === 1 ? '' : 's'}${skippedNote} - reload to see it take effect`; + } else { + status.style.color = 'var(--error)'; + status.textContent = data.error || 'Restore failed'; + } + }) + .catch(() => { + status.style.color = 'var(--error)'; + status.textContent = 'Could not reach the server'; + }); + } + function loadCourseFromPath() { const input = document.getElementById('manual-course-path'); const status = document.getElementById('manual-course-status');