Fix wrapped-course false positive in picker; add build version display
Sort Unsorted's destination picker still listed a course whose real video content sat two levels deep inside its own "display name" folder (a common release-bundle extraction shape), alongside empty leftover folders from the extraction that neither of the two earlier exclusion checks could safely rule out - an empty folder is indistinguishable from a legitimate freshly-created category. Detect this case by comparing token similarity between a folder's own name and its single course child's name: high overlap means they're naming the same thing (a redundant wrapper), not a deliberate category holding one course. Also bake the build's git commit + build time into the Docker image (git rev-parse at build time, .git deleted again immediately after) so a push's effect on the running container is actually visible - shown on the Settings page and in /health - instead of having to guess whether a webhook rebuild picked up the latest commit. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
+4
-1
@@ -1,5 +1,8 @@
|
|||||||
# Version control
|
# Version control
|
||||||
.git
|
# .git is intentionally NOT excluded - the Dockerfile reads it at build
|
||||||
|
# time (git rev-parse) to bake the running commit into the image as
|
||||||
|
# /app/BUILD_VERSION, then deletes .git itself in that same build step so
|
||||||
|
# it never ends up in the final image.
|
||||||
.gitignore
|
.gitignore
|
||||||
.github
|
.github
|
||||||
|
|
||||||
|
|||||||
+12
-2
@@ -5,8 +5,9 @@ FROM python:3.13.5-slim-bookworm
|
|||||||
WORKDIR /app
|
WORKDIR /app
|
||||||
|
|
||||||
# ffprobe (from ffmpeg) reads each video's duration for the library
|
# ffprobe (from ffmpeg) reads each video's duration for the library
|
||||||
# browser's "how long is this course" display
|
# browser's "how long is this course" display; git is only needed
|
||||||
RUN apt-get update && apt-get install -y --no-install-recommends ffmpeg \
|
# transiently below, to bake this build's commit into the image
|
||||||
|
RUN apt-get update && apt-get install -y --no-install-recommends ffmpeg git \
|
||||||
&& rm -rf /var/lib/apt/lists/*
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
# copy the dependencies file to the working directory
|
# copy the dependencies file to the working directory
|
||||||
@@ -19,6 +20,15 @@ RUN pip install -r requirements.txt
|
|||||||
# copy the content of the local src directory to the working directory
|
# copy the content of the local src directory to the working directory
|
||||||
COPY . .
|
COPY . .
|
||||||
|
|
||||||
|
# Record which commit and when this image was built, so the running app
|
||||||
|
# can show it (Settings page, /health) - a full-image-rebuild webhook
|
||||||
|
# deploy gives no other visible confirmation that a push actually landed
|
||||||
|
# in the running container. .git is removed right after so the image
|
||||||
|
# doesn't carry the repo's full history.
|
||||||
|
RUN (git rev-parse --short HEAD 2>/dev/null || echo unknown) > /app/BUILD_VERSION \
|
||||||
|
&& date -u +"%Y-%m-%d %H:%M UTC" >> /app/BUILD_VERSION \
|
||||||
|
&& rm -rf /app/.git
|
||||||
|
|
||||||
EXPOSE 5000
|
EXPOSE 5000
|
||||||
|
|
||||||
# add healthcheck using Python standard library
|
# add healthcheck using Python standard library
|
||||||
|
|||||||
@@ -44,6 +44,14 @@ triggers a full image **rebuild** from the Dockerfile (confirmed, not just a
|
|||||||
container restart), so Dockerfile changes (e.g. adding `ffmpeg`) take effect
|
container restart), so Dockerfile changes (e.g. adding `ffmpeg`) take effect
|
||||||
on the very next push without any manual step.
|
on the very next push without any manual step.
|
||||||
|
|
||||||
|
**Confirming a deploy landed:** the Dockerfile bakes the exact commit and
|
||||||
|
build time into the image as `/app/BUILD_VERSION` (`git rev-parse` at build
|
||||||
|
time, `.git` itself is deleted again right after so it doesn't ship in the
|
||||||
|
image) - shown at the bottom of the Settings page and in `/health`'s JSON
|
||||||
|
response, so after a push you can check the running container actually
|
||||||
|
picked it up instead of guessing. Reads as `dev` outside Docker (no
|
||||||
|
`BUILD_VERSION` file to read).
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Local development
|
## Local development
|
||||||
@@ -109,12 +117,15 @@ silently stay blank instead of erroring.
|
|||||||
the same picker Manage Library's Move action uses. The picker excludes
|
the same picker Manage Library's Move action uses. The picker excludes
|
||||||
actual course/item folders, not just organizational ones: a leaf folder
|
actual course/item folders, not just organizational ones: a leaf folder
|
||||||
holding a single non-video/audio file (an ebook, an audiobook in a
|
holding a single non-video/audio file (an ebook, an audiobook in a
|
||||||
format this app doesn't play, etc.), and a folder whose whole subtree
|
format this app doesn't play, etc.); a folder whose whole subtree has no
|
||||||
has no video/audio anywhere in it at all (a course's bundled source
|
video/audio anywhere in it at all (a course's bundled source code, a
|
||||||
code, a Python virtualenv, project assets, ...) even though it has
|
Python virtualenv, project assets, ...) even though it has plenty of
|
||||||
plenty of subfolders - both cases Library browsing's own course
|
subfolders; and a release-bundle folder that wraps a single real course
|
||||||
detection wouldn't catch either, since there's no video/audio file to
|
one level down under its own "course display name" folder, alongside
|
||||||
key off. Nothing on disk moves until you review and hit Apply. Matching
|
unrelated (possibly empty) junk siblings at the same level - all three
|
||||||
|
cases Library browsing's own course detection wouldn't catch either,
|
||||||
|
since it only checks one level of nesting for video/audio. Nothing on
|
||||||
|
disk moves until you review and hit Apply. Matching
|
||||||
ignores common noise (e-learning
|
ignores common noise (e-learning
|
||||||
platform names, release/distribution-group tags, dates) via a stopword
|
platform names, release/distribution-group tags, dates) via a stopword
|
||||||
list, and beyond that treats a match against a folder's own deliberate
|
list, and beyond that treats a match against a folder's own deliberate
|
||||||
|
|||||||
+74
-7
@@ -30,6 +30,32 @@ from flask import Flask, render_template, request, jsonify, send_file, redirect,
|
|||||||
app = Flask(__name__)
|
app = Flask(__name__)
|
||||||
app.config['SECRET_KEY'] = 'your-secret-key-change-in-production'
|
app.config['SECRET_KEY'] = 'your-secret-key-change-in-production'
|
||||||
|
|
||||||
|
|
||||||
|
def _load_build_version() -> str:
|
||||||
|
"""
|
||||||
|
Commit + build time baked into the image at build time (see
|
||||||
|
Dockerfile) - lets Settings and /health show which version is
|
||||||
|
actually running, the only way to confirm a git push made it into a
|
||||||
|
rebuilt container, since the webhook does a full image rebuild with
|
||||||
|
no other visible confirmation. Falls back to 'dev' outside Docker,
|
||||||
|
where there's no BUILD_VERSION file (e.g. running via --library-path
|
||||||
|
directly).
|
||||||
|
"""
|
||||||
|
version_file = os.path.join(os.path.dirname(os.path.abspath(__file__)), 'BUILD_VERSION')
|
||||||
|
try:
|
||||||
|
with open(version_file, 'r') as f:
|
||||||
|
lines = [line.strip() for line in f if line.strip()]
|
||||||
|
if len(lines) >= 2:
|
||||||
|
return f'{lines[0]} · built {lines[1]}'
|
||||||
|
if lines:
|
||||||
|
return lines[0]
|
||||||
|
except OSError:
|
||||||
|
pass
|
||||||
|
return 'dev'
|
||||||
|
|
||||||
|
|
||||||
|
BUILD_VERSION = _load_build_version()
|
||||||
|
|
||||||
# Supported file types
|
# Supported file types
|
||||||
VIDEO_EXTENSIONS = {'.mp4', '.mkv', '.avi', '.mov', '.webm', '.m4v', '.flv', '.wmv'}
|
VIDEO_EXTENSIONS = {'.mp4', '.mkv', '.avi', '.mov', '.webm', '.m4v', '.flv', '.wmv'}
|
||||||
AUDIO_EXTENSIONS = {'.mp3', '.wav', '.m4a', '.aac', '.ogg', '.flac'}
|
AUDIO_EXTENSIONS = {'.mp3', '.wav', '.m4a', '.aac', '.ogg', '.flac'}
|
||||||
@@ -1324,6 +1350,41 @@ def _is_media_free_subtree(directory: Path) -> bool:
|
|||||||
return not _subtree_has_any_media(directory)
|
return not _subtree_has_any_media(directory)
|
||||||
|
|
||||||
|
|
||||||
|
def _is_course_wrapper(directory: Path, min_similarity: float = 0.5) -> bool:
|
||||||
|
"""
|
||||||
|
A release-bundle folder whose name is really just a course title with
|
||||||
|
extra decoration (platform name, distribution tag, ...), wrapping a
|
||||||
|
single actual course folder one level down alongside unrelated junk
|
||||||
|
at the same level - e.g. a "Udemy...BOOKWARE-XYZ" folder containing
|
||||||
|
both "Course Title Here/Chapter 1/video.mp4" *and* a pile of bundled
|
||||||
|
source code/empty leftover folders from extraction as siblings.
|
||||||
|
_looks_like_course only checks one level of nesting, so this reads as
|
||||||
|
a bare category with a course buried inside it - and unlike
|
||||||
|
_is_media_free_subtree, the junk siblings here can be perfectly
|
||||||
|
ordinary empty folders (indistinguishable from an intentionally empty,
|
||||||
|
freshly-created category), so that check alone doesn't catch it.
|
||||||
|
Detected by token similarity (the same Jaccard measure
|
||||||
|
_find_duplicate_courses uses) between the directory's own name and its
|
||||||
|
one course child's name: high overlap means they're naming the same
|
||||||
|
thing, not a deliberate parent/child relationship like "IT/AWS"
|
||||||
|
holding an "AWS Certified..." course, whose names barely overlap.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
children = [c for c in directory.iterdir() if c.is_dir() and not c.name.startswith('.')]
|
||||||
|
except (PermissionError, OSError):
|
||||||
|
return False
|
||||||
|
course_children = [c for c in children if _looks_like_course(c)]
|
||||||
|
if len(course_children) != 1:
|
||||||
|
return False
|
||||||
|
dir_tokens = _tokenize(directory.name)
|
||||||
|
course_tokens = _tokenize(course_children[0].name)
|
||||||
|
union_tokens = dir_tokens | course_tokens
|
||||||
|
if not union_tokens:
|
||||||
|
return False
|
||||||
|
similarity = len(dir_tokens & course_tokens) / len(union_tokens)
|
||||||
|
return similarity >= min_similarity
|
||||||
|
|
||||||
|
|
||||||
def _build_category_index() -> List[Dict[str, Any]]:
|
def _build_category_index() -> List[Dict[str, Any]]:
|
||||||
"""
|
"""
|
||||||
Every category/subcategory folder currently in the library (e.g. IT,
|
Every category/subcategory folder currently in the library (e.g. IT,
|
||||||
@@ -1345,12 +1406,16 @@ def _build_category_index() -> List[Dict[str, Any]]:
|
|||||||
Stops recursing into a folder once it reads as a course itself
|
Stops recursing into a folder once it reads as a course itself
|
||||||
(_looks_like_course, the same rule the Library browser uses), an
|
(_looks_like_course, the same rule the Library browser uses), an
|
||||||
unrecognized leaf item (_is_unrecognized_leaf_item - an ebook/
|
unrecognized leaf item (_is_unrecognized_leaf_item - an ebook/
|
||||||
audiobook/misc file that isn't video or audio), or a subtree with no
|
audiobook/misc file that isn't video or audio), a subtree with no
|
||||||
video/audio anywhere in it at all (_is_media_free_subtree - a course's
|
video/audio anywhere in it at all (_is_media_free_subtree - a course's
|
||||||
bundled source code/project files, a Python virtualenv, etc.), so
|
bundled source code/project files, a Python virtualenv, etc.), or a
|
||||||
individual courses and their non-lecture contents never show up as if
|
release-bundle wrapper around a single course one level down
|
||||||
they were categories to file things under. The Unsorted folder itself
|
(_is_course_wrapper - the video content sits under an extra "course
|
||||||
is excluded - it's the source, never a valid destination.
|
display name" folder, one level deeper than _looks_like_course checks,
|
||||||
|
alongside unrelated - possibly empty - junk siblings), so individual
|
||||||
|
courses and their non-lecture contents never show up as if they were
|
||||||
|
categories to file things under. The Unsorted folder itself is
|
||||||
|
excluded - it's the source, never a valid destination.
|
||||||
"""
|
"""
|
||||||
library_root = Path(get_library_root())
|
library_root = Path(get_library_root())
|
||||||
try:
|
try:
|
||||||
@@ -1373,7 +1438,8 @@ def _build_category_index() -> List[Dict[str, Any]]:
|
|||||||
continue
|
continue
|
||||||
except OSError:
|
except OSError:
|
||||||
pass
|
pass
|
||||||
if _looks_like_course(entry) or _is_unrecognized_leaf_item(entry) or _is_media_free_subtree(entry):
|
if (_looks_like_course(entry) or _is_unrecognized_leaf_item(entry)
|
||||||
|
or _is_media_free_subtree(entry) or _is_course_wrapper(entry)):
|
||||||
continue
|
continue
|
||||||
new_parts = path_parts + [entry.name]
|
new_parts = path_parts + [entry.name]
|
||||||
path_tokens: Set[str] = set()
|
path_tokens: Set[str] = set()
|
||||||
@@ -3633,6 +3699,7 @@ def settings_page():
|
|||||||
card_style_choices=['flat', 'elevated', 'bordered'],
|
card_style_choices=['flat', 'elevated', 'bordered'],
|
||||||
corner_radius_choices=['sharp', 'rounded', 'pill'],
|
corner_radius_choices=['sharp', 'rounded', 'pill'],
|
||||||
active_tab='settings',
|
active_tab='settings',
|
||||||
|
build_version=BUILD_VERSION,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
@@ -4365,7 +4432,7 @@ def serve_file(filepath):
|
|||||||
@app.route('/health')
|
@app.route('/health')
|
||||||
def healthcheck():
|
def healthcheck():
|
||||||
"""Healthcheck endpoint for Docker"""
|
"""Healthcheck endpoint for Docker"""
|
||||||
return jsonify({"status": "healthy"}), 200
|
return jsonify({"status": "healthy", "version": BUILD_VERSION}), 200
|
||||||
|
|
||||||
@app.route('/reset_course')
|
@app.route('/reset_course')
|
||||||
def reset_course():
|
def reset_course():
|
||||||
|
|||||||
@@ -466,6 +466,10 @@
|
|||||||
<button class="btn btn-secondary" onclick="resetSettings()">Reset to Defaults</button>
|
<button class="btn btn-secondary" onclick="resetSettings()">Reset to Defaults</button>
|
||||||
<span id="save-status">{{ icons.icon('check', 14) }} Saved</span>
|
<span id="save-status">{{ icons.icon('check', 14) }} Saved</span>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
|
<p style="text-align: center; color: var(--text-muted); font-size: 0.8em; margin-top: 30px;">
|
||||||
|
Version {{ build_version }}
|
||||||
|
</p>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user