Refactor shared script helpers

This commit is contained in:
ajp_anton
2026-06-08 02:55:31 +00:00
parent ae05bc9710
commit b6024d8b65
10 changed files with 855 additions and 356 deletions
+24 -8
View File
@@ -26,11 +26,14 @@ Current features:
- video filenames use the beginning timestamp by subtracting rounded duration from the video metadata timestamp
- Sony-style video sidecars like `C0011.MP4` plus `C0011M01.XML` are moved and renamed with the video
- automatic photo burst/HDR grouping per camera
- optional panorama project (`.pto`) grouping and reference updates
- inferred subsecond metadata for grouped same-second photo bursts
The working directory is determined from the dragged items. A single dragged folder is the working directory. For multiple dragged files/folders, their last common ancestor is the working directory. Automatic burst/HDR group folders are created inside that working directory.
The working directory is determined from the dragged items. A single dragged folder is the working directory. For multiple dragged files/folders, their last common ancestor is the working directory. Automatic burst/HDR and `.pto` group folders are created inside that working directory.
If dragged folders contain files in subdirectories, `photo_metadata.py` asks whether ungrouped files should be moved into the working directory. The default is to leave them in their existing subdirectories.
If dragged folders contain files in subdirectories and grouping is enabled, ungrouped files are moved into the working directory too. This makes the run organize everything into a deterministic flat reset state, with grouped files in their group folders and the remaining files directly in the working directory. If grouping is disabled, the script asks whether to flatten subdirectory files into the working directory.
Panorama project files use the `.pto` extension. When enabled, every media file referenced by a `.pto` file belongs to the same group. If those files already belong to burst/HDR groups, the groups are merged. `.pto` files are moved into the group folder with their referenced files, renamed from the first and last referenced media stems, and their file references are rewritten to the new relative paths.
Reference-clock correction is useful when one photo shows a reliable clock. For example, if `DSC01234.JPG` has camera metadata `2026:06:02 22:10:00`, but the clock in the photo shows `22:13:25`, choose reference mode, enter `DSC01234.JPG` as the source, and enter `22:13:25` as the correct time. The script computes a `+00:03:25` shift and applies that correction to the selected files. If you omit the date in the correct time, the script chooses the date closest to the source timestamp, within 12 hours.
@@ -42,14 +45,14 @@ Fast manual grouping for selected files:
python3 grouping.py path/to/file/or/directory [...]
```
It does not ask questions. It sorts the selected filesystem items by name and moves them into one folder named `[first stem]-[last stem]`.
It does not ask questions. It recursively collects all files inside the dragged files/folders, sorts those files by name, and moves them into one folder named `[first stem]-[last stem]`.
Manual grouping uses filesystem items, not media-file detection:
Manual grouping uses the same working-directory rule as `photo_metadata.py`:
- one file creates `[file stem]-[file stem]` and moves that file into it
- one directory acts as if its contents were given; if it contains only one subdirectory, that rule repeats
- one empty directory does nothing
- multiple files/folders are grouped as selected, without flattening folder contents
- one dragged folder is the working directory
- multiple dragged files/folders use their last common ancestor as the working directory
- the group folder is created inside the working directory
- all collected files are flattened into that group folder
### `view_metadata.py`
@@ -77,6 +80,19 @@ The important choices are:
The old `-G` style shows broad groups like `EXIF`, `File`, or `QuickTime`. That is readable, but it can hide where inside a container a value came from. Family 1 groups, from `-G1`, show more exact locations like `IFD0`, `ExifIFD`, `MakerNotes`, or `Track1`, but sometimes lose the broader context. `-G0:1` combines both, so it is the most useful default for debugging metadata without changing which normal tags are extracted. The organization option changes labels, not the underlying metadata extraction; `-a` and `-u` are the parts that affect whether duplicate or unknown printed tags are included.
### `copy_metadata.py`
Copy meaningful metadata from one file to another:
```bash
python3 copy_metadata.py
python3 copy_metadata.py source.jpg destination.jpg
```
If run without files, it asks for source and destination paths. If exactly two files are dragged onto it, it asks which one is the source. Any other number of dragged files is an error.
The script copies normal metadata with ExifTool `-TagsFromFile`, while excluding file/system tags, composite tags, image dimensions, pixel dimensions, and resolution fields that should remain specific to the destination file.
## Requirements
- Python 3.10 or newer
+144
View File
@@ -0,0 +1,144 @@
#!/usr/bin/env python3
"""
Copy meaningful metadata from one file to another.
Run without files to type source/destination paths, or drag exactly two files
onto the script and choose which is the source.
"""
from __future__ import annotations
import subprocess
import sys
from pathlib import Path
from tools.console import pause_if_interactive, prompt_input
from tools.exiftool import require_exiftool
EXCLUDED_COPY_TAGS = [
"--File:all",
"--ExifTool:all",
"--Composite:all",
"--SourceFile",
"--Directory",
"--FileName",
"--FileSize",
"--FileModifyDate",
"--FileAccessDate",
"--FileInodeChangeDate",
"--FilePermissions",
"--FileType",
"--FileTypeExtension",
"--MIMEType",
"--ImageWidth",
"--ImageHeight",
"--ExifImageWidth",
"--ExifImageHeight",
"--PixelXDimension",
"--PixelYDimension",
"--RelatedImageWidth",
"--RelatedImageHeight",
"--ImageSize",
"--Megapixels",
"--XResolution",
"--YResolution",
"--ResolutionUnit",
]
def prompt_path(prompt: str) -> Path:
while True:
value = prompt_input(prompt).strip().strip('"')
path = Path(value).expanduser()
if path.is_file():
return path.resolve()
print("File not found. Enter a single file path.")
def choose_from_two(path_a: Path, path_b: Path) -> tuple[Path, Path] | None:
print("Choose metadata source:")
print(f"1: {path_a}")
print(f"2: {path_b}")
while True:
answer = prompt_input("Source file [1/2, Enter=1]: ").strip()
if answer == "" or answer == "1":
return path_a, path_b
if answer == "2":
return path_b, path_a
print("Please choose 1 or 2.")
def resolve_files(argv: list[str]) -> tuple[Path, Path] | None:
args = argv[1:]
if not args:
source = prompt_path("Source file: ")
destination = prompt_path("Destination file: ")
return source, destination
if len(args) != 2:
print("Error: provide exactly two files, or run without files to type paths.")
return None
paths = [Path(arg).expanduser().resolve() for arg in args]
missing = [path for path in paths if not path.is_file()]
if missing:
for path in missing:
print(f"File not found: {path}")
return None
return choose_from_two(paths[0], paths[1])
def copy_metadata(exiftool: str, source: Path, destination: Path) -> int:
print(f"Source: {source}")
print(f"Destination: {destination}", flush=True)
command = [
exiftool,
"-overwrite_original",
"-TagsFromFile",
str(source),
"-all:all",
*EXCLUDED_COPY_TAGS,
"-charset",
"filename=UTF8",
str(destination),
]
return subprocess.run(command).returncode
def run(argv: list[str]) -> int:
try:
exiftool = require_exiftool()
except RuntimeError as exc:
print(exc)
return 1
try:
resolved = resolve_files(argv)
if resolved is None:
return 1
except RuntimeError as exc:
print(exc)
return 1
source, destination = resolved
if source == destination:
print("Source and destination must be different files.")
return 1
return_code = copy_metadata(exiftool, source, destination)
if return_code == 0:
print("Metadata copy complete.")
return return_code
def main(argv: list[str]) -> int:
try:
return run(argv)
finally:
pause_if_interactive()
if __name__ == "__main__":
raise SystemExit(main(sys.argv))
+22 -99
View File
@@ -6,137 +6,60 @@ Drag files or directories onto this script, or run:
python3 grouping.py path/to/file/or/directory [...]
Files are sorted by their current path and moved into one folder named
[first stem]-[last stem]. Multiple files/folders are grouped as selected.
[first stem]-[last stem] inside the working directory.
"""
from __future__ import annotations
import os
import re
import shutil
import sys
from pathlib import Path
ORDERED_TIMESTAMP_STEM_RE = re.compile(r"^(?P<timestamp>\d{8}_\d{6})-[1-9]\d*$")
from tools.console import pause_if_interactive
from tools.filenames import group_stem
from tools.filesystem import collect_files, determine_working_dir, unique_path
def group_stem(path: Path) -> str:
stem = path.stem
match = ORDERED_TIMESTAMP_STEM_RE.match(stem)
if match:
return match.group("timestamp")
return stem
def collect_group_files(args: list[str]) -> list[Path]:
return collect_files(args, sort_by_name=True)
def unique_path(path: Path) -> Path:
if not path.exists():
return path
for index in range(1, 10000):
candidate = path.with_name(f"{path.stem}-{index}{path.suffix}")
if not candidate.exists():
return candidate
raise RuntimeError(f"Could not find an available name for {path}")
def existing_input_paths(args: list[str]) -> list[Path]:
paths: list[Path] = []
for arg in args:
path = Path(arg).expanduser()
if path.exists():
paths.append(path.resolve())
else:
print(f"Skipping missing path: {arg}")
return paths
def resolve_single_directory(path: Path) -> list[Path]:
current = path
while current.is_dir():
children = sorted(current.iterdir(), key=lambda item: item.name.lower())
if not children:
return []
if len(children) == 1 and children[0].is_dir():
current = children[0]
continue
return [child.resolve() for child in children]
return [current.resolve()]
def resolve_group_items(args: list[str]) -> list[Path]:
paths = existing_input_paths(args)
if len(paths) == 1:
path = paths[0]
if path.is_dir():
return resolve_single_directory(path)
return [path]
return sorted(paths, key=lambda item: item.name.lower())
def working_dir_for_items(items: list[Path]) -> Path:
if len(items) == 1:
item = items[0]
return item if item.is_dir() else item.parent
parents = {item.parent for item in items}
if len(parents) == 1:
return next(iter(parents))
try:
common = Path(os.path.commonpath([str(item) for item in items]))
except ValueError as exc:
raise RuntimeError("Input paths do not have a common ancestor.") from exc
if str(common) == common.anchor:
raise RuntimeError("Input paths only share the filesystem root; choose items closer together.")
return common
def group_items(items: list[Path]) -> Path | None:
if not items:
def group_files(files: list[Path], working_dir: Path) -> Path | None:
if not files:
return None
sorted_items = sorted(items, key=lambda item: item.name.lower())
first = group_stem(sorted_items[0])
last = group_stem(sorted_items[-1])
target_dir = unique_path(working_dir_for_items(sorted_items) / f"{first}-{last}")
target_dir.mkdir(parents=True)
sorted_files = sorted(files, key=lambda item: item.name.lower())
first = group_stem(sorted_files[0])
last = group_stem(sorted_files[-1])
target_dir = working_dir / f"{first}-{last}"
target_dir.mkdir(parents=True, exist_ok=True)
for source in sorted_items:
for source in sorted_files:
destination = unique_path(target_dir / source.name)
shutil.move(str(source), str(destination))
return target_dir
def pause_if_interactive() -> None:
if sys.stdin.isatty():
try:
input("\nPress Enter to close...")
except EOFError:
pass
def run(argv: list[str]) -> int:
if len(argv) < 2:
print("Usage: python3 grouping.py path/to/file/or/directory [...]")
return 2
items = resolve_group_items(argv[1:])
if not items:
print("No items to group.")
return 0
try:
target_dir = group_items(items)
working_dir = determine_working_dir(argv[1:])
except RuntimeError as exc:
print(exc)
return 1
files = collect_group_files(argv[1:])
if not files:
print("No files to group.")
return 0
target_dir = group_files(files, working_dir)
if target_dir is not None:
print(f"Moved {len(items)} item(s) into {target_dir}")
print(f"Moved {len(files)} file(s) into {target_dir}")
return 0
+408 -227
View File
@@ -11,7 +11,6 @@ place only after confirmation.
from __future__ import annotations
import json
import math
import os
import re
@@ -23,9 +22,25 @@ from collections import defaultdict
from dataclasses import dataclass, field
from datetime import datetime, timedelta, timezone
from pathlib import Path
from shutil import which
from typing import Iterable
from tools.console import pause_if_interactive, prompt_input
from tools.exiftool import require_exiftool, run_exiftool_json, run_exiftool_write
from tools.filenames import (
format_filename_stem,
group_stem_from_name,
naive_wall_time,
normalized_extension,
)
from tools.filesystem import (
cleanup_roots_from_args,
collect_files,
determine_working_dir,
display_path_from_working_dir,
remove_empty_dirs,
unique_existing_target,
)
IMAGE_EXTS = {".jpg", ".jpeg", ".heic", ".arw"}
VIDEO_EXTS = {".mp4", ".mov", ".mts"}
@@ -85,6 +100,12 @@ TZ_RE = re.compile(r"^(?P<sign>[+-])(?P<h>\d{1,2})(?::?(?P<m>\d{2}))?$")
OFFSET_RE = re.compile(r"^([+-])(?:(\d+):)?(\d{1,2})(?::(\d{2}))?$")
TOKEN_SHIFT_RE = re.compile(r"(\d+|[a-zA-Z]+)")
ORDERED_TIMESTAMP_STEM_RE = re.compile(r"^(?P<timestamp>\d{8}_\d{6})-[1-9]\d*$")
TIMESTAMP_RANGE_STEM_RE = re.compile(
r"^(?P<start>\d{8}_\d{6})-(?P<end>\d{8}_\d{6})(?P<suffix>_.*)?$"
)
CAMERA_RANGE_STEM_RE = re.compile(
r"^(?P<prefix>[A-Za-z]*)(?P<first>\d+)-(?P=prefix)(?P<last>\d+)(?P<suffix>_.*)$"
)
@dataclass
@@ -140,6 +161,22 @@ class MediaRecord:
return self.kind == "video"
@dataclass
class PtoPlan:
path: Path
target_path: Path
referenced_records: list[MediaRecord]
updated_text: str
replacements: list[tuple[str, str]]
@dataclass
class PtoReference:
path: Path
original_text: str
records: list[MediaRecord]
@dataclass
class UserChoices:
working_dir: Path
@@ -153,6 +190,7 @@ class UserChoices:
rename_files: bool
group_photos: bool
move_to_working_dir: bool
process_pto_files: bool
apply_filename_timestamps: bool
@@ -169,13 +207,6 @@ def ask_yes_no(prompt: str, default: bool) -> bool:
print("Please answer y or n.")
def prompt_input(prompt: str) -> str:
try:
return input(prompt)
except EOFError as exc:
raise RuntimeError("Interactive input ended before choices were complete.") from exc
def parse_timezone_offset(value: str) -> timezone:
match = TZ_RE.match(value.strip())
if not match:
@@ -330,118 +361,12 @@ def parse_timeshift(value: str) -> timedelta:
return sign * timedelta(seconds=seconds)
def naive_wall_time(dt: datetime) -> datetime:
return dt.replace(tzinfo=None)
def collect_media_files(args: list[str]) -> list[Path]:
files: list[Path] = []
seen: set[Path] = set()
for arg in args:
path = Path(arg).expanduser()
if path.is_file():
candidates = [path] if path.suffix.lower() in SUPPORTED_EXTS else []
elif path.is_dir():
candidates = [
child
for child in path.rglob("*")
if child.is_file() and child.suffix.lower() in SUPPORTED_EXTS
]
else:
print(f"Skipping missing path: {arg}")
continue
for candidate in candidates:
resolved = candidate.resolve()
if resolved not in seen:
seen.add(resolved)
files.append(resolved)
return sorted(files, key=lambda p: str(p).lower())
return collect_files(args, allowed_exts=SUPPORTED_EXTS, print_missing=True)
def existing_input_paths(args: list[str]) -> list[Path]:
paths: list[Path] = []
for arg in args:
path = Path(arg).expanduser()
if path.exists():
paths.append(path.resolve())
return paths
def determine_working_dir(args: list[str]) -> Path:
paths = existing_input_paths(args)
if not paths:
raise RuntimeError("No existing input paths were provided.")
if len(paths) == 1:
path = paths[0]
return path if path.is_dir() else path.parent
try:
common = Path(os.path.commonpath([str(path) for path in paths]))
except ValueError as exc:
raise RuntimeError("Input paths do not have a common ancestor.") from exc
if str(common) == common.anchor:
raise RuntimeError("Input paths only share the filesystem root; choose items closer together.")
return common
def cleanup_roots_from_args(args: list[str]) -> list[Path]:
roots: list[Path] = []
seen: set[Path] = set()
for arg in args:
path = Path(arg).expanduser()
if path.is_dir():
resolved = path.resolve()
if resolved not in seen:
seen.add(resolved)
roots.append(resolved)
return roots
def require_exiftool() -> str:
executable = which("exiftool")
if executable:
return executable
raise RuntimeError(
"exiftool was not found on PATH. Install ExifTool and make sure the "
"'exiftool' command works before running this script."
)
def run_exiftool_json(executable: str, paths: list[Path]) -> list[dict]:
results: list[dict] = []
for start in range(0, len(paths), 80):
chunk = paths[start : start + 80]
command = [
executable,
"-j",
"-G",
"-api",
"QuickTimeUTC=1",
"-charset",
"filename=UTF8",
*[str(path) for path in chunk],
]
completed = subprocess.run(command, check=True, capture_output=True, text=True)
results.extend(json.loads(completed.stdout or "[]"))
return results
def run_exiftool_write(executable: str, path: Path, args: list[str]) -> None:
if not args:
return
command = [
executable,
"-overwrite_original",
"-charset",
"filename=UTF8",
*args,
str(path),
]
subprocess.run(command, check=True)
def collect_pto_files(args: list[str]) -> list[Path]:
return collect_files(args, allowed_exts={".pto"})
def first_tag(metadata: dict, tags: Iterable[str]) -> object | None:
@@ -793,7 +718,7 @@ def prompt_artist_action() -> tuple[str, str | None]:
print("Please choose Enter, s, or c.")
def prompt_choices(records: list[MediaRecord], working_dir: Path) -> UserChoices:
def prompt_choices(records: list[MediaRecord], working_dir: Path, has_pto_files: bool) -> UserChoices:
missing_time = [record for record in records if record.original_time is None]
apply_filename_timestamps = False
if missing_time:
@@ -818,13 +743,24 @@ def prompt_choices(records: list[MediaRecord], working_dir: Path) -> UserChoices
if sum(1 for record in records if record.is_image and record.adjusted_time) >= 2:
group_photos = ask_yes_no("Create automatic burst/HDR groups?", default=True)
move_to_working_dir = False
if any(record.path.parent.resolve() != working_dir for record in records):
move_to_working_dir = ask_yes_no(
f"Move ungrouped files from subdirectories into the working directory ({working_dir})?",
default=False,
process_pto_files = False
if has_pto_files:
process_pto_files = ask_yes_no(
"Use panorama project files (.pto) for grouping and update their references?",
default=True,
)
move_to_working_dir = False
has_subdir_media = any(record.path.parent.resolve() != working_dir for record in records)
if has_subdir_media:
if group_photos or process_pto_files:
move_to_working_dir = True
else:
move_to_working_dir = ask_yes_no(
f"Move files from subdirectories into the working directory ({working_dir})?",
default=False,
)
return UserChoices(
working_dir=working_dir,
time_offset=time_offset,
@@ -837,6 +773,7 @@ def prompt_choices(records: list[MediaRecord], working_dir: Path) -> UserChoices
rename_files=rename_files,
group_photos=group_photos,
move_to_working_dir=move_to_working_dir,
process_pto_files=process_pto_files,
apply_filename_timestamps=apply_filename_timestamps,
)
@@ -859,6 +796,16 @@ def photo_sort_key(record: MediaRecord) -> tuple:
)
def group_sort_key(record: MediaRecord) -> tuple:
timestamp = naive_wall_time(record.adjusted_time) if record.adjusted_time is not None else datetime.max
return (
timestamp,
record.subsec if record.subsec is not None else -1,
record.sequence if record.sequence is not None else -1,
str(record.path).lower(),
)
def detect_photo_groups(records: list[MediaRecord]) -> list[list[MediaRecord]]:
groups: list[list[MediaRecord]] = []
by_camera: dict[str, list[MediaRecord]] = defaultdict(list)
@@ -955,7 +902,8 @@ def infer_group_subseconds(groups: list[list[MediaRecord]]) -> None:
for group in groups:
by_second: dict[datetime, list[MediaRecord]] = defaultdict(list)
for record in group:
assert record.adjusted_time is not None
if not record.is_image or record.adjusted_time is None:
continue
second = naive_wall_time(record.adjusted_time).replace(microsecond=0)
by_second[second].append(record)
@@ -982,20 +930,65 @@ def infer_group_subseconds(groups: list[list[MediaRecord]]) -> None:
record.inferred_subsec = int((start_slot + item_index) * 1000 / fps)
def format_filename_stem(dt: datetime) -> str:
return naive_wall_time(dt).strftime("%Y%m%d_%H%M%S")
def filename_range_stem(original_stem: str, base_dt: datetime) -> str | None:
match = TIMESTAMP_RANGE_STEM_RE.match(original_stem)
if not match:
return None
try:
old_start = parse_datetime(match.group("start"))
old_end = parse_datetime(match.group("end"))
except ValueError:
return None
interval = old_end - old_start
new_start = naive_wall_time(base_dt)
new_end = new_start + interval
suffix = match.group("suffix") or ""
return f"{format_filename_stem(new_start)}-{format_filename_stem(new_end)}{suffix}"
def normalized_extension(path: Path) -> str:
return path.suffix.lower()
def unique_records_by_stem(records: list[MediaRecord]) -> dict[str, MediaRecord]:
buckets: dict[str, list[MediaRecord]] = defaultdict(list)
for record in records:
buckets[record.path.stem.lower()].append(record)
return {stem: bucket[0] for stem, bucket in buckets.items() if len(bucket) == 1}
def group_stem_from_name(name: str) -> str:
stem = Path(name).stem
match = ORDERED_TIMESTAMP_STEM_RE.match(stem)
if match:
return match.group("timestamp")
return stem
def camera_range_stem(record: MediaRecord, records_by_stem: dict[str, MediaRecord]) -> str | None:
match = CAMERA_RANGE_STEM_RE.match(record.path.stem)
if not match:
return None
first_number = int(match.group("first"))
last_number = int(match.group("last"))
if last_number <= first_number:
return None
prefix = match.group("prefix")
first_record = records_by_stem.get(f"{prefix}{match.group('first')}".lower())
last_record = records_by_stem.get(f"{prefix}{match.group('last')}".lower())
first_time = first_record.adjusted_time if first_record else None
last_time = last_record.adjusted_time if last_record else None
own_time = record.adjusted_time
if own_time is not None:
start_time = own_time
end_time = last_time or own_time
else:
start_time = first_time or last_time
end_time = last_time or first_time
if start_time is None or end_time is None:
return None
suffix = match.group("suffix")
return f"{format_filename_stem(start_time)}-{format_filename_stem(end_time)}{suffix}"
def filename_stem_for_dt(record: MediaRecord, dt: datetime) -> str:
range_stem = filename_range_stem(record.path.stem, dt)
if range_stem is not None:
return range_stem
return format_filename_stem(dt)
def plan_names(records: list[MediaRecord], choices: UserChoices) -> None:
@@ -1004,9 +997,14 @@ def plan_names(records: list[MediaRecord], choices: UserChoices) -> None:
record.final_name = record.path.name
return
records_by_stem = unique_records_by_stem(records)
for record in records:
if record.adjusted_time is None:
record.rename_skip_reason = "missing timestamp"
range_stem = camera_range_stem(record, records_by_stem)
if range_stem is not None:
record.new_stem = range_stem
else:
record.rename_skip_reason = "missing timestamp"
continue
if record.is_video:
if record.duration_seconds is None and not record.parsed_from_filename:
@@ -1025,9 +1023,11 @@ def plan_names(records: list[MediaRecord], choices: UserChoices) -> None:
if begin.tzinfo is None:
begin = begin.replace(tzinfo=timezone.utc)
local_begin = begin.astimezone(choices.video_filename_timezone)
record.new_stem = format_filename_stem(local_begin)
record.new_stem = filename_stem_for_dt(record, local_begin)
else:
record.new_stem = format_filename_stem(record.adjusted_time)
record.new_stem = camera_range_stem(record, records_by_stem) or filename_stem_for_dt(
record, record.adjusted_time
)
assign_unique_names(records)
@@ -1055,15 +1055,64 @@ def assign_unique_names(records: list[MediaRecord]) -> None:
record.final_name = f"{stem}-{index}{ext}"
def merge_record_groups(
records: list[MediaRecord], group_candidates: list[list[MediaRecord]]
) -> list[list[MediaRecord]]:
record_indexes = {record.path.resolve(): index for index, record in enumerate(records)}
parents = list(range(len(records)))
grouped_indexes: set[int] = set()
def find(index: int) -> int:
while parents[index] != index:
parents[index] = parents[parents[index]]
index = parents[index]
return index
def union(left: int, right: int) -> None:
left_root = find(left)
right_root = find(right)
if left_root != right_root:
parents[right_root] = left_root
for candidate in group_candidates:
indexes = [
record_indexes[record.path.resolve()]
for record in candidate
if record.path.resolve() in record_indexes
]
if not indexes:
continue
grouped_indexes.update(indexes)
first = indexes[0]
for index in indexes[1:]:
union(first, index)
merged: dict[int, list[MediaRecord]] = defaultdict(list)
for index in grouped_indexes:
merged[find(index)].append(records[index])
return sorted(
[sorted(group, key=group_sort_key) for group in merged.values()],
key=lambda group: group_sort_key(group[0]),
)
def assign_groups(records: list[MediaRecord], groups: list[list[MediaRecord]]) -> None:
for record in records:
record.group_id = None
record.group_name = None
for index, group in enumerate(groups, start=1):
ordered = sorted(group, key=photo_sort_key)
ordered = sorted(group, key=group_sort_key)
for record in ordered:
record.group_id = index
def unique_group_name(base_name: str, working_dir: Path, used_names: set[str]) -> str:
if base_name not in used_names and not (working_dir / base_name).exists():
if (working_dir / base_name).exists():
used_names.add(base_name)
return base_name
if base_name not in used_names:
used_names.add(base_name)
return base_name
@@ -1079,7 +1128,7 @@ def unique_group_name(base_name: str, working_dir: Path, used_names: set[str]) -
def assign_group_names(groups: list[list[MediaRecord]], working_dir: Path) -> None:
used_names: set[str] = set()
for group in groups:
ordered = sorted(group, key=photo_sort_key)
ordered = sorted(group, key=group_sort_key)
first = group_stem_from_name(ordered[0].final_name or ordered[0].path.name)
last = group_stem_from_name(ordered[-1].final_name or ordered[-1].path.name)
group_name = unique_group_name(f"{first}-{last}", working_dir, used_names)
@@ -1103,6 +1152,140 @@ def plan_targets(records: list[MediaRecord], choices: UserChoices) -> None:
record.target_sidecar = target_sidecar
def normalize_reference(value: str) -> str:
return value.replace("\\", "/")
def pto_target_for_base(
path: Path, target_dir: Path, base_name: str, used_targets: set[Path]
) -> Path:
desired = target_dir / f"{base_name}.pto"
if desired.resolve() == path.resolve() or (not desired.exists() and desired not in used_targets):
used_targets.add(desired)
return desired
for index in range(1, 10000):
candidate = target_dir / f"{base_name}-{index}.pto"
if candidate.resolve() == path.resolve() or (
not candidate.exists() and candidate not in used_targets
):
used_targets.add(candidate)
return candidate
raise RuntimeError(f"Could not find an available .pto name for {base_name}")
def pto_reference_map(pto_path: Path, records: list[MediaRecord]) -> dict[str, MediaRecord]:
pto_dir = pto_path.parent
basename_counts: dict[str, int] = defaultdict(int)
for record in records:
basename_counts[record.path.name] += 1
mapping: dict[str, MediaRecord] = {}
for record in records:
old_abs = record.path.resolve()
candidates = {
normalize_reference(str(old_abs)),
normalize_reference(os.path.relpath(old_abs, pto_dir)),
}
if basename_counts[record.path.name] == 1:
candidates.add(record.path.name)
for candidate in candidates:
mapping[candidate] = record
return mapping
PTO_QUOTED_VALUE_RE = re.compile(r"([\"'])(.*?)(\1)")
def read_pto_text(path: Path) -> str:
try:
return path.read_text(encoding="utf-8")
except UnicodeDecodeError:
return path.read_text(encoding="utf-8", errors="replace")
def read_pto_references(pto_files: list[Path], records: list[MediaRecord]) -> list[PtoReference]:
references: list[PtoReference] = []
for pto_path in pto_files:
original_text = read_pto_text(pto_path)
mapping = pto_reference_map(pto_path, records)
if not mapping:
continue
referenced: list[MediaRecord] = []
for match in PTO_QUOTED_VALUE_RE.finditer(original_text):
record = mapping.get(normalize_reference(match.group(2)))
if record is not None and record not in referenced:
referenced.append(record)
if referenced:
references.append(PtoReference(path=pto_path, original_text=original_text, records=referenced))
return references
def replacement_for_pto_value(
value: str, mapping: dict[str, MediaRecord], pto_target_dir: Path
) -> tuple[str, MediaRecord] | None:
normalized = normalize_reference(value)
record = mapping.get(normalized)
if record is None:
return None
assert record.target_path is not None
replacement = normalize_reference(os.path.relpath(record.target_path.resolve(), pto_target_dir))
return replacement, record
def plan_pto_updates(pto_references: list[PtoReference], records: list[MediaRecord]) -> list[PtoPlan]:
plans: list[PtoPlan] = []
used_targets: set[Path] = set()
for pto_reference in pto_references:
pto_path = pto_reference.path
original_text = pto_reference.original_text
referenced = sorted(pto_reference.records, key=group_sort_key)
if not referenced:
continue
target_dir = pto_path.parent
if referenced[0].group_name:
target_dir = referenced[0].target_path.parent if referenced[0].target_path else target_dir
mapping = pto_reference_map(pto_path, records)
if not mapping:
continue
replacements: list[tuple[str, str]] = []
def replace_match(match: re.Match) -> str:
quote = match.group(1)
value = match.group(2)
replacement = replacement_for_pto_value(value, mapping, target_dir)
if replacement is None:
return match.group(0)
new_value, record = replacement
if value != new_value:
replacements.append((value, new_value))
return f"{quote}{new_value}{quote}"
updated_text = PTO_QUOTED_VALUE_RE.sub(replace_match, original_text)
first = group_stem_from_name(referenced[0].target_path.name)
last = group_stem_from_name(referenced[-1].target_path.name)
target_path = pto_target_for_base(pto_path, target_dir, f"{first}-{last}", used_targets)
if updated_text == original_text and target_path.resolve() == pto_path.resolve():
continue
plans.append(
PtoPlan(
path=pto_path,
target_path=target_path,
referenced_records=referenced,
updated_text=updated_text,
replacements=replacements,
)
)
return plans
def build_write_plan(record: MediaRecord, choices: UserChoices) -> list[TagWrite]:
writes: list[TagWrite] = []
if choices.artist_action == "set":
@@ -1227,31 +1410,45 @@ def build_write_args(record: MediaRecord, choices: UserChoices) -> list[str]:
return [write.arg for write in record.write_plan if write.will_write]
def display_path_from_working_dir(path: Path, working_dir: Path) -> str:
try:
relative = path.resolve().relative_to(working_dir.resolve())
except ValueError:
return str(path)
return "." if str(relative) == "." else str(relative)
def path_display(path: Path, choices: UserChoices, use_relative: bool) -> str:
if use_relative:
return display_path_from_working_dir(path, choices.working_dir)
return str(path)
def metadata_preview_text(write: TagWrite) -> str:
current = write.current_value if write.current_value is not None else "<missing>"
if write.will_write:
return f"metadata {write.label}: {current} -> {write.new_value}"
return f"metadata {write.label}: {current} -> {write.new_value} (unchanged, skipped)"
def directory_display(path: Path, choices: UserChoices) -> str:
text = path_display(path, choices, use_relative=True)
if text != "." and not text.endswith(os.sep):
return f"{text}{os.sep}"
return text
def print_preview(records: list[MediaRecord], choices: UserChoices) -> None:
def move_rename_preview(
source: Path,
target: Path,
choices: UserChoices,
source_label: str | None = None,
) -> str | None:
is_moving = source.parent.resolve() != target.parent.resolve()
is_renaming = source.name != target.name
source_text = source_label or path_display(source, choices, use_relative=is_moving)
if is_moving and is_renaming:
target_text = path_display(target, choices, use_relative=True)
return f"move+rename {source_text} -> {target_text}"
if is_moving:
target_text = directory_display(target.parent, choices)
return f"move {source_text} -> {target_text}"
if is_renaming:
return f"rename {source_label or source.name} -> {target.name}"
return None
def print_preview(records: list[MediaRecord], choices: UserChoices, pto_plans: list[PtoPlan]) -> None:
print("\nPreview")
print(f"Working directory: {choices.working_dir}")
print(f"Move ungrouped files into working directory: {'yes' if choices.move_to_working_dir else 'no'}")
print(f"Panorama project files (.pto): {'group/update' if choices.process_pto_files else 'ignore'}")
print(f"Time shift: {format_timedelta(choices.time_offset) if choices.time_offset else 'none'}")
print(
"Photo timezone write: "
@@ -1276,27 +1473,12 @@ def print_preview(records: list[MediaRecord], choices: UserChoices) -> None:
else:
print("Artist/author: unchanged")
rows = []
for record in sorted(records, key=lambda item: str(item.path).lower()):
target = record.target_path or record.path
operation = []
source_parent = record.path.parent.resolve()
target_parent = target.parent.resolve()
source_name = record.path.name
target_name = target.name
is_moving = source_parent != target_parent
is_renaming = source_name != target_name
if is_moving:
source_display = path_display(record.path, choices, use_relative=True)
if is_renaming:
target_display = path_display(target, choices, use_relative=True)
else:
target_display = path_display(target.parent, choices, use_relative=True)
operation.append(f"move {source_display} -> {target_display}")
if is_renaming:
operation.append(f"rename {source_name} -> {target_name}")
for write in record.write_plan:
operation.append(metadata_preview_text(write))
operation_text = move_rename_preview(record.path, target, choices)
if operation_text:
operation.append(operation_text)
if record.rename_skip_reason:
operation.append(f"rename skipped: {record.rename_skip_reason}")
for warning in record.warnings:
@@ -1305,48 +1487,27 @@ def print_preview(records: list[MediaRecord], choices: UserChoices) -> None:
sidecar_source = path_display(record.sidecar.path, choices, use_relative=True)
sidecar_target = path_display(record.target_sidecar, choices, use_relative=True)
operation.append(f"sidecar move {sidecar_source} -> {sidecar_target}")
source_text = path_display(record.path, choices, use_relative=is_moving)
rows.append((source_text, "; ".join(operation) or "no change"))
if operation:
print("; ".join(operation))
path_width = min(max((len(row[0]) for row in rows), default=10), 70)
for source, operation in rows:
print(f"{source:<{path_width}} {operation}")
if pto_plans:
print("\nPTO updates:")
for plan in pto_plans:
source = path_display(plan.path, choices, use_relative=True)
operation = move_rename_preview(plan.path, plan.target_path, choices, source_label=source)
print(operation or source)
for old_ref, new_ref in plan.replacements:
print(f" {old_ref} -> {new_ref}")
def unique_existing_target(path: Path, planned_sources: set[Path]) -> Path:
if not path.exists() or path.resolve() in planned_sources:
return path
for index in range(1, 10000):
candidate = path.with_name(f"{path.stem}-{index}{path.suffix}")
if not candidate.exists() or candidate.resolve() in planned_sources:
return candidate
raise RuntimeError(f"Could not find available target name for {path}")
def remove_empty_dirs(cleanup_roots: list[Path], keep_dirs: set[Path]) -> list[Path]:
removed: list[Path] = []
candidates: set[Path] = set()
for root in cleanup_roots:
if not root.exists() or not root.is_dir():
continue
for directory, subdirs, _files in os.walk(root, topdown=False):
path = Path(directory).resolve()
candidates.add(path)
for subdir in subdirs:
child = (Path(directory) / subdir).resolve()
if child.exists() and child.is_dir():
candidates.add(child)
for directory in sorted(candidates, key=lambda path: len(path.parts), reverse=True):
if directory in keep_dirs:
continue
try:
directory.rmdir()
removed.append(directory)
except OSError:
pass
return removed
def has_planned_media_change(record: MediaRecord, write_args: list[str]) -> bool:
if write_args:
return True
if record.target_path and record.target_path.resolve() != record.path.resolve():
return True
if record.sidecar and record.target_sidecar and record.sidecar.path.exists():
return True
return False
def apply_changes(
@@ -1356,8 +1517,10 @@ def apply_changes(
planned_sources = {record.path.resolve() for record in records}
for index, record in enumerate(records, start=1):
changed = False
try:
write_args = build_write_args(record, choices)
changed = has_planned_media_change(record, write_args)
run_exiftool_write(executable, record.path, write_args)
if record.target_path and record.target_path != record.path:
@@ -1373,11 +1536,26 @@ def apply_changes(
shutil.move(str(record.sidecar.path), str(target_sidecar))
except (OSError, subprocess.CalledProcessError, RuntimeError) as exc:
failures.append(f"{record.path}: {exc}")
print(f"Processed {index} of {len(records)}")
action = "Processed" if changed else "Skipped"
print(f"{action} {index} of {len(records)}")
return failures
def apply_pto_changes(pto_plans: list[PtoPlan]) -> list[str]:
failures: list[str] = []
for plan in pto_plans:
try:
plan.path.write_text(plan.updated_text, encoding="utf-8")
if plan.target_path.resolve() != plan.path.resolve():
target = unique_existing_target(plan.target_path, {plan.path.resolve()})
target.parent.mkdir(parents=True, exist_ok=True)
shutil.move(str(plan.path), str(target))
except OSError as exc:
failures.append(f"{plan.path}: {exc}")
return failures
def print_removed_dirs(removed_dirs: list[Path]) -> None:
if not removed_dirs:
return
@@ -1386,12 +1564,19 @@ def print_removed_dirs(removed_dirs: list[Path]) -> None:
print(f" - {directory}")
def prepare_plan(records: list[MediaRecord], choices: UserChoices) -> list[list[MediaRecord]]:
def prepare_plan(
records: list[MediaRecord], choices: UserChoices, pto_references: list[PtoReference]
) -> list[list[MediaRecord]]:
apply_time_offset(records, choices.time_offset)
groups: list[list[MediaRecord]] = []
group_candidates: list[list[MediaRecord]] = []
if choices.group_photos:
groups = detect_photo_groups(records)
group_candidates.extend(detect_photo_groups(records))
if choices.process_pto_files:
group_candidates.extend(reference.records for reference in pto_references)
groups = merge_record_groups(records, group_candidates)
if groups:
assign_groups(records, groups)
infer_group_subseconds(groups)
@@ -1404,14 +1589,6 @@ def prepare_plan(records: list[MediaRecord], choices: UserChoices) -> list[list[
return groups
def pause_if_interactive() -> None:
if sys.stdin.isatty():
try:
input("\nPress Enter to close...")
except EOFError:
pass
def run(argv: list[str]) -> int:
if len(argv) < 2:
print("Usage: python3 photo_metadata.py path/to/file/or/directory [...]")
@@ -1425,6 +1602,7 @@ def run(argv: list[str]) -> int:
cleanup_roots = cleanup_roots_from_args(argv[1:])
files = collect_media_files(argv[1:])
pto_files = collect_pto_files(argv[1:])
if not files:
print("No supported media files found.")
return 1
@@ -1444,13 +1622,15 @@ def run(argv: list[str]) -> int:
records = build_records(files, metadata)
try:
choices = prompt_choices(records, working_dir)
choices = prompt_choices(records, working_dir, has_pto_files=bool(pto_files))
except RuntimeError as exc:
print(exc)
return 1
prepare_plan(records, choices)
print_preview(records, choices)
pto_references = read_pto_references(pto_files, records) if choices.process_pto_files else []
prepare_plan(records, choices, pto_references)
pto_plans = plan_pto_updates(pto_references, records) if choices.process_pto_files else []
print_preview(records, choices, pto_plans)
try:
proceed = ask_yes_no("Proceed with these changes?", default=True)
@@ -1463,6 +1643,7 @@ def run(argv: list[str]) -> int:
return 0
failures = apply_changes(exiftool, records, choices)
failures.extend(apply_pto_changes(pto_plans))
removed_dirs = remove_empty_dirs(cleanup_roots, keep_dirs={working_dir.resolve()})
print_removed_dirs(removed_dirs)
+1
View File
@@ -0,0 +1 @@
"""Shared helpers for the media batch scripts."""
+18
View File
@@ -0,0 +1,18 @@
from __future__ import annotations
import sys
def prompt_input(prompt: str) -> str:
try:
return input(prompt)
except EOFError as exc:
raise RuntimeError("Interactive input ended before choices were complete.") from exc
def pause_if_interactive() -> None:
if sys.stdin.isatty():
try:
input("\nPress Enter to close...")
except EOFError:
pass
+49
View File
@@ -0,0 +1,49 @@
from __future__ import annotations
import json
import subprocess
from pathlib import Path
from shutil import which
def require_exiftool() -> str:
executable = which("exiftool")
if executable:
return executable
raise RuntimeError(
"exiftool was not found on PATH. Install ExifTool and make sure the "
"'exiftool' command works before running this script."
)
def run_exiftool_json(executable: str, paths: list[Path]) -> list[dict]:
results: list[dict] = []
for start in range(0, len(paths), 80):
chunk = paths[start : start + 80]
command = [
executable,
"-j",
"-G",
"-api",
"QuickTimeUTC=1",
"-charset",
"filename=UTF8",
*[str(path) for path in chunk],
]
completed = subprocess.run(command, check=True, capture_output=True, text=True)
results.extend(json.loads(completed.stdout or "[]"))
return results
def run_exiftool_write(executable: str, path: Path, args: list[str]) -> None:
if not args:
return
command = [
executable,
"-overwrite_original",
"-charset",
"filename=UTF8",
*args,
str(path),
]
subprocess.run(command, check=True)
+32
View File
@@ -0,0 +1,32 @@
from __future__ import annotations
import re
from datetime import datetime
from pathlib import Path
ORDERED_TIMESTAMP_STEM_RE = re.compile(r"^(?P<timestamp>\d{8}_\d{6})-[1-9]\d*$")
def naive_wall_time(dt: datetime) -> datetime:
return dt.replace(tzinfo=None)
def format_filename_stem(dt: datetime) -> str:
return naive_wall_time(dt).strftime("%Y%m%d_%H%M%S")
def normalized_extension(path: Path) -> str:
return path.suffix.lower()
def group_stem_from_name(name: str) -> str:
stem = Path(name).stem
match = ORDERED_TIMESTAMP_STEM_RE.match(stem)
if match:
return match.group("timestamp")
return stem
def group_stem(path: Path) -> str:
return group_stem_from_name(path.name)
+150
View File
@@ -0,0 +1,150 @@
from __future__ import annotations
import os
from pathlib import Path
def existing_input_paths(args: list[str], *, print_missing: bool = False) -> list[Path]:
paths: list[Path] = []
for arg in args:
path = Path(arg).expanduser()
if path.exists():
paths.append(path.resolve())
elif print_missing:
print(f"Skipping missing path: {arg}")
return paths
def determine_working_dir(args: list[str]) -> Path:
paths = existing_input_paths(args)
if not paths:
raise RuntimeError("No existing input paths were provided.")
if len(paths) == 1:
path = paths[0]
return path if path.is_dir() else path.parent
try:
common = Path(os.path.commonpath([str(path) for path in paths]))
except ValueError as exc:
raise RuntimeError("Input paths do not have a common ancestor.") from exc
if str(common) == common.anchor:
raise RuntimeError("Input paths only share the filesystem root; choose items closer together.")
return common
def collect_files(
args: list[str],
*,
allowed_exts: set[str] | None = None,
print_missing: bool = False,
sort_by_name: bool = False,
) -> list[Path]:
files: list[Path] = []
seen: set[Path] = set()
for arg in args:
path = Path(arg).expanduser()
if path.is_file():
candidates = [path] if allowed_exts is None or path.suffix.lower() in allowed_exts else []
elif path.is_dir():
candidates = [
child
for child in path.rglob("*")
if child.is_file()
and (allowed_exts is None or child.suffix.lower() in allowed_exts)
]
else:
if print_missing:
print(f"Skipping missing path: {arg}")
continue
for candidate in candidates:
resolved = candidate.resolve()
if resolved not in seen:
seen.add(resolved)
files.append(resolved)
if sort_by_name:
return sorted(files, key=lambda path: path.name.lower())
return sorted(files, key=lambda path: str(path).lower())
def first_file_from_path(path: Path) -> Path | None:
path = path.expanduser()
if path.is_file():
return path.resolve()
if path.is_dir():
for child in sorted(path.rglob("*"), key=lambda item: str(item).lower()):
if child.is_file():
return child.resolve()
return None
def cleanup_roots_from_args(args: list[str]) -> list[Path]:
roots: list[Path] = []
seen: set[Path] = set()
for arg in args:
path = Path(arg).expanduser()
if path.is_dir():
resolved = path.resolve()
if resolved not in seen:
seen.add(resolved)
roots.append(resolved)
return roots
def unique_path(path: Path) -> Path:
if not path.exists():
return path
for index in range(1, 10000):
candidate = path.with_name(f"{path.stem}-{index}{path.suffix}")
if not candidate.exists():
return candidate
raise RuntimeError(f"Could not find an available name for {path}")
def unique_existing_target(path: Path, planned_sources: set[Path]) -> Path:
if not path.exists() or path.resolve() in planned_sources:
return path
for index in range(1, 10000):
candidate = path.with_name(f"{path.stem}-{index}{path.suffix}")
if not candidate.exists() or candidate.resolve() in planned_sources:
return candidate
raise RuntimeError(f"Could not find available target name for {path}")
def remove_empty_dirs(cleanup_roots: list[Path], keep_dirs: set[Path]) -> list[Path]:
removed: list[Path] = []
candidates: set[Path] = set()
for root in cleanup_roots:
if not root.exists() or not root.is_dir():
continue
for directory, subdirs, _files in os.walk(root, topdown=False):
path = Path(directory).resolve()
candidates.add(path)
for subdir in subdirs:
child = (Path(directory) / subdir).resolve()
if child.exists() and child.is_dir():
candidates.add(child)
for directory in sorted(candidates, key=lambda path: len(path.parts), reverse=True):
if directory in keep_dirs:
continue
try:
directory.rmdir()
removed.append(directory)
except OSError:
pass
return removed
def display_path_from_working_dir(path: Path, working_dir: Path) -> str:
try:
relative = path.resolve().relative_to(working_dir.resolve())
except ValueError:
return str(path)
return "." if str(relative) == "." else str(relative)
+7 -22
View File
@@ -13,18 +13,10 @@ from __future__ import annotations
import subprocess
import sys
from pathlib import Path
from shutil import which
def first_file_from_path(path: Path) -> Path | None:
path = path.expanduser()
if path.is_file():
return path.resolve()
if path.is_dir():
for child in sorted(path.rglob("*"), key=lambda item: str(item).lower()):
if child.is_file():
return child.resolve()
return None
from tools.console import pause_if_interactive
from tools.exiftool import require_exiftool
from tools.filesystem import first_file_from_path
def choose_file(args: list[str]) -> Path | None:
@@ -35,22 +27,15 @@ def choose_file(args: list[str]) -> Path | None:
return None
def pause_if_interactive() -> None:
if sys.stdin.isatty():
try:
input("\nPress Enter to close...")
except EOFError:
pass
def run(argv: list[str]) -> int:
if len(argv) < 2:
print("Usage: python3 view_metadata.py path/to/file/or/directory [...]")
return 2
exiftool = which("exiftool")
if exiftool is None:
print("exiftool was not found on PATH.")
try:
exiftool = require_exiftool()
except RuntimeError as exc:
print(exc)
return 1
file_path = choose_file(argv[1:])