224 lines
15 KiB
Markdown
224 lines
15 KiB
Markdown
# Media Batch Tools
|
|
|
|
Small Python scripts for organizing photo collections and preparing or encoding edited videos.
|
|
|
|
## Scripts
|
|
|
|
### `photo_metadata.py`
|
|
|
|
Interactive metadata cleanup for photo collections:
|
|
|
|
```bash
|
|
python3 photo_metadata.py path/to/file/or/directory [...]
|
|
```
|
|
|
|
Directory inputs are scanned recursively. The script reads metadata, asks for the choices it needs, prints a preview, and only then applies changes in place after confirmation.
|
|
|
|
Current features:
|
|
|
|
- supports `.jpg`, `.jpeg`, `.heic`, `.arw`, `.mp4`, `.mov`, and `.mts`
|
|
- reads and writes metadata through `exiftool` from `PATH`
|
|
- optional direct time shifts, such as `+1:30`, `-02:00:00`, or `+1h 2m`
|
|
- optional reference-clock correction using a source photo timestamp or filename plus the correct time
|
|
- timezone handling for photos and local-time video filenames
|
|
- artist/author set, clear, or leave unchanged
|
|
- timestamp-based renaming to `YYYYMMDD_HHMMSS.ext`
|
|
- video filenames use the beginning timestamp by subtracting rounded duration from the video metadata timestamp
|
|
- Sony-style video sidecars like `C0011.MP4` plus `C0011M01.XML` are moved and renamed with the video
|
|
- automatic photo burst/HDR grouping per camera
|
|
- optional panorama project (`.pto`) grouping and reference updates
|
|
- inferred subsecond metadata for grouped same-second photo bursts
|
|
|
|
The working directory is determined from the dragged items. A single dragged folder is the working directory. For multiple dragged files/folders, their last common ancestor is the working directory. Automatic burst/HDR and `.pto` group folders are created inside that working directory.
|
|
|
|
If dragged folders contain files in subdirectories and grouping is enabled, ungrouped files are moved into the working directory too. This makes the run organize everything into a deterministic flat reset state, with grouped files in their group folders and the remaining files directly in the working directory. If grouping is disabled, the script asks whether to flatten subdirectory files into the working directory.
|
|
|
|
Panorama project files use the `.pto` extension. When enabled, every media file referenced by a `.pto` file belongs to the same group. If those files already belong to burst/HDR groups, the groups are merged. `.pto` files are moved into the group folder with their referenced files, renamed from the first and last referenced media stems, and their file references are rewritten to the new relative paths.
|
|
|
|
Reference-clock correction is useful when one photo shows a reliable clock. For example, if `DSC01234.JPG` has camera metadata `2026:06:02 22:10:00`, but the clock in the photo shows `22:13:25`, choose reference mode, enter `DSC01234.JPG` as the source, and enter `22:13:25` as the correct time. The script computes a `+00:03:25` shift and applies that correction to the selected files. If you omit the date in the correct time, the script chooses the date closest to the source timestamp, within 12 hours.
|
|
|
|
### `grouping.py`
|
|
|
|
Fast manual grouping for selected files:
|
|
|
|
```bash
|
|
python3 grouping.py path/to/file/or/directory [...]
|
|
```
|
|
|
|
It does not ask questions. It recursively collects all files inside the dragged files/folders, sorts those files by name, and moves them into one folder named `[first stem]-[last stem]`.
|
|
|
|
Manual grouping uses the same working-directory rule as `photo_metadata.py`:
|
|
|
|
- one dragged folder is the working directory
|
|
- multiple dragged files/folders use their last common ancestor as the working directory
|
|
- the group folder is created inside the working directory
|
|
- all collected files are flattened into that group folder
|
|
|
|
### `view_metadata.py`
|
|
|
|
Show all readable metadata for the first file provided:
|
|
|
|
```bash
|
|
python3 view_metadata.py path/to/file/or/directory [...]
|
|
```
|
|
|
|
This script only displays one file. If several files are given, it displays the first one. If a directory is given, it searches recursively and displays the first file it finds.
|
|
|
|
It runs ExifTool as:
|
|
|
|
```bash
|
|
exiftool -a -u -ee -G0:1 -s file
|
|
```
|
|
|
|
The important choices are:
|
|
|
|
- `-a` shows duplicate tag names instead of hiding later duplicates.
|
|
- `-u` includes unknown tags ExifTool can identify well enough to print.
|
|
- `-ee` extracts embedded metadata where ExifTool supports it, which is useful for some video/container formats.
|
|
- `-G0:1` shows both the broad metadata container and the more exact location, such as `EXIF:IFD0` or `QuickTime:Track1`.
|
|
- `-s` uses compact tag names, which are easier to copy into scripts than the descriptive labels.
|
|
|
|
The old `-G` style shows broad groups like `EXIF`, `File`, or `QuickTime`. That is readable, but it can hide where inside a container a value came from. Family 1 groups, from `-G1`, show more exact locations like `IFD0`, `ExifIFD`, `MakerNotes`, or `Track1`, but sometimes lose the broader context. `-G0:1` combines both, so it is the most useful default for debugging metadata without changing which normal tags are extracted. The organization option changes labels, not the underlying metadata extraction; `-a` and `-u` are the parts that affect whether duplicate or unknown printed tags are included.
|
|
|
|
### `copy_metadata.py`
|
|
|
|
Copy meaningful metadata from one file to another:
|
|
|
|
```bash
|
|
python3 copy_metadata.py
|
|
python3 copy_metadata.py source.jpg destination.jpg
|
|
```
|
|
|
|
If run without files, it asks for source and destination paths. If exactly two files are dragged onto it, it asks which one is the source. Any other number of dragged files is an error.
|
|
|
|
The script copies normal metadata with ExifTool `-TagsFromFile`, while excluding file/system tags, composite tags, image dimensions, pixel dimensions, and resolution fields that should remain specific to the destination file.
|
|
|
|
### `video_encode.py`
|
|
|
|
Prepare, validate, encode, and remux Avisynth-edited videos:
|
|
|
|
```bash
|
|
python3 video_encode.py path/to/file/or/directory [...]
|
|
```
|
|
|
|
Source videos are untouched until you explicitly choose a final action after encoding. The script recursively collects `.mp4`, `.mov`, `.mkv`, `.mts`, `.m2ts`, and `.avi` files, builds a collapsed drag tree, and creates a workspace in `video_workspace/`.
|
|
|
|
The workflow is:
|
|
|
|
1. Probe the source files with `ffprobe` and ExifTool, including frame timing, embedded timezone, location, and HDR metadata.
|
|
2. Create editable `.avs` scripts in `video_workspace/` and generated source/validation files in `video_workspace/.source/`.
|
|
3. Edit the visible `.avs` files. Existing editable files are kept unless you confirm an overwrite.
|
|
4. Press Enter for fast blank-frame validation, `r` for real-frame validation, or `q` to stop after workspace creation.
|
|
5. Review encoding settings, encode accepted scripts, then leave outputs in the workspace or move them to the source directories.
|
|
|
|
Generated source scripts use FFMS2 `FFVideoSource`, mark every frame with source identity properties, and leave the source clip in `last`. Custom Avisynth filters must preserve those frame properties. Validation renders every output frame and rejects scripts that lose or corrupt the identity mapping.
|
|
|
|
The visible scripts document the available helpers, including `MBT_Drop`, `MBT_DropEvery`, `MBT_Info`, `MBT_AbsoluteTimeStretch`, `Resize`, `RotateCrop`, `MBT_CorrectMatrix`, `MBT_ToSDR`, `Cropf`, `DeShake`, and `FixContrast`. `MBT_Drop` omits frames while preserving their timeline duration; normal operations such as `Trim` and `SelectEven` remove source time. `MBT_AbsoluteTimeStretch()` adjusts the absolute timestamps used for metadata and `MBT_Info()` without changing playback speed.
|
|
|
|
On Windows, ending a visible `.avs` file with a commented `#32bit` or `# 32-bit` marker selects a bundled/configured 32-bit runner for that script. Linux uses the 64-bit runner.
|
|
|
|
The bundled runner is used automatically when found. Build it on Linux with:
|
|
|
|
```bash
|
|
tools/avisynth_runner/build_linux.sh
|
|
```
|
|
|
|
On Windows, use the bundled `tools/avisynth_runner/mbt_avs_runner.exe` with installed AviSynth+. Set `MBT_AVS_RUNNER`, `MBT_AVS_RUNNER64`, or `MBT_AVS_RUNNER32` to override runner locations. Generated scripts normally rely on FFMS2 autoload; set `MBT_FFMS2_PLUGIN` to generate `LoadPlugin("...")`, or `MBT_ASK_FFMS2_PLUGIN=1` to ask interactively.
|
|
|
|
#### Encoding
|
|
|
|
- x265 is the default encoder. x264, x265, and unchanged-video stream copy are available when compatible with the validated output.
|
|
- The tool supports CFR, VFR, trims, playback-speed changes, `MBT_Drop`, normal frame deletion, and mixed deletion/drop edits.
|
|
- VFR and normal-deletion output preserves frame timing through timestamp-v2 files and MKVToolNix. x264 receives those timestamps through `--tcfile-in`; x265 receives them during the muxing step.
|
|
- Video filenames can use the edited beginning timestamp in a chosen timezone. Video container metadata is written with the edited ending timestamp. Meaningful source metadata, including location, is copied while physical stream properties remain specific to the output.
|
|
- Existing output names can be overwritten or given `-1`, `-2`, and so on. Same-batch collisions always receive a suffix.
|
|
- Profile and level can be unset, calculated as the per-clip minimum, or selected as a floor. x264 is unavailable for output above 10-bit, and output above 12-bit is rejected.
|
|
- MP4 and MKV support AAC, Opus, FLAC, copied audio, or no audio. Stream-copy compatibility is checked before encoding.
|
|
|
|
Audio is read from the original source by default. An optional `[visible script stem]_audio.avs` is rendered to temporary WAV and used instead when present. AAC audio is trimmed and tempo-adjusted for the validated output. Audio stream copy is limited to whole-video and simple beginning/end trims, so normal frame deletion and speed changes require audio re-encoding or no audio.
|
|
|
|
After successful encoding, choose to leave files in `video_workspace/`, replace the originals, or move originals to `video_workspace/originals/` before moving the new files to the source directories.
|
|
|
|
#### HDR
|
|
|
|
HDR clips remain HDR unless the script calls `MBT_ToSDR()`. HDR-preserving x265 encodes retain validated color properties and static HDR metadata. HDR10+ additionally requires `hdr10plus_tool`; other dynamic HDR formats currently require video stream copy. `MBT_ToSDR()` uses HDRTools and offers `MPC` (default), `Hable`, `Mobius`, `Reinhard`, `ACES`, and `BT2446C`. Tone-mapped output should be visually checked before relying on it for archival work.
|
|
|
|
#### Non-interactive Encoding
|
|
|
|
Set `MBT_ENCODE_VIDEO=1` to encode non-interactively after validation. The following optional variables configure that run:
|
|
|
|
- `MBT_VIDEO_TIMESTAMP_NAMES=1` names outputs as `YYYYMMDD_HHMMSS.mp4`; `0` keeps original-style workspace names.
|
|
- `MBT_VIDEO_TIMEZONE=+03:00` selects the timezone used for timestamp filenames. If omitted, one consistent embedded source timezone is preferred; conflicting or missing source offsets fall back to the system timezone. No timezone is needed when timestamp filenames are disabled.
|
|
- `MBT_VIDEO_CONTAINER=mp4` or `mkv` selects the output container. MP4 is the default.
|
|
- `MBT_VIDEO_CODEC=x264`, `x265`, or `copy` selects FFmpeg `libx264`, `libx265`, or direct source-video remuxing; x265 is the default. `copy` is only available for unchanged validated video timelines.
|
|
- `MBT_VIDEO_CRF=16` sets CRF. Interactive defaults are 16 for x264 and 21 for x265.
|
|
- `MBT_VIDEO_PRESET=slow` sets the encoder preset.
|
|
- `MBT_VIDEO_PROFILE=minimum` and `MBT_VIDEO_LEVEL=minimum` calculate per-clip constraints. A named profile or level acts as a requested floor. Omit either variable to apply no constraint for it.
|
|
- `MBT_VIDEO_THREADS=auto` (the default) leaves encoder thread selection automatic. Set it to a positive integer to limit x264 threads or x265 worker pools.
|
|
- `MBT_VIDEO_AUDIO=aac`, `opus`, `flac`, `copy`, or `none` selects audio handling. AAC, Opus, and FLAC are valid in either supported container, subject to player compatibility.
|
|
- `MBT_VIDEO_AUDIO_BITRATE=192k` sets AAC bitrate.
|
|
- `MBT_VIDEO_AUDIO_PITCH=preserve` or `shift` selects whether speed-changed AAC audio keeps pitch or shifts pitch with playback speed. The default is `preserve`.
|
|
- `MBT_VIDEO_FINAL_ACTION=leave`, `replace`, or `backup` selects the final disposition.
|
|
- `MBT_VIDEO_DELETE_WORKSPACE=1` deletes `video_workspace` after moving outputs.
|
|
|
|
Missing executables are requested by absolute path for the current run when possible. `mkvmerge` also checks the standard Windows installation path.
|
|
|
|
## Testing
|
|
|
|
Run the Python tests with:
|
|
|
|
```bash
|
|
python3 -m unittest discover -s unit_tests
|
|
```
|
|
|
|
The pure Python tests always run. The video end-to-end scenario tests skip unless the local test fixtures and toolchain are available: `tests/`, ExifTool, FFmpeg/FFprobe, FFMS2 for Avisynth, and the bundled `mbt_avs_runner`.
|
|
|
|
Current video scenarios copy fixture videos to temporary directories before processing and validate:
|
|
|
|
- CFR MP4 encode with AAC audio and metadata copy
|
|
- VFR MP4 trim with copied GPS metadata and adjusted end timestamp
|
|
- normal frame deletion such as `SelectEven()` with segmented AAC audio
|
|
- mixed normal frame deletion plus `MBT_Drop`
|
|
- `MBT_Drop(start,end)` and `MBT_DropEvery(...)` helper behavior
|
|
- MKV output with Opus audio
|
|
- MP4 video/audio stream-copy remuxing
|
|
|
|
## Requirements
|
|
|
|
- Python 3.10 or newer
|
|
- ExifTool available as `exiftool` on `PATH`
|
|
- FFmpeg available as `ffmpeg` and `ffprobe` on `PATH` for `video_encode.py`
|
|
- MKVToolNix available as `mkvmerge` on `PATH` for VFR or frame-deletion output (Windows also checks its standard installation path)
|
|
- the x264 command-line encoder available as `x264` on `PATH` for timestamp-aware x264 output
|
|
- `hdr10plus_tool` available on `PATH` only when re-encoding HDR10+ video while retaining HDR
|
|
- FFMS2 available to Avisynth as `FFVideoSource` before generated `.avs` scripts can load real videos, either through plugin autoload or a `LoadPlugin` path
|
|
- AviSynth+ development files and a C compiler are needed only if building the bundled runner from source
|
|
|
|
On Debian/Ubuntu-like Linux systems, ExifTool is commonly installed with:
|
|
|
|
```bash
|
|
sudo apt install libimage-exiftool-perl
|
|
```
|
|
|
|
FFmpeg/FFprobe is commonly installed with:
|
|
|
|
```bash
|
|
sudo apt install ffmpeg x264 mkvtoolnix
|
|
```
|
|
|
|
On Windows, install ExifTool and make sure the `exiftool` command is available in a normal terminal. For timestamp-driven video output, also install MKVToolNix; the usual `C:\Program Files\MKVToolNix\mkvmerge.exe` location is detected automatically, otherwise add `mkvmerge` to `PATH`. Timestamp-aware x264 output additionally needs the x264 command-line encoder as `x264` on `PATH`.
|
|
|
|
No virtual environment is required. The scripts use only the Python standard library plus external command-line tools.
|
|
|
|
## Safety
|
|
|
|
`photo_metadata.py` edits files in place, but it prints a preview and asks for confirmation before writing metadata or moving files. Test media should be copied to a temporary directory before running destructive checks.
|
|
|
|
`video_encode.py` creates files under `video_workspace/`, including encoded outputs when requested. It only modifies source directories or source videos if you explicitly choose a final action that moves outputs back to the source folders.
|
|
|
|
`tests/`, `old/`, and `video_workspace/` are intentionally ignored by git. The `tests/` directory is for reusable source fixtures that should not be modified directly.
|
|
|
|
## AI Note
|
|
|
|
This code was written with AI assistance and should be reviewed and tested before relying on it for important media archives.
|