Add Avisynth video encoding workflow

README.md

@@ -93,10 +93,135 @@ If run without files, it asks for source and destination paths. If exactly two f
 
 The script copies normal metadata with ExifTool `-TagsFromFile`, while excluding file/system tags, composite tags, image dimensions, pixel dimensions, and resolution fields that should remain specific to the destination file.
 
+### `video_encode.py`
+
+Probe video files for the future editing and encoding workflow:
+
+```bash
+python3 video_encode.py path/to/file/or/directory [...]
+```
+
+This script currently leaves source videos untouched. It recursively collects supported videos, builds the collapsed drag tree, prints probe data from `ffprobe` and `exiftool`, creates the Avisynth workspace layout under `video_workspace/`, pauses for edits, then validates the edited scripts before optional encoding.
+
+Current probe output includes:
+
+- codec, dimensions, duration, frame count, stream rates, and time base
+- CFR/VFR classification based on frame timestamps
+- beginning-to-end timestamp range, using an embedded source timezone such as Samsung `AndroidTimeZone` or Sony `TimeZone` when available, otherwise UTC
+- rounded-duration beginning timestamp, matching the photo metadata script's video naming rule
+
+Current workspace output includes:
+
+- editable scripts in `video_workspace/`
+- generated hidden source scripts in `video_workspace/.source/`
+- generated validation wrappers in `video_workspace/.source/`
+- per-frame `ConditionalReader` data for timestamps and durations
+
+Existing editable `.avs` scripts are not overwritten unless you confirm it.
+
+After creating the workspace, the script waits while you edit the visible `.avs` files. Press Enter for fast blank-frame validation, type `r` for real-frame validation, or type `q` to stop after workspace creation.
+
+The hidden source scripts currently target FFMS2's `FFVideoSource`, import bundled helpers, and mark source frames with pure Avisynth+ `propSet` frame identity properties. By default, generated scripts rely on Avisynth autoload for FFMS2. Set `MBT_FFMS2_PLUGIN` to a plugin path to generate `LoadPlugin("...")`, or set `MBT_ASK_FFMS2_PLUGIN=1` to ask for that path interactively.
+
+Python keeps the probed timestamp table and later maps output frames back to source timing from `mbt_source_id` and `mbt_source_frame`.
+
+The bundled helpers currently include `MBT_Drop`, `MBT_Undrop`, `MBT_DropEvery`, `MBT_UndropEvery`, `MBT_Info`, `Resize`, `RotateCrop`, `MBT_CorrectMatrix`, `Cropf`, `DeShake`, and `FixContrast`. `RotateCrop(angle, dar)` uses the external manyPlus `Turn` function, then crops the largest rectangle of the requested aspect ratio that contains no rotation background; `dar` defaults to the source aspect ratio. Source scripts inject the detected matrix into the standard `_Matrix` frame property. `Resize` reads `_Matrix` when `input_matrix` is not supplied, chooses an output matrix from the output resolution, updates `_Matrix`, and auto-enables linear-light resizing when either dimension is downscaled by 1.5x or more. The linear path requires `y_gamma_to_linear`/`y_linear_to_gamma`; pass `linear=false` or `linear=true` to override that decision. `DeShake` still requires the usual Avisynth stabilization plugins to be available if you call it.
+
+Generated editable `.avs` files include source size/chroma/bitdepth, frame count, frame rate, duration, and timing classification. The hidden source scripts also generate `ConditionalReader` sidecar files for each source frame's relative video time, absolute timestamp, and frame duration, then attach those values as frame properties. Use `MBT_Info(last)` while editing to overlay the user-facing frame properties. The internal `mbt_source_id` property is still present for batch validation, but it is not shown by that helper.
+
+Validation wrappers are named `[original filename].validate.avs`. Fast validation is the default: the runner sets an in-memory AviSynth variable before importing the wrapper, making the hidden source use a `BlankClip` with the source clip's format, frame count, and frame rate. This exercises frame selection/properties without decoding every source image. Real-frame validation uses the same wrapper but disables that variable. The bundled runner writes a temporary frame table containing `output_frame,source_id,source_frame,drop_frame,matrix`; Python reads it to check that frame identity survived the edited script, then removes it after successful validation.
+
+Set `MBT_ENCODE_VIDEO=1` to encode after validation. This milestone encodes accepted CFR timelines, VFR timelines, CFR speed changes, `MBT_Drop` output, normal frame deletion such as `SelectEven()`, and mixed deletion/drop edits. When normal frame deletion removes source time, AAC audio is cut into matching segments before encoding; `MBT_Drop` durations keep audio continuous.
+
+Validation also reports the actual Avisynth output frame rate and playback duration. If the script changes playback speed, for example with `AssumeFPS`, AAC audio is tempo-adjusted to match the encoded video duration.
+
+After validation, interactive runs switch to an encoding review screen. The top of the screen lists the validated clips and edited beginning timestamps in UTC; below it, answered encoding settings are shown in a small table. Type `b` to go back. If a setting changes another setting's meaning, such as switching between x264 and x265, dependent answers such as CRF/profile/level are cleared and return to their codec-specific defaults.
+
+x265 is the default video encoder. Profile and level can each be left unset, calculated as the minimum required for every clip individually, or selected manually as a requested floor. A manual value that is too low for one clip is upgraded only for that clip. Choices shown in red do not directly support every clip in the batch. x264 is unavailable when any validated Avisynth output is above 10-bit; output above 12-bit fails validation and is excluded from encoding. For x265, tier follows the level constraint: x265 tries Main tier first and may use High tier when the selected level requires it.
+
+The repo includes source for a small bundled Avisynth runner in `tools/avisynth_runner/`. Build it on Linux with:
+
+```bash
+tools/avisynth_runner/build_linux.sh
+```
+
+When `tools/avisynth_runner/mbt_avs_runner` exists, `video_encode.py` uses it automatically for validation, Y4M video piping, timeline-preserving `drop_frame` omission, and optional WAV audio rendering. Encoding shows the Avisynth rendering and encoder-output progress separately, so encoder lookahead is visible. To use a runner from another location, set `MBT_AVS_RUNNER` to that executable.
+
+On Windows, the bundled 64-bit runner is `tools/avisynth_runner/mbt_avs_runner.exe`. It loads the installed 64-bit AviSynth+ runtime at execution time, so AviSynth+ still needs to be installed. The script automatically tries the bundled 64-bit runner first. If a bundled or configured 32-bit runner exists, it can be selected for a specific editable `.avs` by ending that file with a commented marker like `#32bit` or `# 32-bit`. Trailing blank lines are ignored. To override runner locations, set `MBT_AVS_RUNNER64`, `MBT_AVS_RUNNER32`, or `MBT_AVS_RUNNER`.
+
+Piping Y4M through the bundled runner does not add video quality loss, because Y4M is uncompressed video. The tradeoff is that a raw video pipe does not carry audio, metadata, frame properties, or arbitrary per-frame timestamps; those are handled separately by Python. FFmpeg is used for encoding and muxing, not as the Avisynth host.
+
+Audio is not piped together with Y4M video. By default, FFmpeg reads audio from the original source file and the Python tool applies the validated trim/speed plan through FFmpeg filters. If an optional `[visible script stem]_audio.avs` exists, the runner renders that script's audio to a temporary WAV with `--wav`, and FFmpeg uses that WAV as the audio source. This keeps video piping simple while still allowing Avisynth-based audio edits.
+
+For VFR output or normal frame deletion, the tool writes one timestamp-v2 sidecar and keeps only a compressed temporary video. x264 uses a raw H.264 intermediate and receives those timestamps during encoding through `--tcfile-in`, so its CRF rate control sees individual frame durations. x265 has no equivalent VFR rate-control input, so it encodes an intermediate Matroska video at the edited timeline's average frame rate. MKVToolNix then applies the exact frame timestamps to either compressed stream, and FFmpeg stream-copies the timed video while adding audio and writing the requested MP4 or MKV container. This avoids the previous temporary PNG sequence.
+
+Timestamp-driven output requires `mkvmerge` on `PATH`. On Windows, the standard `C:\Program Files\MKVToolNix\mkvmerge.exe` installation is also detected automatically. Timestamp-driven x264 output also requires the `x264` command-line encoder on `PATH`; CFR x264 output continues to use FFmpeg's `libx264` encoder.
+
+In an interactive run, if an external executable such as `mkvmerge`, `x264`, `ffmpeg`, `ffprobe`, or `exiftool` is missing from `PATH` and has no supported default location, the script asks for its absolute path. That path is used for the rest of the current batch only, so a late missing-tool discovery does not discard the validated workspace or encoding choices.
+
+When encoding, the script can name outputs from the edited beginning timestamp, using nearest-second rounding and a chosen timezone. Before encoding begins, existing output names are listed and can be overwritten (default) or assigned `-1`, `-2`, and so on. Same-batch naming collisions always use that suffixing. Source metadata, including location metadata, is copied to the output while excluding physical media properties such as dimensions, duration, bitrate, frame rate, codec, and audio layout. MP4 output then gets video container timestamp tags written to the edited end timestamp in UTC. The encoded files are written next to the editable `.avs` scripts under `video_workspace/`.
+
+Audio support currently covers validated outputs that the current encoder path supports:
+
+- AAC re-encode trims audio to the validated output span and tempo-adjusts it when the Avisynth output duration changes.
+- Normal frame deletion uses segmented AAC audio cuts so removed frame durations are removed from audio too.
+- Audio copy is allowed for whole-file outputs and simple beginning/end trims. FFmpeg does input-side stream-copy trimming for that case, so the cut can be limited by packet/keyframe boundaries. Normal frame deletion and speed changes still require audio re-encoding or no audio.
+- If copy is requested for a trimmed or speed-changed output, the output is video-only rather than risking bad sync.
+- MKV output can use Opus or FLAC audio. MP4 output intentionally stays limited to AAC, copy, or no audio for compatibility.
+- Optional audio edits are supported by copying the editable video `.avs` to `[visible script stem]_audio.avs` next to it. The normal generated source import already includes source audio when the video has audio, so the audio edit script can be the same script shape as the video edit script. Its video output is ignored, and its whole audio output is rendered to WAV, then trimmed by Python to match the validated video timeline.
+
+After successful encodes, the script asks what to do with the finished files:
+
+- leave them in `video_workspace/`
+- move them to the source directories and delete the originals
+- move them to the source directories and move originals into `video_workspace/originals/`
+
+Non-interactive encode options:
+
+- `MBT_VIDEO_TIMESTAMP_NAMES=1` names outputs as `YYYYMMDD_HHMMSS.mp4`; `0` keeps original-style workspace names.
+- `MBT_VIDEO_TIMEZONE=+03:00` selects the timezone used for timestamp filenames. If omitted, one consistent embedded source timezone is preferred; conflicting or missing source offsets fall back to the system timezone. No timezone is needed when timestamp filenames are disabled.
+- `MBT_VIDEO_CONTAINER=mp4` or `mkv` selects the output container. MP4 is the default.
+- `MBT_VIDEO_CODEC=x264` or `x265` selects FFmpeg `libx264` or `libx265`; x265 is the default.
+- `MBT_VIDEO_CRF=16` sets CRF. Interactive defaults are 16 for x264 and 21 for x265.
+- `MBT_VIDEO_PRESET=slow` sets the encoder preset.
+- `MBT_VIDEO_PROFILE=minimum` and `MBT_VIDEO_LEVEL=minimum` calculate per-clip constraints. A named profile or level acts as a requested floor. Omit either variable to apply no constraint for it.
+- `MBT_VIDEO_THREADS=auto` (the default) leaves encoder thread selection automatic. Set it to a positive integer to limit x264 threads or x265 worker pools.
+- `MBT_VIDEO_AUDIO=aac`, `opus`, `flac`, `copy`, or `none` selects audio handling. Opus and FLAC require MKV.
+- `MBT_VIDEO_AUDIO_BITRATE=192k` sets AAC bitrate.
+- `MBT_VIDEO_AUDIO_PITCH=preserve` or `shift` selects whether speed-changed AAC audio keeps pitch or shifts pitch with playback speed. The default is `preserve`.
+- `MBT_VIDEO_FINAL_ACTION=leave`, `replace`, or `backup` selects the final disposition.
+- `MBT_VIDEO_DELETE_WORKSPACE=1` deletes `video_workspace` after moving outputs.
+
+Supported initial video extensions are `.mp4`, `.mov`, `.mkv`, `.mts`, `.m2ts`, and `.avi`.
+
+## Testing
+
+Run the Python tests with:
+
+```bash
+python3 -m unittest discover -s unit_tests
+```
+
+The pure Python tests always run. The video end-to-end scenario tests skip unless the local test fixtures and toolchain are available: `tests/`, ExifTool, FFmpeg/FFprobe, FFMS2 for Avisynth, and the bundled `mbt_avs_runner`.
+
+Current video scenarios copy fixture videos to temporary directories before processing and validate:
+
+- CFR MP4 encode with AAC audio and metadata copy
+- VFR MP4 trim with copied GPS metadata and adjusted end timestamp
+- normal frame deletion such as `SelectEven()` with segmented AAC audio
+- mixed normal frame deletion plus `MBT_Drop`
+- `MBT_Drop(start,end)` and `MBT_DropEvery(...)` helper behavior
+- MKV output with Opus audio
+
 ## Requirements
 
 - Python 3.10 or newer
 - ExifTool available as `exiftool` on `PATH`
+- FFmpeg available as `ffmpeg` and `ffprobe` on `PATH` for `video_encode.py`
+- MKVToolNix available as `mkvmerge` on `PATH` for VFR or frame-deletion output (Windows also checks its standard installation path)
+- the x264 command-line encoder available as `x264` on `PATH` for timestamp-aware x264 output
+- FFMS2 available to Avisynth as `FFVideoSource` before generated `.avs` scripts can load real videos, either through plugin autoload or a `LoadPlugin` path
+- AviSynth+ development files and a C compiler are needed only if building the bundled runner from source
 
 On Debian/Ubuntu-like Linux systems, ExifTool is commonly installed with:
 
@@ -104,7 +229,13 @@ On Debian/Ubuntu-like Linux systems, ExifTool is commonly installed with:
 sudo apt install libimage-exiftool-perl
 ```
 
-On Windows, install ExifTool and make sure the `exiftool` command is available in a normal terminal.
+FFmpeg/FFprobe is commonly installed with:
+
+```bash
+sudo apt install ffmpeg x264 mkvtoolnix
+```
+
+On Windows, install ExifTool and make sure the `exiftool` command is available in a normal terminal. For timestamp-driven video output, also install MKVToolNix; the usual `C:\Program Files\MKVToolNix\mkvmerge.exe` location is detected automatically, otherwise add `mkvmerge` to `PATH`. Timestamp-aware x264 output additionally needs the x264 command-line encoder as `x264` on `PATH`.
 
 No virtual environment is required. The scripts use only the Python standard library plus external command-line tools.
 
@@ -112,11 +243,13 @@ No virtual environment is required. The scripts use only the Python standard lib
 
 `photo_metadata.py` edits files in place, but it prints a preview and asks for confirmation before writing metadata or moving files. Test media should be copied to a temporary directory before running destructive checks.
 
-`tests/`, `old/`, and `PLAN.md` are intentionally ignored by git. The `tests/` directory is for reusable source fixtures that should not be modified directly.
+`video_encode.py` creates files under `video_workspace/`, including encoded outputs when requested. It only modifies source directories or source videos if you explicitly choose a final action that moves outputs back to the source folders.
+
+`tests/`, `old/`, `PLAN.md`, `VIDEO_PLAN.md`, and `video_workspace/` are intentionally ignored by git. The `tests/` directory is for reusable source fixtures that should not be modified directly.
 
 ## Later Work
 
-Separate video tools may be added later for remuxing, re-encoding, audio extraction/conversion, and timecode handling. Those features are intentionally not mixed into `photo_metadata.py`.
+`video_encode.py` will grow into the separate video tool for Avisynth editing, remuxing, re-encoding, audio handling, and timecode handling. Those features are intentionally not mixed into `photo_metadata.py`.
 
 ## AI Note