Enhance Avisynth video workflow and metadata tools

This commit is contained in:
ajp_anton
2026-07-30 03:20:19 +00:00
parent d60ce51b14
commit 100cad1cb5
26 changed files with 2350 additions and 1161 deletions
-2
View File
@@ -1,7 +1,5 @@
old/ old/
tests/ tests/
PLAN.md
VIDEO_PLAN.md
__pycache__/ __pycache__/
video_workspace/ video_workspace/
tools/avisynth_runner/mbt_avs_runner tools/avisynth_runner/mbt_avs_runner
+31 -67
View File
@@ -1,6 +1,6 @@
# Media Batch Tools # Media Batch Tools
Small Python scripts for organizing photo collections, with videos handled as companion files when they are part of the same folder of photos. Small Python scripts for organizing photo collections and preparing or encoding edited videos.
## Scripts ## Scripts
@@ -95,90 +95,57 @@ The script copies normal metadata with ExifTool `-TagsFromFile`, while excluding
### `video_encode.py` ### `video_encode.py`
Probe video files for the future editing and encoding workflow: Prepare, validate, encode, and remux Avisynth-edited videos:
```bash ```bash
python3 video_encode.py path/to/file/or/directory [...] python3 video_encode.py path/to/file/or/directory [...]
``` ```
This script currently leaves source videos untouched. It recursively collects supported videos, builds the collapsed drag tree, prints probe data from `ffprobe` and `exiftool`, creates the Avisynth workspace layout under `video_workspace/`, pauses for edits, then validates the edited scripts before optional encoding. Source videos are untouched until you explicitly choose a final action after encoding. The script recursively collects `.mp4`, `.mov`, `.mkv`, `.mts`, `.m2ts`, and `.avi` files, builds a collapsed drag tree, and creates a workspace in `video_workspace/`.
Current probe output includes: The workflow is:
- codec, dimensions, duration, frame count, stream rates, and time base 1. Probe the source files with `ffprobe` and ExifTool, including frame timing, embedded timezone, location, and HDR metadata.
- CFR/VFR classification based on frame timestamps 2. Create editable `.avs` scripts in `video_workspace/` and generated source/validation files in `video_workspace/.source/`.
- beginning-to-end timestamp range, using an embedded source timezone such as Samsung `AndroidTimeZone` or Sony `TimeZone` when available, otherwise UTC 3. Edit the visible `.avs` files. Existing editable files are kept unless you confirm an overwrite.
- rounded-duration beginning timestamp, matching the photo metadata script's video naming rule 4. Press Enter for fast blank-frame validation, `r` for real-frame validation, or `q` to stop after workspace creation.
5. Review encoding settings, encode accepted scripts, then leave outputs in the workspace or move them to the source directories.
Current workspace output includes: Generated source scripts use FFMS2 `FFVideoSource`, mark every frame with source identity properties, and leave the source clip in `last`. Custom Avisynth filters must preserve those frame properties. Validation renders every output frame and rejects scripts that lose or corrupt the identity mapping.
- editable scripts in `video_workspace/` The visible scripts document the available helpers, including `MBT_Drop`, `MBT_DropEvery`, `MBT_Info`, `MBT_AbsoluteTimeStretch`, `Resize`, `RotateCrop`, `MBT_CorrectMatrix`, `MBT_ToSDR`, `Cropf`, `DeShake`, and `FixContrast`. `MBT_Drop` omits frames while preserving their timeline duration; normal operations such as `Trim` and `SelectEven` remove source time. `MBT_AbsoluteTimeStretch()` adjusts the absolute timestamps used for metadata and `MBT_Info()` without changing playback speed.
- generated hidden source scripts in `video_workspace/.source/`
- generated validation wrappers in `video_workspace/.source/`
- per-frame `ConditionalReader` data for timestamps and durations
Existing editable `.avs` scripts are not overwritten unless you confirm it. On Windows, ending a visible `.avs` file with a commented `#32bit` or `# 32-bit` marker selects a bundled/configured 32-bit runner for that script. Linux uses the 64-bit runner.
After creating the workspace, the script waits while you edit the visible `.avs` files. Press Enter for fast blank-frame validation, type `r` for real-frame validation, or type `q` to stop after workspace creation. The bundled runner is used automatically when found. Build it on Linux with:
The hidden source scripts currently target FFMS2's `FFVideoSource`, import bundled helpers, and mark source frames with pure Avisynth+ `propSet` frame identity properties. By default, generated scripts rely on Avisynth autoload for FFMS2. Set `MBT_FFMS2_PLUGIN` to a plugin path to generate `LoadPlugin("...")`, or set `MBT_ASK_FFMS2_PLUGIN=1` to ask for that path interactively.
Python keeps the probed timestamp table and later maps output frames back to source timing from `mbt_source_id` and `mbt_source_frame`.
The bundled helpers currently include `MBT_Drop`, `MBT_Undrop`, `MBT_DropEvery`, `MBT_UndropEvery`, `MBT_Info`, `Resize`, `RotateCrop`, `MBT_CorrectMatrix`, `Cropf`, `DeShake`, and `FixContrast`. `RotateCrop(angle, dar)` uses the external manyPlus `Turn` function, then crops the largest rectangle of the requested aspect ratio that contains no rotation background; `dar` defaults to the source aspect ratio. Source scripts inject the detected matrix into the standard `_Matrix` frame property. `Resize` reads `_Matrix` when `input_matrix` is not supplied, chooses an output matrix from the output resolution, updates `_Matrix`, and auto-enables linear-light resizing when either dimension is downscaled by 1.5x or more. The linear path requires `y_gamma_to_linear`/`y_linear_to_gamma`; pass `linear=false` or `linear=true` to override that decision. `DeShake` still requires the usual Avisynth stabilization plugins to be available if you call it.
Generated editable `.avs` files include source size/chroma/bitdepth, frame count, frame rate, duration, and timing classification. The hidden source scripts also generate `ConditionalReader` sidecar files for each source frame's relative video time, absolute timestamp, and frame duration, then attach those values as frame properties. Use `MBT_Info(last)` while editing to overlay the user-facing frame properties. The internal `mbt_source_id` property is still present for batch validation, but it is not shown by that helper.
Validation wrappers are named `[original filename].validate.avs`. Fast validation is the default: the runner sets an in-memory AviSynth variable before importing the wrapper, making the hidden source use a `BlankClip` with the source clip's format, frame count, and frame rate. This exercises frame selection/properties without decoding every source image. Real-frame validation uses the same wrapper but disables that variable. The bundled runner writes a temporary frame table containing `output_frame,source_id,source_frame,drop_frame,matrix`; Python reads it to check that frame identity survived the edited script, then removes it after successful validation.
Set `MBT_ENCODE_VIDEO=1` to encode after validation. This milestone encodes accepted CFR timelines, VFR timelines, CFR speed changes, `MBT_Drop` output, normal frame deletion such as `SelectEven()`, and mixed deletion/drop edits. When normal frame deletion removes source time, AAC audio is cut into matching segments before encoding; `MBT_Drop` durations keep audio continuous.
Validation also reports the actual Avisynth output frame rate and playback duration. If the script changes playback speed, for example with `AssumeFPS`, AAC audio is tempo-adjusted to match the encoded video duration.
After validation, interactive runs switch to an encoding review screen. The top of the screen lists the validated clips and edited beginning timestamps in UTC; below it, answered encoding settings are shown in a small table. Type `b` to go back. If a setting changes another setting's meaning, such as switching between x264 and x265, dependent answers such as CRF/profile/level are cleared and return to their codec-specific defaults.
x265 is the default video encoder. Profile and level can each be left unset, calculated as the minimum required for every clip individually, or selected manually as a requested floor. A manual value that is too low for one clip is upgraded only for that clip. Choices shown in red do not directly support every clip in the batch. x264 is unavailable when any validated Avisynth output is above 10-bit; output above 12-bit fails validation and is excluded from encoding. For x265, tier follows the level constraint: x265 tries Main tier first and may use High tier when the selected level requires it.
When every validated script preserves the complete source video timeline and its basic video properties, the video codec list also offers `copy`. This directly remuxes the source video stream, so it does not run Avisynth video effects. Audio may independently be copied or re-encoded. If a selected MP4 container cannot stream-copy a source video or audio codec, the container row shows a red warning and the run cannot start until you choose a compatible container or re-encode that stream. Use MKV for stream-copying formats such as PCM audio that are not supported by this MP4 workflow.
The repo includes source for a small bundled Avisynth runner in `tools/avisynth_runner/`. Build it on Linux with:
```bash ```bash
tools/avisynth_runner/build_linux.sh tools/avisynth_runner/build_linux.sh
``` ```
When `tools/avisynth_runner/mbt_avs_runner` exists, `video_encode.py` uses it automatically for validation, Y4M video piping, timeline-preserving `drop_frame` omission, and optional WAV audio rendering. Encoding shows the Avisynth rendering and encoder-output progress separately, so encoder lookahead is visible. To use a runner from another location, set `MBT_AVS_RUNNER` to that executable. On Windows, use the bundled `tools/avisynth_runner/mbt_avs_runner.exe` with installed AviSynth+. Set `MBT_AVS_RUNNER`, `MBT_AVS_RUNNER64`, or `MBT_AVS_RUNNER32` to override runner locations. Generated scripts normally rely on FFMS2 autoload; set `MBT_FFMS2_PLUGIN` to generate `LoadPlugin("...")`, or `MBT_ASK_FFMS2_PLUGIN=1` to ask interactively.
On Windows, the bundled 64-bit runner is `tools/avisynth_runner/mbt_avs_runner.exe`. It loads the installed 64-bit AviSynth+ runtime at execution time, so AviSynth+ still needs to be installed. The script automatically tries the bundled 64-bit runner first. If a bundled or configured 32-bit runner exists, it can be selected for a specific editable `.avs` by ending that file with a commented marker like `#32bit` or `# 32-bit`. Trailing blank lines are ignored. To override runner locations, set `MBT_AVS_RUNNER64`, `MBT_AVS_RUNNER32`, or `MBT_AVS_RUNNER`. #### Encoding
Piping Y4M through the bundled runner does not add video quality loss, because Y4M is uncompressed video. The tradeoff is that a raw video pipe does not carry audio, metadata, frame properties, or arbitrary per-frame timestamps; those are handled separately by Python. FFmpeg is used for encoding and muxing, not as the Avisynth host. - x265 is the default encoder. x264, x265, and unchanged-video stream copy are available when compatible with the validated output.
- The tool supports CFR, VFR, trims, playback-speed changes, `MBT_Drop`, normal frame deletion, and mixed deletion/drop edits.
- VFR and normal-deletion output preserves frame timing through timestamp-v2 files and MKVToolNix. x264 receives those timestamps through `--tcfile-in`; x265 receives them during the muxing step.
- Video filenames can use the edited beginning timestamp in a chosen timezone. Video container metadata is written with the edited ending timestamp. Meaningful source metadata, including location, is copied while physical stream properties remain specific to the output.
- Existing output names can be overwritten or given `-1`, `-2`, and so on. Same-batch collisions always receive a suffix.
- Profile and level can be unset, calculated as the per-clip minimum, or selected as a floor. x264 is unavailable for output above 10-bit, and output above 12-bit is rejected.
- MP4 and MKV support AAC, Opus, FLAC, copied audio, or no audio. Stream-copy compatibility is checked before encoding.
Audio is not piped together with Y4M video. By default, FFmpeg reads audio from the original source file and the Python tool applies the validated trim/speed plan through FFmpeg filters. If an optional `[visible script stem]_audio.avs` exists, the runner renders that script's audio to a temporary WAV with `--wav`, and FFmpeg uses that WAV as the audio source. This keeps video piping simple while still allowing Avisynth-based audio edits. Audio is read from the original source by default. An optional `[visible script stem]_audio.avs` is rendered to temporary WAV and used instead when present. AAC audio is trimmed and tempo-adjusted for the validated output. Audio stream copy is limited to whole-video and simple beginning/end trims, so normal frame deletion and speed changes require audio re-encoding or no audio.
For VFR output or normal frame deletion, the tool writes one timestamp-v2 sidecar and keeps only a compressed temporary video. x264 uses a raw H.264 intermediate and receives those timestamps during encoding through `--tcfile-in`, so its CRF rate control sees individual frame durations. x265 has no equivalent VFR rate-control input, so it encodes an intermediate Matroska video at the edited timeline's average frame rate. MKVToolNix then applies the exact frame timestamps to either compressed stream, and FFmpeg stream-copies the timed video while adding audio and writing the requested MP4 or MKV container. This avoids the previous temporary PNG sequence. After successful encoding, choose to leave files in `video_workspace/`, replace the originals, or move originals to `video_workspace/originals/` before moving the new files to the source directories.
Timestamp-driven output requires `mkvmerge` on `PATH`. On Windows, the standard `C:\Program Files\MKVToolNix\mkvmerge.exe` installation is also detected automatically. Timestamp-driven x264 output also requires the `x264` command-line encoder on `PATH`; CFR x264 output continues to use FFmpeg's `libx264` encoder. #### HDR
In an interactive run, if an external executable such as `mkvmerge`, `x264`, `ffmpeg`, `ffprobe`, or `exiftool` is missing from `PATH` and has no supported default location, the script asks for its absolute path. That path is used for the rest of the current batch only, so a late missing-tool discovery does not discard the validated workspace or encoding choices. HDR clips remain HDR unless the script calls `MBT_ToSDR()`. HDR-preserving x265 encodes retain validated color properties and static HDR metadata. HDR10+ additionally requires `hdr10plus_tool`; other dynamic HDR formats currently require video stream copy. `MBT_ToSDR()` uses HDRTools and offers `MPC` (default), `Hable`, `Mobius`, `Reinhard`, `ACES`, and `BT2446C`. Tone-mapped output should be visually checked before relying on it for archival work.
When encoding, the script can name outputs from the edited beginning timestamp, using nearest-second rounding and a chosen timezone. Before encoding begins, existing output names are listed and can be overwritten (default) or assigned `-1`, `-2`, and so on. Same-batch naming collisions always use that suffixing. Source metadata, including location metadata, is copied to the output while excluding physical media properties such as dimensions, duration, bitrate, frame rate, codec, and audio layout. MP4 output then gets video container timestamp tags written to the edited end timestamp in UTC. The encoded files are written next to the editable `.avs` scripts under `video_workspace/`. #### Non-interactive Encoding
Audio support currently covers validated outputs that the current encoder path supports: Set `MBT_ENCODE_VIDEO=1` to encode non-interactively after validation. The following optional variables configure that run:
- AAC re-encode trims audio to the validated output span and tempo-adjusts it when the Avisynth output duration changes.
- Normal frame deletion uses segmented AAC audio cuts so removed frame durations are removed from audio too.
- Audio copy is allowed for whole-file outputs and simple beginning/end trims. FFmpeg does input-side stream-copy trimming for that case, so the cut can be limited by packet/keyframe boundaries. Normal frame deletion and speed changes still require audio re-encoding or no audio.
- If copy is requested for a trimmed or speed-changed output, the output is video-only rather than risking bad sync.
- MP4 and MKV can use AAC, Opus, FLAC, copied audio, or no audio. AAC remains the most widely compatible MP4 audio choice; Opus and especially FLAC in MP4 may not play on older devices or applications.
- Optional audio edits are supported by copying the editable video `.avs` to `[visible script stem]_audio.avs` next to it. The normal generated source import already includes source audio when the video has audio, so the audio edit script can be the same script shape as the video edit script. Its video output is ignored, and its whole audio output is rendered to WAV, then trimmed by Python to match the validated video timeline.
After successful encodes, the script asks what to do with the finished files:
- leave them in `video_workspace/`
- move them to the source directories and delete the originals
- move them to the source directories and move originals into `video_workspace/originals/`
Non-interactive encode options:
- `MBT_VIDEO_TIMESTAMP_NAMES=1` names outputs as `YYYYMMDD_HHMMSS.mp4`; `0` keeps original-style workspace names. - `MBT_VIDEO_TIMESTAMP_NAMES=1` names outputs as `YYYYMMDD_HHMMSS.mp4`; `0` keeps original-style workspace names.
- `MBT_VIDEO_TIMEZONE=+03:00` selects the timezone used for timestamp filenames. If omitted, one consistent embedded source timezone is preferred; conflicting or missing source offsets fall back to the system timezone. No timezone is needed when timestamp filenames are disabled. - `MBT_VIDEO_TIMEZONE=+03:00` selects the timezone used for timestamp filenames. If omitted, one consistent embedded source timezone is preferred; conflicting or missing source offsets fall back to the system timezone. No timezone is needed when timestamp filenames are disabled.
@@ -194,7 +161,7 @@ Non-interactive encode options:
- `MBT_VIDEO_FINAL_ACTION=leave`, `replace`, or `backup` selects the final disposition. - `MBT_VIDEO_FINAL_ACTION=leave`, `replace`, or `backup` selects the final disposition.
- `MBT_VIDEO_DELETE_WORKSPACE=1` deletes `video_workspace` after moving outputs. - `MBT_VIDEO_DELETE_WORKSPACE=1` deletes `video_workspace` after moving outputs.
Supported initial video extensions are `.mp4`, `.mov`, `.mkv`, `.mts`, `.m2ts`, and `.avi`. Missing executables are requested by absolute path for the current run when possible. `mkvmerge` also checks the standard Windows installation path.
## Testing ## Testing
@@ -223,6 +190,7 @@ Current video scenarios copy fixture videos to temporary directories before proc
- FFmpeg available as `ffmpeg` and `ffprobe` on `PATH` for `video_encode.py` - FFmpeg available as `ffmpeg` and `ffprobe` on `PATH` for `video_encode.py`
- MKVToolNix available as `mkvmerge` on `PATH` for VFR or frame-deletion output (Windows also checks its standard installation path) - MKVToolNix available as `mkvmerge` on `PATH` for VFR or frame-deletion output (Windows also checks its standard installation path)
- the x264 command-line encoder available as `x264` on `PATH` for timestamp-aware x264 output - the x264 command-line encoder available as `x264` on `PATH` for timestamp-aware x264 output
- `hdr10plus_tool` available on `PATH` only when re-encoding HDR10+ video while retaining HDR
- FFMS2 available to Avisynth as `FFVideoSource` before generated `.avs` scripts can load real videos, either through plugin autoload or a `LoadPlugin` path - FFMS2 available to Avisynth as `FFVideoSource` before generated `.avs` scripts can load real videos, either through plugin autoload or a `LoadPlugin` path
- AviSynth+ development files and a C compiler are needed only if building the bundled runner from source - AviSynth+ development files and a C compiler are needed only if building the bundled runner from source
@@ -248,11 +216,7 @@ No virtual environment is required. The scripts use only the Python standard lib
`video_encode.py` creates files under `video_workspace/`, including encoded outputs when requested. It only modifies source directories or source videos if you explicitly choose a final action that moves outputs back to the source folders. `video_encode.py` creates files under `video_workspace/`, including encoded outputs when requested. It only modifies source directories or source videos if you explicitly choose a final action that moves outputs back to the source folders.
`tests/`, `old/`, `PLAN.md`, `VIDEO_PLAN.md`, and `video_workspace/` are intentionally ignored by git. The `tests/` directory is for reusable source fixtures that should not be modified directly. `tests/`, `old/`, and `video_workspace/` are intentionally ignored by git. The `tests/` directory is for reusable source fixtures that should not be modified directly.
## Later Work
`video_encode.py` will grow into the separate video tool for Avisynth editing, remuxing, re-encoding, audio handling, and timecode handling. Those features are intentionally not mixed into `photo_metadata.py`.
## AI Note ## AI Note
+2 -37
View File
@@ -24,7 +24,7 @@ from datetime import datetime, timedelta, timezone
from pathlib import Path from pathlib import Path
from typing import Iterable from typing import Iterable
from tools.console import pause_if_interactive, prompt_input from tools.console import pause_if_interactive, prompt_input, prompt_yes_no as ask_yes_no
from tools.exiftool import require_exiftool, run_exiftool_json, run_exiftool_write from tools.exiftool import require_exiftool, run_exiftool_json, run_exiftool_write
from tools.filenames import ( from tools.filenames import (
format_filename_stem, format_filename_stem,
@@ -40,6 +40,7 @@ from tools.filesystem import (
remove_empty_dirs, remove_empty_dirs,
unique_existing_target, unique_existing_target,
) )
from tools.timezones import parse_timezone_offset, timezone_to_string
IMAGE_EXTS = {".jpg", ".jpeg", ".heic", ".arw"} IMAGE_EXTS = {".jpg", ".jpeg", ".heic", ".arw"}
@@ -96,7 +97,6 @@ DATETIME_RE = re.compile(
TIME_ONLY_RE = re.compile( TIME_ONLY_RE = re.compile(
r"^(?P<h>\d{1,2})(?::?(?P<m>\d{2}))(?::?(?P<s>\d{2}))?$" r"^(?P<h>\d{1,2})(?::?(?P<m>\d{2}))(?::?(?P<s>\d{2}))?$"
) )
TZ_RE = re.compile(r"^(?P<sign>[+-])(?P<h>\d{1,2})(?::?(?P<m>\d{2}))?$")
OFFSET_RE = re.compile(r"^([+-])(?:(\d+):)?(\d{1,2})(?::(\d{2}))?$") OFFSET_RE = re.compile(r"^([+-])(?:(\d+):)?(\d{1,2})(?::(\d{2}))?$")
TOKEN_SHIFT_RE = re.compile(r"(\d+|[a-zA-Z]+)") TOKEN_SHIFT_RE = re.compile(r"(\d+|[a-zA-Z]+)")
ORDERED_TIMESTAMP_STEM_RE = re.compile(r"^(?P<timestamp>\d{8}_\d{6})-[1-9]\d*$") ORDERED_TIMESTAMP_STEM_RE = re.compile(r"^(?P<timestamp>\d{8}_\d{6})-[1-9]\d*$")
@@ -195,19 +195,6 @@ class UserChoices:
apply_filename_timestamps: bool apply_filename_timestamps: bool
def ask_yes_no(prompt: str, default: bool) -> bool:
suffix = "Y/n" if default else "y/N"
while True:
answer = prompt_input(f"{prompt} ({suffix}) ").strip().lower()
if not answer:
return default
if answer in {"y", "yes"}:
return True
if answer in {"n", "no"}:
return False
print("Please answer y or n.")
def ask_group_min_size() -> int: def ask_group_min_size() -> int:
while True: while True:
answer = prompt_input( answer = prompt_input(
@@ -225,28 +212,6 @@ def ask_group_min_size() -> int:
print("Please answer y, n, or an integer minimum group size.") print("Please answer y, n, or an integer minimum group size.")
def parse_timezone_offset(value: str) -> timezone:
match = TZ_RE.match(value.strip())
if not match:
raise ValueError("timezone must look like +09:00 or -05:30")
sign = 1 if match.group("sign") == "+" else -1
hours = int(match.group("h"))
minutes = int(match.group("m") or "0")
if minutes not in {0, 15, 30, 45}:
raise ValueError("timezone minutes must be 00, 15, 30, or 45")
return timezone(sign * timedelta(hours=hours, minutes=minutes))
def timezone_to_string(tz: timezone) -> str:
offset = tz.utcoffset(None)
if offset is None:
return "+00:00"
total_minutes = int(offset.total_seconds() // 60)
sign = "+" if total_minutes >= 0 else "-"
total_minutes = abs(total_minutes)
return f"{sign}{total_minutes // 60:02d}:{total_minutes % 60:02d}"
def parse_datetime(value: str, default_date: datetime | None = None) -> datetime: def parse_datetime(value: str, default_date: datetime | None = None) -> datetime:
text = value.strip() text = value.strip()
match = DATETIME_RE.search(text) match = DATETIME_RE.search(text)
+409 -64
View File
@@ -4,18 +4,43 @@
# small and dependency-light unless the visible scripts explicitly document the # small and dependency-light unless the visible scripts explicitly document the
# required plugins. # required plugins.
function MBT_MarkSource(clip c, int source_id, int "matrix") function MBT_MarkSource(clip c, int source_id, int "matrix", int "primaries", int "transfer", int "color_range", float "hdr_peak", float "absolute_timestamp_end_seconds", float "source_duration_seconds")
{ {
# Attach source identity to each frame. Python reads these properties after # Attach source identity to each frame. Python reads these properties after
# running the user-edited script and maps frames back to ffprobe timestamps. # running the user-edited script and maps frames back to ffprobe timestamps.
# Timestamp/duration variables are supplied by generated ConditionalReader # Timing values are attached separately after ConditionalReader has supplied
# files when available. # their per-frame variables.
matrix = Default(matrix, MBT_GetMatrixCode(c)) matrix = Defined(matrix) ? matrix : 2
primaries = Defined(primaries) ? primaries : 2
transfer = Defined(transfer) ? transfer : 2
color_range = Defined(color_range) ? color_range : 0
hdr_peak = Default(hdr_peak, 1000.0)
absolute_timestamp_end_seconds = Default(absolute_timestamp_end_seconds, 0.0)
source_duration_seconds = Default(source_duration_seconds, 0.0)
runtime = """ runtime = """
last.propSet("mbt_source_id", """ + String(source_id) + """).\ last.propSet("mbt_source_id", """ + String(source_id) + """).\
propSet("mbt_source_frame", current_frame).\ propSet("mbt_source_frame", current_frame).\
propSet("mbt_drop_frame", 0).\ propSet("mbt_drop_frame", 0).\
propSet("mbt_absolute_timestretch", 1.0).\
propSet("mbt_absolute_timestamp_end_seconds", """ + String(absolute_timestamp_end_seconds) + """).\
propSet("mbt_source_duration_seconds", """ + String(source_duration_seconds) + """).\
propSet("mbt_absolute_timestamp_start_seconds", """ + String(absolute_timestamp_end_seconds - source_duration_seconds) + """).\
propSet("_Matrix", """ + String(matrix) + """).\ propSet("_Matrix", """ + String(matrix) + """).\
propSet("mbt_hdr_peak", """ + String(hdr_peak) + """).\
propSet("_Primaries", """ + String(primaries) + """).\
propSet("_Transfer", """ + String(transfer) + """).\
propSet("_ColorRange", """ + String(color_range) + """)
"""
source = c.PropSet("mbt_hdr_peak", hdr_peak).\
MBT_SetColor(MBT_MatrixNameFromCode(matrix), primaries, transfer, color_range)
return source.ScriptClip(runtime)
}
function MBT_MarkTiming(clip c)
{
runtime = """
source_start = propGetFloat("mbt_absolute_timestamp_start_seconds")
last.propSet("mbt_absolute_timestamp_seconds", source_start + mbt_frame_time_seconds).\
propSet("mbt_frame_time_seconds", mbt_frame_time_seconds).\ propSet("mbt_frame_time_seconds", mbt_frame_time_seconds).\
propSet("mbt_relative_timestamp", mbt_relative_timestamp).\ propSet("mbt_relative_timestamp", mbt_relative_timestamp).\
propSet("mbt_absolute_timestamp", mbt_absolute_timestamp).\ propSet("mbt_absolute_timestamp", mbt_absolute_timestamp).\
@@ -25,6 +50,86 @@ function MBT_MarkSource(clip c, int source_id, int "matrix")
return c.ScriptClip(runtime) return c.ScriptClip(runtime)
} }
function MBT_AbsoluteTimeStretch(clip c, float factor)
{
Assert(factor > 0.0, "MBT_AbsoluteTimeStretch factor must be greater than zero.")
runtime = """
source_end = propGetFloat("mbt_absolute_timestamp_end_seconds")
source_duration = propGetFloat("mbt_source_duration_seconds")
previous_factor = propGetFloat("mbt_absolute_timestretch")
total_factor = previous_factor * """ + String(factor) + """
Assert(source_end > 0.0 && source_duration > 0.0, "MBT_AbsoluteTimeStretch requires source end timestamp and duration metadata.")
source_start = source_end - source_duration * total_factor
stretched_timestamp = source_start + mbt_frame_time_seconds * total_factor
last.propSet("mbt_absolute_timestamp_start_seconds", source_start).\
propSet("mbt_absolute_timestamp_seconds", stretched_timestamp).\
propSet("mbt_absolute_timestamp", MBT_FormatUtcTimestamp(stretched_timestamp)).\
propSet("mbt_absolute_timestretch", total_factor)
"""
return c.ScriptClip(runtime)
}
function MBT_IsLeapYear(int year)
{
return year % 4 == 0 && (year % 100 != 0 || year % 400 == 0)
}
function MBT_DaysInYear(int year)
{
return MBT_IsLeapYear(year) ? 366 : 365
}
function MBT_DaysInMonth(int year, int month)
{
return month == 2 ? (MBT_IsLeapYear(year) ? 29 : 28)
\ : (month == 4 || month == 6 || month == 9 || month == 11) ? 30 : 31
}
function MBT_YearFromEpochDays(int days, int "year")
{
year = Default(year, 1970)
return days < MBT_DaysInYear(year) ? year : MBT_YearFromEpochDays(days - MBT_DaysInYear(year), year + 1)
}
function MBT_DayOfYear(int days, int "year")
{
year = Default(year, 1970)
return days < MBT_DaysInYear(year) ? days : MBT_DayOfYear(days - MBT_DaysInYear(year), year + 1)
}
function MBT_MonthFromDayOfYear(int year, int day, int "month")
{
month = Default(month, 1)
return day < MBT_DaysInMonth(year, month) ? month : MBT_MonthFromDayOfYear(year, day - MBT_DaysInMonth(year, month), month + 1)
}
function MBT_DayOfMonth(int year, int day, int "month")
{
month = Default(month, 1)
return day < MBT_DaysInMonth(year, month) ? day + 1 : MBT_DayOfMonth(year, day - MBT_DaysInMonth(year, month), month + 1)
}
function MBT_Pad(int value, int width)
{
text = String(value)
return StrLen(text) >= width ? text : "0" + MBT_Pad(value, width - 1)
}
function MBT_FormatUtcTimestamp(float timestamp)
{
whole_seconds = Int(Floor(timestamp))
microseconds = Round((timestamp - whole_seconds) * 1000000.0)
whole_seconds = microseconds >= 1000000 ? whole_seconds + 1 : whole_seconds
microseconds = microseconds >= 1000000 ? 0 : microseconds
days = Int(Floor(whole_seconds / 86400.0))
day_seconds = whole_seconds - days * 86400
year = MBT_YearFromEpochDays(days)
day = MBT_DayOfYear(days)
month = MBT_MonthFromDayOfYear(year, day)
return MBT_Pad(year, 4) + "-" + MBT_Pad(month, 2) + "-" + MBT_Pad(MBT_DayOfMonth(year, day), 2) + "T" + \
MBT_Pad(day_seconds / 3600, 2) + ":" + MBT_Pad((day_seconds % 3600) / 60, 2) + ":" + MBT_Pad(day_seconds % 60, 2) + "." + MBT_Pad(microseconds, 6) + "Z"
}
function MBT_AudioSource(clip video, string source_path, string audio_cache_path) function MBT_AudioSource(clip video, string source_path, string audio_cache_path)
{ {
audio = FFAudioSource(source_path, cachefile=audio_cache_path) audio = FFAudioSource(source_path, cachefile=audio_cache_path)
@@ -94,12 +199,17 @@ function MBT_Info(clip c, int "size", int "align")
align = Default(align, 7) align = Default(align, 7)
runtime = """ runtime = """
drop_value = propGetInt("mbt_drop_frame") == 0 ? "False" : "True" drop_value = propGetInt("mbt_drop_frame") == 0 ? "False" : "True"
matrix = MBT_MatrixNameFromCode(propGetInt("_Matrix"))
transfer = propGetInt("_Transfer")
transfer_name = transfer == 16 ? "PQ" : (transfer == 18 ? "HLG" : "SDR")
range_name = propGetInt("_ColorRange") == 1 ? "full range" : "limited range"
text = "Absolute timestamp: " + propGetString("mbt_absolute_timestamp") + Chr(10) + \ text = "Absolute timestamp: " + propGetString("mbt_absolute_timestamp") + Chr(10) + \
"Relative timestamp: " + propGetString("mbt_relative_timestamp") + Chr(10) + \ "Relative timestamp: " + propGetString("mbt_relative_timestamp") + Chr(10) + \
Chr(10) + \ Chr(10) + \
"Source frame: " + String(propGetInt("mbt_source_frame")) + Chr(10) + \ "Source frame: " + String(propGetInt("mbt_source_frame")) + Chr(10) + \
"Frame duration: " + propGetString("mbt_frame_duration") + Chr(10) + \ "Frame duration: " + propGetString("mbt_frame_duration") + Chr(10) + \
"Drop frame: " + drop_value "Drop frame: " + drop_value + Chr(10) + \
"Color: " + matrix + ", " + transfer_name + ", " + range_name
Subtitle(last, text, size=""" + String(size) + """, align=""" + String(align) + """) Subtitle(last, text, size=""" + String(size) + """, align=""" + String(align) + """)
""" """
return c.ScriptClip(runtime) return c.ScriptClip(runtime)
@@ -176,8 +286,8 @@ function Resize(clip v, int "w", int "h", float "dar", float "xcrop", float "ycr
crop_w = cropped_w - crop_extra_l - crop_extra_r crop_w = cropped_w - crop_extra_l - crop_extra_r
crop_h = cropped_h - crop_extra_t - crop_extra_b crop_h = cropped_h - crop_extra_t - crop_extra_b
input_matrix = Default(input_matrix, v.IsRGB ? MBT_DefaultMatrix(wo, ho) : MBT_GetMatrixName(v)) input_matrix = Default(input_matrix, v.IsRGB ? MBT_DefaultMatrix(wo, ho) : mbt_color_matrix)
output_matrix = Default(output_matrix, MBT_DefaultMatrix(w, h)) output_matrix = Default(output_matrix, MBT_DefaultOutputMatrix(w, h))
return v.MBT_LinearResize(w, h, crop_l, crop_t, crop_w, crop_h, linear=linear, input_matrix=input_matrix, output_matrix=output_matrix) return v.MBT_LinearResize(w, h, crop_l, crop_t, crop_w, crop_h, linear=linear, input_matrix=input_matrix, output_matrix=output_matrix)
} }
@@ -200,51 +310,85 @@ function MBT_LinearResize(clip src, int tw, int th,
p = Default(p , 30.0) p = Default(p , 30.0)
bitdepth = Default(bitdepth , src.BitsPerComponent) bitdepth = Default(bitdepth , src.BitsPerComponent)
linear = Defined(linear) ? linear : (sw / Float(tw) >= 1.5 || sh / Float(th) >= 1.5) linear = Defined(linear) ? linear : (sw / Float(tw) >= 1.5 || sh / Float(th) >= 1.5)
input_matrix = Default(input_matrix, src.IsRGB ? MBT_DefaultMatrix(src.Width, src.Height) : MBT_GetMatrixName(src)) input_matrix = Default(input_matrix, src.IsRGB ? MBT_DefaultMatrix(src.Width, src.Height) : mbt_color_matrix)
output_matrix = Default(output_matrix, MBT_DefaultMatrix(tw, th)) output_matrix = Default(output_matrix, MBT_DefaultOutputMatrix(tw, th))
Assert(
\ !linear || (FunctionExists("y_gamma_to_linear") && FunctionExists("y_linear_to_gamma")),
\ "Resize(linear=true) requires y_gamma_to_linear and y_linear_to_gamma. Load the missing dependency or call Resize(..., linear=false)."
\)
clp = (src.IsRGB ? (src.HasAlpha ? src.ConvertToPlanarRGBA return MBT_IsHDR()
\ : src.ConvertToPlanarRGB) \ ? (linear
\ : src.ConvertToPlanarRGB(matrix=input_matrix)) \ ? src.MBT_HDRLinearResize(tw, th, sl, st, sw, sh, hkernel, vkernel, b, c, taps, p, bitdepth, input_matrix, output_matrix)
clp = clp.ConvertBits(32) \ : src.MBT_HDRResize(tw, th, sl, st, sw, sh, hkernel, vkernel, b, c, taps, p, input_matrix, output_matrix))
\ : src.MBT_SDRResize(tw, th, sl, st, sw, sh, hkernel, vkernel, b, c, taps, p, bitdepth, linear, input_matrix, output_matrix)
}
function MBT_HDRResize(clip src, int tw, int th, float sl, float st, float sw, float sh,
\ string hkernel, string vkernel, float b, float c, int "taps", float "p",
\ string "input_matrix", string "output_matrix")
{
Assert(input_matrix == output_matrix,
\ "HDR Resize preserves its color system. Use MBT_ToSDR before Resize to change it.")
return src.MBT_ResizePlanes(tw, th, sl, st, sw, sh, hkernel, vkernel, b, c, taps, p).PropCopy(src)
}
function MBT_SDRResize(clip src, int tw, int th, float sl, float st, float sw, float sh,
\ string hkernel, string vkernel, float b, float c, int "taps", float "p", int "bitdepth",
\ bool "linear", string "input_matrix", string "output_matrix")
{
Assert(!linear || (FunctionExists("y_gamma_to_linear") && FunctionExists("y_linear_to_gamma")),
\ "Resize(linear=true) requires y_gamma_to_linear and y_linear_to_gamma. Load the missing dependency or call Resize(..., linear=false).")
clp = (src.IsRGB ? (src.HasAlpha ? src.ConvertToPlanarRGBA : src.ConvertToPlanarRGB) : src.ConvertToPlanarRGB(matrix=input_matrix)).ConvertBits(32)
clp = linear ? clp.y_gamma_to_linear : clp clp = linear ? clp.y_gamma_to_linear : clp
clp = clp.MBT_ResizePlanes(tw, th, sl, st, sw, sh, hkernel, vkernel, b, c, taps, p)
if (hkernel != vkernel) {
htaps = Default(taps, (hkernel == "Lanczos" ? 3 : 4))
vtaps = Default(taps, (vkernel == "Lanczos" ? 3 : 4))
clp = clp.MBT_LinearResize(tw, src.Height, sl, 0, sw, 0, hkernel, b, c, htaps, p, linear=false, input_matrix=input_matrix, output_matrix=input_matrix)
clp = clp.MBT_LinearResize(tw, th, 0, st, 0, sh, vkernel, b, c, vtaps, p, linear=false, input_matrix=input_matrix, output_matrix=output_matrix)
}
else {
taps = Default(taps, (hkernel == "Lanczos" ? 3 : 4))
clp =
\ (hkernel == "Bicubic" ) ? clp.BicubicResize(tw, th, b=b, c=c, src_left=sl, src_top=st, src_width=sw, src_height=sh)
\ : (hkernel == "Bilinear" ) ? clp.BilinearResize(tw, th, src_left=sl, src_top=st, src_width=sw, src_height=sh)
\ : (hkernel == "Blackman" ) ? clp.BlackmanResize(tw, th, src_left=sl, src_top=st, src_width=sw, src_height=sh, taps=taps)
\ : (hkernel == "Gauss" ) ? clp.GaussResize(tw, th, src_left=sl, src_top=st, src_width=sw, src_height=sh, p=p)
\ : (hkernel == "Lanczos" ) ? clp.LanczosResize(tw, th, src_left=sl, src_top=st, src_width=sw, src_height=sh, taps=taps)
\ : (hkernel == "Lanczos4" ) ? clp.Lanczos4Resize(tw, th, src_left=sl, src_top=st, src_width=sw, src_height=sh)
\ : (hkernel == "Point" ) ? clp.PointResize(tw, th, src_left=sl, src_top=st, src_width=sw, src_height=sh)
\ : (hkernel == "Sinc" ) ? clp.SincResize(tw, th, src_left=sl, src_top=st, src_width=sw, src_height=sh, taps=taps)
\ : (hkernel == "Spline16" ) ? clp.Spline16Resize(tw, th, src_left=sl, src_top=st, src_width=sw, src_height=sh)
\ : (hkernel == "Spline36" ) ? clp.Spline36Resize(tw, th, src_left=sl, src_top=st, src_width=sw, src_height=sh)
\ : (hkernel == "Spline64" ) ? clp.Spline64Resize(tw, th, src_left=sl, src_top=st, src_width=sw, src_height=sh)
\ : (hkernel == "CatmullRom" ) ? clp.BicubicResize(tw, th, b=0.0, c=0.5, src_left=sl, src_top=st, src_width=sw, src_height=sh)
\ : Assert(false, "Invalid resize kernel.")
}
clp = linear ? clp.y_linear_to_gamma : clp clp = linear ? clp.y_linear_to_gamma : clp
clp = src.IsRGB ? (src.HasAlpha ? clp.ConvertToPlanarRGBA : clp.ConvertToPlanarRGB) clp = src.IsRGB ? (src.HasAlpha ? clp.ConvertToPlanarRGBA : clp.ConvertToPlanarRGB)
\ : (src.Is420 ? clp.ConvertToYUV420(matrix=output_matrix).ConvertBits(bitdepth).MBT_SetMatrix(output_matrix) \ : (src.Is420 ? clp.ConvertToYUV420(matrix=output_matrix)
\ : (src.Is422 ? clp.ConvertToYUV422(matrix=output_matrix).ConvertBits(bitdepth).MBT_SetMatrix(output_matrix) \ : (src.Is422 ? clp.ConvertToYUV422(matrix=output_matrix) : clp.ConvertToYUV444(matrix=output_matrix)))
\ : (src.Is444 ? clp.ConvertToYUV444(matrix=output_matrix).ConvertBits(bitdepth).MBT_SetMatrix(output_matrix) : clp.ConvertBits(bitdepth).MBT_SetMatrix(output_matrix)))) global mbt_color_matrix = output_matrix
clp = src.IsRGB ? clp.ConvertBits(bitdepth) : clp global mbt_color_primaries = MBT_PrimariesForMatrix(output_matrix)
global mbt_color_transfer = 1
return src.IsRGB ? clp.ConvertBits(bitdepth).PropCopy(src) : clp.ConvertBits(bitdepth).PropCopy(src).MBT_SetSDRColor(output_matrix, mbt_color_range)
}
return clp function MBT_HDRLinearResize(clip src, int tw, int th, float sl, float st, float sw, float sh,
\ string hkernel, string vkernel, float b, float c, int "taps", float "p", int "bitdepth",
\ string "input_matrix", string "output_matrix")
{
Assert(!src.IsRGB, "HDR Resize currently requires planar YUV input.")
Assert(input_matrix == output_matrix,
\ "HDR Resize preserves its color system. Use MBT_ToSDR before Resize to change it.")
Assert(mbt_color_primaries == 9,
\ "HDR Resize(linear=true) currently requires Rec2020 primaries. P3 HDR needs mastering-primary support.")
Assert(FunctionExists("ConvertYUVtoLinearRGB") && FunctionExists("ConvertLinearRGBtoYUV"),
\ "HDR Resize(linear=true) requires HDRTools. Load HDRTools or call Resize(..., linear=false).")
clp = src.MBT_LinearHDR(MBT_HDRMode(), mbt_color_range)
clp = clp.MBT_ResizePlanes(tw, th, sl, st, sw, sh, hkernel, vkernel, b, c, taps, p)
clp = clp.ConvertLinearRGBtoYUV(Color=0, OutputMode=MBT_YuvOutputMode(src), HDRMode=MBT_HDRMode(), OOTF=false, fullrange=mbt_color_range == 1)
return clp.ConvertBits(bitdepth).PropCopy(src).MBT_SetColor(input_matrix, mbt_color_primaries, mbt_color_transfer, mbt_color_range)
}
function MBT_ResizePlanes(clip clp, int tw, int th, float sl, float st, float sw, float sh,
\ string hkernel, string vkernel, float b, float c, int "taps", float "p")
{
if (hkernel != vkernel) {
htaps = Default(taps, hkernel == "Lanczos" ? 3 : 4)
vtaps = Default(taps, vkernel == "Lanczos" ? 3 : 4)
clp = clp.MBT_ResizePlanes(tw, clp.Height, sl, 0, sw, clp.Height, hkernel, hkernel, b, c, htaps, p)
return clp.MBT_ResizePlanes(tw, th, 0, st, clp.Width, sh, vkernel, vkernel, b, c, vtaps, p)
}
taps = Default(taps, hkernel == "Lanczos" ? 3 : 4)
return
\ (hkernel == "Bicubic" ) ? clp.BicubicResize(tw, th, b=b, c=c, src_left=sl, src_top=st, src_width=sw, src_height=sh)
\ : (hkernel == "Bilinear" ) ? clp.BilinearResize(tw, th, src_left=sl, src_top=st, src_width=sw, src_height=sh)
\ : (hkernel == "Blackman" ) ? clp.BlackmanResize(tw, th, src_left=sl, src_top=st, src_width=sw, src_height=sh, taps=taps)
\ : (hkernel == "Gauss" ) ? clp.GaussResize(tw, th, src_left=sl, src_top=st, src_width=sw, src_height=sh, p=p)
\ : (hkernel == "Lanczos" ) ? clp.LanczosResize(tw, th, src_left=sl, src_top=st, src_width=sw, src_height=sh, taps=taps)
\ : (hkernel == "Lanczos4" ) ? clp.Lanczos4Resize(tw, th, src_left=sl, src_top=st, src_width=sw, src_height=sh)
\ : (hkernel == "Point" ) ? clp.PointResize(tw, th, src_left=sl, src_top=st, src_width=sw, src_height=sh)
\ : (hkernel == "Sinc" ) ? clp.SincResize(tw, th, src_left=sl, src_top=st, src_width=sw, src_height=sh, taps=taps)
\ : (hkernel == "Spline16" ) ? clp.Spline16Resize(tw, th, src_left=sl, src_top=st, src_width=sw, src_height=sh)
\ : (hkernel == "Spline36" ) ? clp.Spline36Resize(tw, th, src_left=sl, src_top=st, src_width=sw, src_height=sh)
\ : (hkernel == "Spline64" ) ? clp.Spline64Resize(tw, th, src_left=sl, src_top=st, src_width=sw, src_height=sh)
\ : (hkernel == "CatmullRom" ) ? clp.BicubicResize(tw, th, b=0.0, c=0.5, src_left=sl, src_top=st, src_width=sw, src_height=sh)
\ : Assert(false, "Invalid resize kernel.")
} }
function MBT_DefaultMatrix(int w, int h) function MBT_DefaultMatrix(int w, int h)
@@ -252,6 +396,58 @@ function MBT_DefaultMatrix(int w, int h)
return (w >= 3840 || h >= 2160) ? "Rec2020" : ((w >= 1280 || h >= 720) ? "Rec709" : "Rec601") return (w >= 3840 || h >= 2160) ? "Rec2020" : ((w >= 1280 || h >= 720) ? "Rec709" : "Rec601")
} }
function MBT_IsHDR()
{
return mbt_color_transfer == 16 || mbt_color_transfer == 18
}
function MBT_DefaultOutputMatrix(int w, int h)
{
return MBT_IsHDR() ? mbt_color_matrix : MBT_DefaultMatrix(w, h)
}
function MBT_GetPrimariesCode(clip c)
{
primaries = c.PropGetInt("_Primaries")
return primaries == 0 ? 2 : primaries
}
function MBT_GetTransferCode(clip c)
{
transfer = c.PropGetInt("_Transfer")
return transfer == 0 ? 2 : transfer
}
function MBT_GetRangeCode(clip c)
{
return c.PropGetInt("_ColorRange")
}
function MBT_PrimariesForMatrix(string matrix)
{
return matrix == "Rec2020" ? 9 : 1
}
function MBT_SDRColorForMatrix(string matrix)
{
return matrix == "Rec2020" ? 1 : (matrix == "Rec601" ? 3 : 2)
}
function MBT_YuvOutputMode(clip c)
{
return c.Is420 ? 2 : (c.Is422 ? 1 : 0)
}
function MBT_LinearHDR(clip c, int hdr_mode, int color_range)
{
return c.ConvertYUVtoLinearRGB(Color=0, OutputMode=2, HDRMode=hdr_mode, OOTF=false, fullrange=color_range == 1)
}
function MBT_HDRMode()
{
return mbt_color_transfer == 18 ? 2 : 0
}
function MBT_MatrixNameFromCode(int matrix) function MBT_MatrixNameFromCode(int matrix)
{ {
return (matrix == 1) ? "Rec709" return (matrix == 1) ? "Rec709"
@@ -278,33 +474,182 @@ function MBT_MatrixInt(string matrix)
\ : 2 \ : 2
} }
function MBT_GetMatrixCode(clip c)
{
matrix = MBT_MatrixNameFromCode(c.PropGetInt("_Matrix"))
return (matrix == "") ? MBT_MatrixInt(MBT_DefaultMatrix(c.Width, c.Height)) : c.PropGetInt("_Matrix")
}
function MBT_GetMatrixName(clip c)
{
matrix = MBT_MatrixNameFromCode(MBT_GetMatrixCode(c))
return (matrix == "") ? MBT_DefaultMatrix(c.Width, c.Height) : matrix
}
function MBT_SetMatrix(clip c, string matrix) function MBT_SetMatrix(clip c, string matrix)
{ {
return c.PropSet("_Matrix", MBT_MatrixInt(matrix)) return c.PropSet("_Matrix", MBT_MatrixInt(matrix))
} }
function MBT_SetColor(clip c, string matrix, int primaries, int transfer, int color_range)
{
return c.PropSet("_Matrix", MBT_MatrixInt(matrix)).\
PropSet("_Primaries", primaries).\
PropSet("_Transfer", transfer).\
PropSet("_ColorRange", color_range)
}
function MBT_SetSDRColor(clip c, string matrix, int color_range)
{
return c.MBT_SetColor(matrix, MBT_PrimariesForMatrix(matrix), 1, color_range)
}
function MBT_CorrectMatrix(clip c, string "input_matrix", string "output_matrix") function MBT_CorrectMatrix(clip c, string "input_matrix", string "output_matrix")
{ {
input_matrix = Default(input_matrix, MBT_GetMatrixName(c)) input_matrix = Default(input_matrix, mbt_color_matrix)
output_matrix = Default(output_matrix, MBT_DefaultMatrix(c.Width, c.Height)) output_matrix = Default(output_matrix, MBT_DefaultOutputMatrix(c.Width, c.Height))
was_hdr = MBT_IsHDR()
Assert(!was_hdr || input_matrix == output_matrix,
\ "MBT_CorrectMatrix does not convert HDR color systems. Use MBT_ToSDR or an explicit HDR conversion.")
if (!was_hdr) {
global mbt_color_matrix = output_matrix
global mbt_color_primaries = MBT_PrimariesForMatrix(output_matrix)
global mbt_color_transfer = 1
}
return was_hdr ? c.MBT_SetMatrix(input_matrix) : c.MBT_CorrectSDRMatrix(input_matrix, output_matrix)
}
function MBT_CorrectSDRMatrix(clip c, string input_matrix, string output_matrix)
{
matrix = MBT_MatrixCodeForConvert(input_matrix) + ":auto=>" + MBT_MatrixCodeForConvert(output_matrix) + ":same" matrix = MBT_MatrixCodeForConvert(input_matrix) + ":auto=>" + MBT_MatrixCodeForConvert(output_matrix) + ":same"
bitdepth = c.BitsPerComponent bitdepth = c.BitsPerComponent
return c.IsRGB ? c return c.IsRGB ? c
\ : (c.Is420 ? c.ConvertToYUV420(matrix=matrix).ConvertBits(bitdepth).MBT_SetMatrix(output_matrix) \ : (c.Is420 ? c.ConvertToYUV420(matrix=matrix).ConvertBits(bitdepth).PropCopy(c).MBT_SetSDRColor(output_matrix, mbt_color_range)
\ : (c.Is422 ? c.ConvertToYUV422(matrix=matrix).ConvertBits(bitdepth).MBT_SetMatrix(output_matrix) \ : (c.Is422 ? c.ConvertToYUV422(matrix=matrix).ConvertBits(bitdepth).PropCopy(c).MBT_SetSDRColor(output_matrix, mbt_color_range)
\ : (c.Is444 ? c.ConvertToYUV444(matrix=matrix).ConvertBits(bitdepth).MBT_SetMatrix(output_matrix) : c.MBT_SetMatrix(output_matrix)))) \ : (c.Is444 ? c.ConvertToYUV444(matrix=matrix).ConvertBits(bitdepth).PropCopy(c).MBT_SetSDRColor(output_matrix, mbt_color_range) : c.MBT_SetSDRColor(output_matrix, mbt_color_range))))
}
function MBT_ToSDR(clip c, float "hdr_peak", float "sdr_peak", string "method", float "exposure", bool "chroma_correction", string "output_matrix")
{
transfer = MBT_GetTransferCode(c)
primaries = MBT_GetPrimariesCode(c)
color_range = MBT_GetRangeCode(c)
hdr_mode = transfer == 18 ? 2 : 0
Assert(transfer == 16 || transfer == 18, "MBT_ToSDR requires PQ or HLG input.")
Assert(!c.IsRGB, "MBT_ToSDR currently requires planar YUV input.")
Assert(primaries == 9,
\ "MBT_ToSDR currently requires Rec2020 primaries. P3 HDR needs mastering-primary support.")
method = Default(method, "MPC")
Assert(
\ method == "MPC" || method == "Hable" || method == "Mobius" || method == "Reinhard" || method == "ACES" || method == "BT2446C",
\ "MBT_ToSDR method must be MPC, Hable, Mobius, Reinhard, ACES, or BT2446C.")
chroma_correction = Default(chroma_correction, false)
hdr_peak = Default(hdr_peak, hdr_mode == 0 ? 10000.0 : 1000.0)
sdr_peak = Default(sdr_peak, 125.0)
output_matrix = Default(output_matrix, MBT_DefaultMatrix(c.Width, c.Height))
return method == "BT2446C"
\ ? c.MBT_ToSDR_BT2446C(hdr_mode, color_range, hdr_peak, sdr_peak, chroma_correction, output_matrix)
\ : method == "MPC"
\ ? c.MBT_ToSDR_MPC(hdr_mode, color_range, hdr_peak, sdr_peak, exposure, output_matrix)
\ : c.MBT_ToSDR_RGB(hdr_mode, color_range, method, exposure, output_matrix)
}
function MBT_ToSDR_BT2446C(clip c, int hdr_mode, int color_range, float hdr_peak, float sdr_peak, bool chroma_correction, string output_matrix)
{
Assert(FunctionExists("ConvertYUVtoXYZ") && FunctionExists("ConvertXYZtoYUV") && FunctionExists("ConverXYZ_BT2446_C_HDRtoSDR"),
\ "MBT_ToSDR requires HDRTools. Load HDRTools before calling it.")
output_color = MBT_SDRColorForMatrix(output_matrix)
linear = c.ConvertYUVtoXYZ(Color=0, OutputMode=2, HDRMode=hdr_mode, OOTF=false, fullrange=color_range == 1)
linear = linear.ConverXYZ_BT2446_C_HDRtoSDR(ChromaC=chroma_correction, PQMode=hdr_mode == 0, Lhdr=hdr_peak, Lsdr=sdr_peak, pColor=0)
result = linear.ConvertXYZtoYUV(Color=output_color, OutputMode=MBT_YuvOutputMode(c), OOTF=false, fullrange=false, pColor=0)
return result.MBT_SetToSDRProperties(c, output_matrix)
}
function MBT_ToSDR_RGB(clip c, int hdr_mode, int color_range, string method, float "exposure", string "output_matrix")
{
Assert(FunctionExists("ConvertYUVtoLinearRGB") && FunctionExists("ConvertRGBtoXYZ") && FunctionExists("ConvertXYZtoYUV"),
\ "MBT_ToSDR requires HDRTools. Load HDRTools before calling it.")
Assert(
\ (method == "Hable" && FunctionExists("ConvertRGB_Hable_HDRtoSDR")) ||
\ (method == "Mobius" && FunctionExists("ConvertRGB_Mobius_HDRtoSDR")) ||
\ (method == "Reinhard" && FunctionExists("ConvertRGB_Reinhard_HDRtoSDR")) ||
\ (method == "ACES" && FunctionExists("ConvertRGB_ACES_HDRtoSDR")),
\ "MBT_ToSDR method is not provided by the loaded HDRTools plugin.")
linear = c.MBT_LinearHDR(hdr_mode, color_range)
scale = Defined(exposure) ? exposure : (method == "ACES" ? 1.0 : 2.0)
mapped = method == "Hable" ? linear.ConvertRGB_Hable_HDRtoSDR(exposure_R=scale)
\ : method == "Mobius" ? linear.ConvertRGB_Mobius_HDRtoSDR(exposure_R=scale)
\ : method == "Reinhard" ? linear.ConvertRGB_Reinhard_HDRtoSDR(exposure_R=scale)
\ : linear.ConvertRGB_ACES_HDRtoSDR(exposure_R=scale)
xyz = mapped.ConvertRGBtoXYZ(Color=1, OOTF=false, EOTF=false)
result = xyz.ConvertXYZtoYUV(Color=MBT_SDRColorForMatrix(output_matrix), OutputMode=MBT_YuvOutputMode(c), OOTF=false, fullrange=false, pColor=1)
return result.MBT_SetToSDRProperties(c, output_matrix)
}
function MBT_ToSDR_MPC(clip c, int hdr_mode, int color_range, float hdr_peak, float sdr_peak, float "exposure", string "output_matrix")
{
Assert(FunctionExists("ConvertYUVtoLinearRGB") && FunctionExists("ConvertRGBtoXYZ") && FunctionExists("ConvertXYZtoRGB") && FunctionExists("ConvertLinearRGBtoYUV"),
\ "MBT_ToSDR requires HDRTools. Load HDRTools before calling it.")
scale = Defined(exposure) ? exposure : hdr_peak / sdr_peak
output_color = MBT_SDRColorForMatrix(output_matrix)
linear = c.MBT_LinearHDR(hdr_mode, color_range)
xyz = linear.ConvertRGBtoXYZ(Color=1, OOTF=false, EOTF=false)
xyz = xyz.MBT_HableLuminance(scale)
rgb = xyz.ConvertXYZtoRGB(Color=output_color, OutputMode=0, OOTF=false, EOTF=false, pColor=1)
rgb = rgb.MBT_GamutMap(output_matrix)
result = rgb.ConvertLinearRGBtoYUV(Color=output_color, OutputMode=MBT_YuvOutputMode(c), OOTF=false, EOTF=true, fullrange=false)
return result.MBT_SetToSDRProperties(c, output_matrix)
}
function MBT_HableLuminance(clip xyz, float exposure)
{
luminance = CombinePlanes(xyz, planes="Y", source_planes="G", pixel_type="Y32")
mapped = luminance.Expr(MBT_HableExpression(exposure), format="Y32", clamp_float=true)
scale = Expr(luminance, mapped, "y x 0.000001 max /", format="Y32", clamp_float=false)
scale = MergeRGB(scale, scale, scale, pixel_type="RGBPS")
return Expr(xyz, scale, "x y *", "x y *", "x y *", format="RGBPS", clamp_float=false).PropCopy(xyz)
}
function MBT_HableExpression(float exposure)
{
return "x " + String(exposure) + " * T^ " + \
"T T 0.15 * 0.05 + * 0.004 + N^ " + \
"N T T 0.15 * 0.5 + * 0.06 + / 0.06666666666666667 - " + String(MBT_HableWhiteScale()) + " * 0 max 1 min"
}
function MBT_GamutMap(clip rgb, string matrix)
{
red = CombinePlanes(rgb, planes="Y", source_planes="R", pixel_type="Y32")
green = CombinePlanes(rgb, planes="Y", source_planes="G", pixel_type="Y32")
blue = CombinePlanes(rgb, planes="Y", source_planes="B", pixel_type="Y32")
luma = MBT_LumaExpression(matrix)
return Expr(
\ red, green, blue,
\ MBT_GamutExpression("x", luma),
\ MBT_GamutExpression("y", luma),
\ MBT_GamutExpression("z", luma),
\ format="RGBPS",
\ clamp_float=true
\).PropCopy(rgb)
}
function MBT_LumaExpression(string matrix)
{
return matrix == "Rec601" ? "x 0.299 * y 0.587 * + z 0.114 * +"
\ : matrix == "Rec2020" ? "x 0.2627 * y 0.678 * + z 0.0593 * +"
\ : "x 0.2126 * y 0.7152 * + z 0.0722 * +"
}
function MBT_GamutExpression(string channel, string luma)
{
return luma + " L^ " + \
"x y min z min N^ x y max z max X^ " + \
"X 1 > 1 L - X L - / 1 ? S^ " + \
"N 0 < L L N - / 1 ? T^ S T min S^ " + \
channel + " L - S * L +"
}
function MBT_HableWhiteScale()
{
# 1 / hable(4.8), using the normalized Hable curve constants.
return 1.7896902226524685
}
function MBT_SetToSDRProperties(clip c, clip source, string output_matrix)
{
global mbt_color_matrix = output_matrix
global mbt_color_primaries = MBT_PrimariesForMatrix(output_matrix)
global mbt_color_transfer = 1
global mbt_color_range = 0
return c.ConvertBits(source.BitsPerComponent).PropCopy(source).MBT_SetSDRColor(output_matrix, 0)
} }
function DeShake(clip c, int "w", int "h", float "dar", float "crop", float "xcrop", float "ycrop", int "pos", bool "zoom", bool "rot", float "freq") function DeShake(clip c, int "w", int "h", float "dar", float "crop", float "xcrop", float "ycrop", int "pos", bool "zoom", bool "rot", float "freq")
+5
View File
@@ -0,0 +1,5 @@
from __future__ import annotations
def avs_path(path: str) -> str:
return path.replace("\\", "/").replace('"', '\\"')
+51 -2
View File
@@ -36,6 +36,7 @@ static AVS_Library* avs_library = NULL;
#define avs_is_yv16 avs_library->avs_is_yv16 #define avs_is_yv16 avs_library->avs_is_yv16
#define avs_is_yv24 avs_library->avs_is_yv24 #define avs_is_yv24 avs_library->avs_is_yv24
#define avs_prop_get_int avs_library->avs_prop_get_int #define avs_prop_get_int avs_library->avs_prop_get_int
#define avs_prop_get_float avs_library->avs_prop_get_float
#define avs_release_clip avs_library->avs_release_clip #define avs_release_clip avs_library->avs_release_clip
#define avs_release_value avs_library->avs_release_value #define avs_release_value avs_library->avs_release_value
#define avs_release_video_frame avs_library->avs_release_video_frame #define avs_release_video_frame avs_library->avs_release_video_frame
@@ -217,6 +218,7 @@ static int require_frame_property_api(FILE* log) {
if ( if (
avs_get_frame_props_ro avs_get_frame_props_ro
&& avs_prop_get_int && avs_prop_get_int
&& avs_prop_get_float
) { ) {
return 1; return 1;
} }
@@ -322,6 +324,13 @@ static int read_int_frame_prop(
int64_t* value_out int64_t* value_out
); );
static int read_float_frame_prop(
AVS_ScriptEnvironment* env,
const AVS_VideoFrame* frame,
const char* name,
double* value_out
);
static int write_wav(AVS_Clip* clip, const AVS_VideoInfo* vi) { static int write_wav(AVS_Clip* clip, const AVS_VideoInfo* vi) {
if (!avs_has_audio(vi)) { if (!avs_has_audio(vi)) {
fprintf(stderr, "script has no audio\n"); fprintf(stderr, "script has no audio\n");
@@ -463,6 +472,29 @@ static int read_int_frame_prop(
return 1; return 1;
} }
static int read_float_frame_prop(
AVS_ScriptEnvironment* env,
const AVS_VideoFrame* frame,
const char* name,
double* value_out
) {
const AVS_Map* props = avs_get_frame_props_ro(env, frame);
if (!props) {
return 0;
}
int error = AVS_GETPROPERROR_SUCCESS;
double value = avs_prop_get_float(env, props, name, 0, &error);
if (error == AVS_GETPROPERROR_UNSET || error == AVS_GETPROPERROR_INDEX) {
return 0;
}
if (error != AVS_GETPROPERROR_SUCCESS) {
return -1;
}
*value_out = value;
return 1;
}
static int write_validation_row( static int write_validation_row(
FILE* csv, FILE* csv,
AVS_ScriptEnvironment* env, AVS_ScriptEnvironment* env,
@@ -473,11 +505,21 @@ static int write_validation_row(
int64_t source_frame = 0; int64_t source_frame = 0;
int64_t drop_frame = 0; int64_t drop_frame = 0;
int64_t matrix = 0; int64_t matrix = 0;
int64_t primaries = 0;
int64_t transfer = 0;
int64_t color_range = 0;
double absolute_timestretch = 0.0;
int source_id_status = read_int_frame_prop(env, frame, "mbt_source_id", &source_id); int source_id_status = read_int_frame_prop(env, frame, "mbt_source_id", &source_id);
int source_frame_status = read_int_frame_prop(env, frame, "mbt_source_frame", &source_frame); int source_frame_status = read_int_frame_prop(env, frame, "mbt_source_frame", &source_frame);
int drop_frame_status = read_int_frame_prop(env, frame, "mbt_drop_frame", &drop_frame); int drop_frame_status = read_int_frame_prop(env, frame, "mbt_drop_frame", &drop_frame);
int matrix_status = read_int_frame_prop(env, frame, "_Matrix", &matrix); int matrix_status = read_int_frame_prop(env, frame, "_Matrix", &matrix);
if (source_id_status < 0 || source_frame_status < 0 || drop_frame_status < 0 || matrix_status < 0) { int primaries_status = read_int_frame_prop(env, frame, "_Primaries", &primaries);
int transfer_status = read_int_frame_prop(env, frame, "_Transfer", &transfer);
int color_range_status = read_int_frame_prop(env, frame, "_ColorRange", &color_range);
int absolute_timestretch_status = read_float_frame_prop(
env, frame, "mbt_absolute_timestretch", &absolute_timestretch
);
if (source_id_status < 0 || source_frame_status < 0 || drop_frame_status < 0 || matrix_status < 0 || primaries_status < 0 || transfer_status < 0 || color_range_status < 0 || absolute_timestretch_status < 0) {
fprintf(stderr, "failed to read media-batch-tools frame properties at frame %d\n", output_frame); fprintf(stderr, "failed to read media-batch-tools frame properties at frame %d\n", output_frame);
return 12; return 12;
} }
@@ -494,6 +536,13 @@ static int write_validation_row(
if (matrix_status > 0) { if (matrix_status > 0) {
fprintf(csv, "%lld", (long long)matrix); fprintf(csv, "%lld", (long long)matrix);
} }
fprintf(csv, ",%lld,%lld,%lld,",
primaries_status > 0 ? (long long)primaries : 0LL,
transfer_status > 0 ? (long long)transfer : 0LL,
color_range_status > 0 ? (long long)color_range : 0LL);
if (absolute_timestretch_status > 0) {
fprintf(csv, "%.17g", absolute_timestretch);
}
fputc('\n', csv); fputc('\n', csv);
return 0; return 0;
} }
@@ -546,7 +595,7 @@ static int render_clip(
fprintf(stderr, "failed to open validation CSV for writing: %s\n", validation_csv_path); fprintf(stderr, "failed to open validation CSV for writing: %s\n", validation_csv_path);
return 11; return 11;
} }
fputs("output_frame,source_id,source_frame,drop_frame,matrix\n", validation_csv); fputs("output_frame,source_id,source_frame,drop_frame,matrix,primaries,transfer,color_range,absolute_timestretch\n", validation_csv);
fflush(validation_csv); fflush(validation_csv);
} }
} }
Binary file not shown.
Binary file not shown.
+23 -7
View File
@@ -15,6 +15,10 @@ class FrameIdentity:
source_frame: int | None source_frame: int | None
drop_frame: bool = False drop_frame: bool = False
matrix: int | None = None matrix: int | None = None
primaries: int | None = None
transfer: int | None = None
color_range: int | None = None
absolute_timestretch: float | None = None
@dataclass(frozen=True) @dataclass(frozen=True)
@@ -112,23 +116,26 @@ def read_frame_identity_csv(path: Path) -> list[FrameIdentity]:
with path.open(newline="", encoding="utf-8") as handle: with path.open(newline="", encoding="utf-8") as handle:
reader = csv.reader(handle) reader = csv.reader(handle)
for line_number, row in enumerate(reader, start=1): for line_number, row in enumerate(reader, start=1):
if line_number == 1 and row in ( if line_number == 1 and row[0:4] == [
["output_frame", "source_id", "source_frame", "drop_frame"], "output_frame", "source_id", "source_frame", "drop_frame"
["output_frame", "source_id", "source_frame", "drop_frame", "matrix"], ]:
):
continue continue
if not row or all(not value.strip() for value in row): if not row or all(not value.strip() for value in row):
continue continue
if len(row) not in {4, 5}: if len(row) not in {4, 5, 6, 9}:
raise RuntimeError( raise RuntimeError(
f"{path}:{line_number}: expected 4 or 5 CSV columns, got {len(row)}" f"{path}:{line_number}: expected 4, 5, 6, or 9 CSV columns, got {len(row)}"
) )
try: try:
output_frame = int(row[0]) output_frame = int(row[0])
source_id = _optional_int(row[1]) source_id = _optional_int(row[1])
source_frame = _optional_int(row[2]) source_frame = _optional_int(row[2])
drop_frame = bool(int(row[3] or "0")) drop_frame = bool(int(row[3] or "0"))
matrix = _optional_int(row[4]) if len(row) == 5 else None matrix = _optional_int(row[4]) if len(row) >= 5 else None
primaries = _optional_int(row[5]) if len(row) == 9 else None
transfer = _optional_int(row[6]) if len(row) == 9 else None
color_range = _optional_int(row[7]) if len(row) == 9 else None
absolute_timestretch = _optional_float(row[-1]) if len(row) >= 6 else None
except ValueError as exc: except ValueError as exc:
raise RuntimeError( raise RuntimeError(
f"{path}:{line_number}: invalid frame identity row: {row}" f"{path}:{line_number}: invalid frame identity row: {row}"
@@ -140,6 +147,10 @@ def read_frame_identity_csv(path: Path) -> list[FrameIdentity]:
source_frame=source_frame, source_frame=source_frame,
drop_frame=drop_frame, drop_frame=drop_frame,
matrix=matrix, matrix=matrix,
primaries=primaries,
transfer=transfer,
color_range=color_range,
absolute_timestretch=absolute_timestretch,
) )
) )
return frames return frames
@@ -150,6 +161,11 @@ def _optional_int(value: str) -> int | None:
return int(value) if value else None return int(value) if value else None
def _optional_float(value: str) -> float | None:
value = value.strip()
return float(value) if value else None
def _format_int_list(values: list[int]) -> str: def _format_int_list(values: list[int]) -> str:
if len(values) <= 10: if len(values) <= 10:
return ", ".join(str(value) for value in values) return ", ".join(str(value) for value in values)
+80 -26
View File
@@ -8,6 +8,7 @@ from datetime import timedelta, timezone
from pathlib import Path from pathlib import Path
from statistics import median from statistics import median
from tools.avisynth_paths import avs_path
from tools.video_formatting import ( from tools.video_formatting import (
format_frame_count, format_frame_count,
format_framerate, format_framerate,
@@ -16,7 +17,13 @@ from tools.video_formatting import (
video_parts, video_parts,
) )
from tools.video_inputs import VideoInput from tools.video_inputs import VideoInput
from tools.video_probe import VideoProbe from tools.video_probe import (
VideoProbe,
avisynth_primaries_from_color_primaries,
avisynth_range_from_color_range,
avisynth_transfer_from_color_transfer,
hdr_kind,
)
WORKSPACE_DIR = Path("video_workspace") WORKSPACE_DIR = Path("video_workspace")
@@ -197,7 +204,7 @@ def _visible_script(
item: VideoInput, item: VideoInput,
probe: VideoProbe, probe: VideoProbe,
) -> str: ) -> str:
import_path = _avs_path(os.path.relpath(source_script, visible_script.parent)) import_path = avs_path(os.path.relpath(source_script, visible_script.parent))
audio_script_name = f"{visible_script.stem}_audio.avs" audio_script_name = f"{visible_script.stem}_audio.avs"
timing_kind = probe.timing.timing_kind.upper() timing_kind = probe.timing.timing_kind.upper()
final_clip = _default_final_clip(probe) final_clip = _default_final_clip(probe)
@@ -220,24 +227,37 @@ def _visible_script(
"# Frame properties must be preserved.\n" "# Frame properties must be preserved.\n"
"\n" "\n"
"# Bundled helper functions:\n" "# Bundled helper functions:\n"
"# MBT_Drop(last, int start, int \"end\"), MBT_Undrop(...)\n" "# MBT_Drop(clip c, int start, int \"end\"), MBT_Undrop(...)\n"
"# MBT_DropEvery(last, int cycle, int offset0, ...), MBT_UndropEvery(...)\n" "# MBT_DropEvery(clip c, int cycle, int offset0, ...), MBT_UndropEvery(...)\n"
"# Similar syntax as Trim/SelectEvery. end defaults to -1, meaning one frame.\n" "# Similar syntax as Trim/SelectEvery. end defaults to -1, meaning one frame.\n"
"# Examples: MBT_Drop(last, 100), MBT_Drop(last, 100, -20),\n" "# Examples: MBT_Drop(last, 100), MBT_Drop(last, 100, -20),\n"
"# MBT_DropEvery(last, 4, 1, 3)\n" "# MBT_DropEvery(last, 4, 1, 3)\n"
"# MBT_Info(last, int \"size\", int \"align\")\n" "# MBT_Info(clip c, int \"size\", int \"align\")\n"
"# Overlay source frame, video time, absolute time, duration, and drop marker.\n" "# Overlay source frame, video time, absolute time, duration, and drop marker.\n"
"# MBT_AbsoluteTimeStretch(clip c, float factor)\n"
"# Map source timestamps to real time, anchored to the metadata end timestamp.\n"
"# Example: MBT_AbsoluteTimeStretch(last, 1.0/8.0) for 8x slow motion.\n"
"# Resize(clip v, int \"w\", int \"h\", float \"dar\", float \"xcrop\", float \"ycrop\", int \"pos\", bool \"linear\", string \"input_matrix\", string \"output_matrix\")\n" "# Resize(clip v, int \"w\", int \"h\", float \"dar\", float \"xcrop\", float \"ycrop\", int \"pos\", bool \"linear\", string \"input_matrix\", string \"output_matrix\")\n"
"# Resize/crop. linear auto-enables for large downscales unless set explicitly.\n" "# Resize/crop. HDR linear resizing requires HDRTools; linear=false preserves HDR unchanged.\n"
"# MBT_CorrectMatrix(last)\n" "# MBT_CorrectMatrix(clip c, string \"input_matrix\", string \"output_matrix\")\n"
"# Convert to the resolution-expected matrix using _Matrix as input metadata.\n" "# Correct SDR matrix metadata. HDR color systems are preserved.\n"
"# MBT_ToSDR(clip c, float \"hdr_peak\", float \"sdr_peak\", string \"method\", float \"exposure\", bool \"chroma_correction\", string \"output_matrix\")\n"
"# HDR to SDR. MBT_ToSDR(10000, 125, \"MPC\") uses a 10,000 cd/m² PQ source\n"
"# peak and a 125 cd/m² SDR display target. Methods: MPC (default), Hable, Mobius,\n"
"# Reinhard, ACES, BT2446C. exposure overrides the method's default curve scale; chroma_correction\n"
"# applies only to BT2446C.\n"
"# MPC applies the Hable curve to XYZ luminance, then preserves chromaticity and compresses\n"
"# target-gamut chroma. Defaults: PQ hdr_peak=10000, HLG hdr_peak=1000, sdr_peak=125,\n"
"# chroma_correction=false, output_matrix=Rec709/Rec601 by output size.\n"
"# HDR10+ metadata is dynamic and is not used by HDRTools; this is a static tone map.\n"
"# Add LoadPlugin(\"path/to/HDRTools.dll\") below when using HDRTools.\n"
"# DeShake(clip c, int \"w\", int \"h\", float \"dar\", float \"crop\", float \"xcrop\", float \"ycrop\", int \"pos\", bool \"zoom\", bool \"rot\", float \"freq\")\n" "# DeShake(clip c, int \"w\", int \"h\", float \"dar\", float \"crop\", float \"xcrop\", float \"ycrop\", int \"pos\", bool \"zoom\", bool \"rot\", float \"freq\")\n"
"# Stabilize through MVTools + DePan; those plugins must be loaded separately.\n" "# Stabilize through MVTools + DePan; those plugins must be loaded separately.\n"
"# Cropf(...)\n" "# Cropf(clip c, float l, float t, float w, float h)\n"
"# Crop using fractional dimensions.\n" "# Crop using fractional dimensions.\n"
"# RotateCrop(clip c, float angle, float \"dar\")\n" "# RotateCrop(clip c, float angle, float \"dar\")\n"
"# Rotate with manyPlus Turn, then crop the largest background-free rectangle.\n" "# Rotate with manyPlus Turn, then crop the largest background-free rectangle.\n"
"# FixContrast(...)\n" "# FixContrast(clip c, int \"low\", int \"high\")\n"
"# Remap luma levels to limited-range video luma.\n" "# Remap luma levels to limited-range video luma.\n"
"\n" "\n"
"# Optional audio edits:\n" "# Optional audio edits:\n"
@@ -265,23 +285,23 @@ def _hidden_source_script(
duration_values: Path, duration_values: Path,
duration_label_values: Path, duration_label_values: Path,
) -> str: ) -> str:
source_path = _avs_path(str(item.path.resolve())) source_path = avs_path(str(item.path.resolve()))
import_path = _avs_path(os.path.relpath(helper_script, source_script.parent)) import_path = avs_path(os.path.relpath(helper_script, source_script.parent))
cache_path = _avs_path( cache_path = avs_path(
str(source_script.with_name(f"{item.collapsed_relative.name}.ffindex").resolve()) str(source_script.with_name(f"{item.collapsed_relative.name}.ffindex").resolve())
) )
audio_cache_path = _avs_path( audio_cache_path = avs_path(
str(source_script.with_name(f"{item.collapsed_relative.name}.audio.ffindex").resolve()) str(source_script.with_name(f"{item.collapsed_relative.name}.audio.ffindex").resolve())
) )
timestamp_values_path = _avs_path(str(timestamp_values.resolve())) timestamp_values_path = avs_path(str(timestamp_values.resolve()))
frame_time_values_path = _avs_path(str(frame_time_values.resolve())) frame_time_values_path = avs_path(str(frame_time_values.resolve()))
frame_time_label_values_path = _avs_path(str(frame_time_label_values.resolve())) frame_time_label_values_path = avs_path(str(frame_time_label_values.resolve()))
duration_values_path = _avs_path(str(duration_values.resolve())) duration_values_path = avs_path(str(duration_values.resolve()))
duration_label_values_path = _avs_path(str(duration_label_values.resolve())) duration_label_values_path = avs_path(str(duration_label_values.resolve()))
if ffms2_plugin_path is None: if ffms2_plugin_path is None:
plugin_setup = "# FFMS2 is expected from Avisynth autoload or a previously loaded plugin.\n" plugin_setup = "# FFMS2 is expected from Avisynth autoload or a previously loaded plugin.\n"
else: else:
plugin_setup = f'LoadPlugin("{_avs_path(str(ffms2_plugin_path.resolve()))}")\n' plugin_setup = f'LoadPlugin("{avs_path(str(ffms2_plugin_path.resolve()))}")\n'
audio_setup = ( audio_setup = (
"try {\n" "try {\n"
" mbt_video_only = mbt_video_only\n" " mbt_video_only = mbt_video_only\n"
@@ -310,16 +330,22 @@ def _hidden_source_script(
"\n" "\n"
f"{plugin_setup}" f"{plugin_setup}"
f'Import("{import_path}")\n' f'Import("{import_path}")\n'
f'global mbt_color_matrix = "{probe.color_matrix or probe.expected_color_matrix or "Rec709"}"\n'
f"global mbt_color_primaries = {_primaries_code_for_source(probe)}\n"
f"global mbt_color_transfer = {_transfer_code_for_source(probe)}\n"
f"global mbt_color_range = {_range_code_for_source(probe)}\n"
f"global mbt_hdr_peak = {_hdr_peak_for_source(probe):.6f}\n"
"\n" "\n"
f'video = FFVideoSource("{source_path}", cachefile="{cache_path}")\n' f'video = FFVideoSource("{source_path}", cachefile="{cache_path}")\n'
f"{validation_blank_setup}" f"{validation_blank_setup}"
f"{audio_setup}" f"{audio_setup}"
f"last = MBT_MarkSource(source, {source_id}, {_matrix_code_for_source(probe)})\n" f"last = MBT_MarkSource(source, {source_id}, {_matrix_code_for_source(probe)}, {_primaries_code_for_source(probe)}, {_transfer_code_for_source(probe)}, {_range_code_for_source(probe)}, {_hdr_peak_for_source(probe):.6f}, {_source_end_epoch_seconds(probe):.9f}, {(probe.duration_seconds or 0.0):.9f})\n"
f'last = ConditionalReader(last, "{timestamp_values_path}", "mbt_absolute_timestamp", false)\n' f'last = ConditionalReader(last, "{timestamp_values_path}", "mbt_absolute_timestamp", false)\n'
f'last = ConditionalReader(last, "{frame_time_values_path}", "mbt_frame_time_seconds", false)\n' f'last = ConditionalReader(last, "{frame_time_values_path}", "mbt_frame_time_seconds", false)\n'
f'last = ConditionalReader(last, "{frame_time_label_values_path}", "mbt_relative_timestamp", false)\n' f'last = ConditionalReader(last, "{frame_time_label_values_path}", "mbt_relative_timestamp", false)\n'
f'last = ConditionalReader(last, "{duration_values_path}", "mbt_frame_duration_seconds", false)\n' f'last = ConditionalReader(last, "{duration_values_path}", "mbt_frame_duration_seconds", false)\n'
f'last = ConditionalReader(last, "{duration_label_values_path}", "mbt_frame_duration", false)\n' f'last = ConditionalReader(last, "{duration_label_values_path}", "mbt_frame_duration", false)\n'
"last = MBT_MarkTiming(last)\n"
"last\n" "last\n"
) )
@@ -328,7 +354,7 @@ def _validation_script(
visible_script: Path, visible_script: Path,
validation_script: Path, validation_script: Path,
) -> str: ) -> str:
import_path = _avs_path(os.path.relpath(visible_script, validation_script.parent)) import_path = avs_path(os.path.relpath(visible_script, validation_script.parent))
return ( return (
"# Generated by media-batch-tools. This file is regenerated by video_encode.py.\n" "# Generated by media-batch-tools. This file is regenerated by video_encode.py.\n"
"# The media-batch-tools runner reads frame identity properties from this clip.\n" "# The media-batch-tools runner reads frame identity properties from this clip.\n"
@@ -356,6 +382,15 @@ def _timestamp_values_file(probe: VideoProbe) -> str:
return "\n".join(lines) + "\n" return "\n".join(lines) + "\n"
def _source_end_epoch_seconds(probe: VideoProbe) -> float:
if probe.metadata_end is None:
return 0.0
source_end = probe.metadata_end
if source_end.tzinfo is None:
source_end = source_end.replace(tzinfo=timezone.utc)
return source_end.timestamp()
def _frame_time_values_file(probe: VideoProbe) -> str: def _frame_time_values_file(probe: VideoProbe) -> str:
lines = [ lines = [
"# Generated by media-batch-tools for AviSynth ConditionalReader.", "# Generated by media-batch-tools for AviSynth ConditionalReader.",
@@ -453,6 +488,9 @@ def _format_video_summary(probe: VideoProbe) -> str:
bit_depth=probe.bit_depth, bit_depth=probe.bit_depth,
) )
parts.append(_format_matrix(probe)) parts.append(_format_matrix(probe))
detected_hdr = hdr_kind(probe.color_transfer, dynamic=probe.hdr10plus)
if detected_hdr is not None:
parts.append(detected_hdr)
return join_parts(parts) return join_parts(parts)
@@ -471,6 +509,24 @@ def _matrix_code_for_source(probe: VideoProbe) -> int:
return _matrix_code(probe.color_matrix or probe.expected_color_matrix) return _matrix_code(probe.color_matrix or probe.expected_color_matrix)
def _primaries_code_for_source(probe: VideoProbe) -> int:
return avisynth_primaries_from_color_primaries(probe.color_primaries)
def _transfer_code_for_source(probe: VideoProbe) -> int:
return avisynth_transfer_from_color_transfer(probe.color_transfer)
def _range_code_for_source(probe: VideoProbe) -> int:
return avisynth_range_from_color_range(probe.color_range)
def _hdr_peak_for_source(probe: VideoProbe) -> float:
# Zero means unknown. MBT_ToSDR then uses HDRTools' own HDR-mode default
# instead of pretending that an absent mastering display is 1000 cd/m².
return probe.hdr_peak_luminance or 0.0
def _matrix_code(matrix: str | None) -> int: def _matrix_code(matrix: str | None) -> int:
if matrix == "Rec709": if matrix == "Rec709":
return 1 return 1
@@ -482,6 +538,8 @@ def _matrix_code(matrix: str | None) -> int:
def _default_final_clip(probe: VideoProbe) -> str: def _default_final_clip(probe: VideoProbe) -> str:
if hdr_kind(probe.color_transfer) is not None:
return "last"
source_matrix = probe.color_matrix source_matrix = probe.color_matrix
expected_matrix = probe.expected_color_matrix expected_matrix = probe.expected_color_matrix
if source_matrix is None or expected_matrix is None or source_matrix == expected_matrix: if source_matrix is None or expected_matrix is None or source_matrix == expected_matrix:
@@ -493,9 +551,5 @@ def _format_frame_count(probe: VideoProbe) -> str:
return format_frame_count(probe.stream_frame_count, len(probe.frame_timestamps)) return format_frame_count(probe.stream_frame_count, len(probe.frame_timestamps))
def _avs_path(path: str) -> str:
return path.replace("\\", "/").replace('"', '\\"')
def helper_script_path() -> Path: def helper_script_path() -> Path:
return Path(__file__).resolve().with_name(HELPER_SCRIPT_NAME) return Path(__file__).resolve().with_name(HELPER_SCRIPT_NAME)
+114
View File
@@ -0,0 +1,114 @@
from __future__ import annotations
import json
import subprocess
from pathlib import Path
from typing import Any
def extract_hdr10plus_metadata(
*,
ffmpeg: Path,
hdr10plus_tool: Path,
source: Path,
output: Path,
) -> None:
"""Extract HDR10+ JSON without creating a full intermediate video file."""
output.parent.mkdir(parents=True, exist_ok=True)
ffmpeg_process = subprocess.Popen(
[
str(ffmpeg),
"-v", "error", "-i", str(source), "-map", "0:v:0",
"-c", "copy", "-bsf:v", "hevc_mp4toannexb", "-f", "hevc", "-",
],
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
assert ffmpeg_process.stdout is not None
try:
extracted = subprocess.run(
[str(hdr10plus_tool), "extract", "-o", str(output), "-"],
stdin=ffmpeg_process.stdout,
capture_output=True,
text=True,
)
finally:
ffmpeg_process.stdout.close()
ffmpeg_stderr = ffmpeg_process.stderr.read().decode(errors="replace")
ffmpeg_process.stderr.close()
ffmpeg_returncode = ffmpeg_process.wait()
if ffmpeg_returncode != 0:
raise RuntimeError(
f"FFmpeg could not extract the HEVC stream from {source}:\n{ffmpeg_stderr.strip()}"
)
if extracted.returncode != 0:
raise RuntimeError(
f"hdr10plus_tool could not extract metadata from {source}:\n"
f"{extracted.stderr.strip() or extracted.stdout.strip()}"
)
if not output.is_file():
raise RuntimeError(f"hdr10plus_tool did not create metadata JSON for {source}")
def select_hdr10plus_frames(
*,
source_json: Path,
source_frames: list[int],
output_json: Path,
) -> None:
"""Keep metadata for the encoded source frames and rebuild scene indices."""
payload = json.loads(source_json.read_text(encoding="utf-8"))
entries = payload.get("SceneInfo")
if not isinstance(entries, list) or not entries:
raise RuntimeError(f"HDR10+ JSON has no SceneInfo entries: {source_json}")
if not source_frames:
raise RuntimeError("Cannot create HDR10+ metadata for an empty output")
if min(source_frames) < 0 or max(source_frames) >= len(entries):
raise RuntimeError(
"HDR10+ metadata frame count does not cover the validated source frames "
f"({len(entries)} metadata entries, highest source frame {max(source_frames)})"
)
selected = [dict(entries[index]) for index in source_frames]
_reindex_scenes(selected)
payload["SceneInfo"] = selected
payload["SceneInfoSummary"] = _scene_summary(selected)
output_json.write_text(json.dumps(payload, indent=2) + "\n", encoding="utf-8")
def _reindex_scenes(entries: list[dict[str, Any]]) -> None:
scene_id = 0
scene_frame_index = 0
previous: dict[str, Any] | None = None
for sequence_frame_index, entry in enumerate(entries):
if previous is not None and _scene_payload(entry) != _scene_payload(previous):
scene_id += 1
scene_frame_index = 0
entry["SceneFrameIndex"] = scene_frame_index
entry["SceneId"] = scene_id
entry["SequenceFrameIndex"] = sequence_frame_index
scene_frame_index += 1
previous = entry
def _scene_payload(entry: dict[str, Any]) -> dict[str, Any]:
return {
key: value
for key, value in entry.items()
if key not in {"SceneFrameIndex", "SceneId", "SequenceFrameIndex"}
}
def _scene_summary(entries: list[dict[str, Any]]) -> dict[str, list[int]]:
starts = [
index
for index, entry in enumerate(entries)
if entry.get("SceneFrameIndex") == 0
]
return {
"SceneFirstFrameIndex": starts,
"SceneFrameNumbers": [
next_start - start
for start, next_start in zip(starts, [*starts[1:], len(entries)])
],
}
+13 -12
View File
@@ -5,6 +5,7 @@ import subprocess
from pathlib import Path from pathlib import Path
from tools.exiftool import run_exiftool_command from tools.exiftool import run_exiftool_command
from tools.filesystem import unique_path
EXCLUDED_COPY_TAGS = [ EXCLUDED_COPY_TAGS = [
@@ -66,9 +67,16 @@ def copy_meaningful_metadata(
exiftool: str, exiftool: str,
source: Path, source: Path,
destination: Path, destination: Path,
*,
extra_excluded_tags: list[str] | None = None,
) -> None: ) -> None:
if destination.suffix.lower() in {".mp4", ".mov"}: if destination.suffix.lower() in {".mp4", ".mov"}:
copy_meaningful_metadata_with_exiftool(exiftool, source, destination) copy_meaningful_metadata_with_exiftool(
exiftool,
source,
destination,
extra_excluded_tags=extra_excluded_tags,
)
return return
copy_container_metadata_with_ffmpeg(source, destination) copy_container_metadata_with_ffmpeg(source, destination)
@@ -77,6 +85,8 @@ def copy_meaningful_metadata_with_exiftool(
exiftool: str, exiftool: str,
source: Path, source: Path,
destination: Path, destination: Path,
*,
extra_excluded_tags: list[str] | None = None,
) -> None: ) -> None:
args = [ args = [
"-overwrite_original", "-overwrite_original",
@@ -84,6 +94,7 @@ def copy_meaningful_metadata_with_exiftool(
str(source), str(source),
"-all:all", "-all:all",
*EXCLUDED_COPY_TAGS, *EXCLUDED_COPY_TAGS,
*(extra_excluded_tags or []),
str(destination), str(destination),
] ]
run_exiftool_command( run_exiftool_command(
@@ -102,7 +113,7 @@ def copy_container_metadata_with_ffmpeg(source: Path, destination: Path) -> None
return return
temp_output = destination.with_name(f"{destination.stem}.metadata-copy{destination.suffix}") temp_output = destination.with_name(f"{destination.stem}.metadata-copy{destination.suffix}")
temp_output = _unique_path(temp_output) temp_output = unique_path(temp_output)
command = [ command = [
ffmpeg, ffmpeg,
"-y", "-y",
@@ -127,13 +138,3 @@ def copy_container_metadata_with_ffmpeg(source: Path, destination: Path) -> None
finally: finally:
if temp_output.exists(): if temp_output.exists():
temp_output.unlink() temp_output.unlink()
def _unique_path(path: Path) -> Path:
if not path.exists():
return path
for index in range(1, 10000):
candidate = path.with_name(f"{path.stem}-{index}{path.suffix}")
if not candidate.exists():
return candidate
raise RuntimeError(f"Could not find a unique temporary path for {path}")
+1 -1
View File
@@ -4,7 +4,7 @@ import re
from datetime import datetime, timedelta, timezone from datetime import datetime, timedelta, timezone
TZ_RE = re.compile(r"^(?P<sign>[+-])(?P<h>\d{2})(?::?(?P<m>\d{2}))?$") TZ_RE = re.compile(r"^(?P<sign>[+-])(?P<h>\d{1,2})(?::?(?P<m>\d{2}))?$")
def local_timezone() -> timezone: def local_timezone() -> timezone:
+503 -271
View File
@@ -2,7 +2,8 @@ from __future__ import annotations
import os import os
import sys import sys
from dataclasses import dataclass, replace import tempfile
from dataclasses import dataclass, field, replace
from datetime import datetime, timezone from datetime import datetime, timezone
from pathlib import Path from pathlib import Path
@@ -14,17 +15,20 @@ from tools.avisynth_workspace import (
visible_script_path, visible_script_path,
) )
from tools.console import clear_screen, light_red, prompt_input, prompt_yes_no, red_strikethrough from tools.console import clear_screen, light_red, prompt_input, prompt_yes_no, red_strikethrough
from tools.video_color import ColorMetadata
from tools.filenames import format_rounded_filename_stem from tools.filenames import format_rounded_filename_stem
from tools.hdr10plus import extract_hdr10plus_metadata, select_hdr10plus_frames
from tools.timezones import local_timezone, parse_timezone_offset from tools.timezones import local_timezone, parse_timezone_offset
from tools.timezones import timezone_to_string from tools.timezones import timezone_to_string
from tools.video_encode_output import ( from tools.video_encode_output import (
ENCODER_PRESETS, ENCODER_PRESETS,
AudioEncodeOptions, AudioEncodeOptions,
VideoEncodeResult,
VideoEncodeRequest,
VideoCodecOptions, VideoCodecOptions,
choose_video_pixel_format, choose_video_pixel_format,
default_audio_bitrate, default_audio_bitrate,
encode_video_only_y4m, encode_video_only_y4m,
ffmpeg_colorspace_from_matrix,
pixel_format_bit_depth, pixel_format_bit_depth,
print_encoder_statistics, print_encoder_statistics,
remux_video, remux_video,
@@ -55,22 +59,18 @@ from tools.video_codec_constraints import (
) )
from tools.video_inputs import VideoInput from tools.video_inputs import VideoInput
from tools.video_formatting import chroma_family, matrix_name from tools.video_formatting import chroma_family, matrix_name
from tools.video_options import (
OutputNamingOptions,
ask_audio_options,
ask_output_naming_options,
ask_video_codec_options,
)
from tools.video_outputs import ( from tools.video_outputs import (
EncodedItem, EncodedItem,
OutputNamingOptions,
copy_video_metadata, copy_video_metadata,
finalize_encoded_outputs, finalize_encoded_outputs,
output_begin_timestamp, output_begin_timestamp,
planned_output_path, planned_output_path,
validate_encoded_video, validate_encoded_video,
validate_output_color_metadata,
write_video_end_timestamp, write_video_end_timestamp,
) )
from tools.video_probe import VideoProbe, require_tool from tools.video_probe import VideoProbe, is_hdr_transfer, probe_video, require_tool
from tools.video_reporting import format_validation_summary_lines from tools.video_reporting import format_validation_summary_lines
from tools.video_timeline import OutputTimeline from tools.video_timeline import OutputTimeline
@@ -96,6 +96,54 @@ class EncodingSettings:
audio_options: AudioEncodeOptions audio_options: AudioEncodeOptions
@dataclass(frozen=True)
class EncodingTools:
root: Path
ffmpeg: Path
ffprobe: str
exiftool: str
avs_runners: AvisynthRunnerSet
mkvmerge: Path | None
x264: Path | None
@dataclass
class EncodingAnswers:
timezone_default: timezone
video_copy_available: bool
values: dict[str, object] = field(default_factory=dict)
def get(self, key: str, default: object = None) -> object:
return self.values.get(key, default)
def set(self, key: str, value: object) -> None:
old_value = self.values.get(key)
self.values[key] = value
if old_value is None or old_value == value:
return
resets = {
"codec": {"crf", "preset", "profile", "level", "threads"},
"container": {"audio", "audio_bitrate"},
"timestamp_names": {"timezone"} if value is False else set(),
"audio": {"audio_bitrate", "audio_pitch"}
if value not in {"aac", "opus"}
else set(),
}
for dependent in resets.get(key, set()):
self.values.pop(dependent, None)
_ENV_CONTAINERS = {
"mp4": ".mp4", ".mp4": ".mp4",
"m": ".mkv", "mkv": ".mkv", ".mkv": ".mkv", "matroska": ".mkv",
}
_ENV_CODECS = {
"x264": "libx264", "h264": "libx264", "libx264": "libx264",
"x265": "libx265", "h265": "libx265", "hevc": "libx265", "libx265": "libx265",
"c": "copy", "copy": "copy", "streamcopy": "copy", "stream-copy": "copy",
}
def _plan_batch_outputs( def _plan_batch_outputs(
*, *,
inputs: list[VideoInput], inputs: list[VideoInput],
@@ -151,9 +199,6 @@ def encode_validated_video_only(
print("\nNo validated outputs to encode.") print("\nNo validated outputs to encode.")
return return
ffmpeg = Path(require_tool("ffmpeg")).resolve() ffmpeg = Path(require_tool("ffmpeg")).resolve()
default_timezone = _default_output_timezone(
[probes_by_path[item.path.resolve()] for item in inputs]
)
if sys.stdin.isatty() and not _encoding_env_is_set(): if sys.stdin.isatty() and not _encoding_env_is_set():
settings = ask_interactive_encoding_settings( settings = ask_interactive_encoding_settings(
inputs=inputs, inputs=inputs,
@@ -161,20 +206,22 @@ def encode_validated_video_only(
clip_infos=clip_infos, clip_infos=clip_infos,
probes_by_path=probes_by_path, probes_by_path=probes_by_path,
) )
if settings is None:
print("\nEncoding skipped.")
return
naming = settings.naming
codec_options = settings.codec_options
audio_options = settings.audio_options
else: else:
naming = ask_output_naming_options(default_timezone=default_timezone) settings = _environment_encoding_settings(
codec_options = ask_video_codec_options() _default_output_timezone([probes_by_path[item.path.resolve()] for item in inputs])
audio_options = ask_audio_options(
has_speed_changes=has_avisynth_speed_changes(timelines, clip_infos),
container_extension=codec_options.extension,
) )
if settings is None:
print("\nEncoding skipped.")
return
naming, codec_options, audio_options = (
settings.naming,
settings.codec_options,
settings.audio_options,
)
specs = _encoding_specs(inputs, timelines, clip_infos, probes_by_path) specs = _encoding_specs(inputs, timelines, clip_infos, probes_by_path)
specs_by_path = {
item.path.resolve(): spec for item, spec in zip(inputs, specs)
}
copy_available = _video_copy_available(inputs, timelines, clip_infos, probes_by_path) copy_available = _video_copy_available(inputs, timelines, clip_infos, probes_by_path)
if codec_options.codec == "copy" and not copy_available: if codec_options.codec == "copy" and not copy_available:
raise RuntimeError( raise RuntimeError(
@@ -237,6 +284,7 @@ def encode_validated_video_only(
naming=naming, naming=naming,
extension=codec_options.extension, extension=codec_options.extension,
) )
tools = EncodingTools(root, ffmpeg, ffprobe, exiftool, avs_runners, mkvmerge, x264)
encoded_items: list[EncodedItem] = [] encoded_items: list[EncodedItem] = []
for item_index, item in enumerate(inputs, start=1): for item_index, item in enumerate(inputs, start=1):
timeline = timelines[item.path.resolve()] timeline = timelines[item.path.resolve()]
@@ -248,165 +296,34 @@ def encode_validated_video_only(
clip_info = clip_infos.get(item.path.resolve()) clip_info = clip_infos.get(item.path.resolve())
probe = probes_by_path[item.path.resolve()] probe = probes_by_path[item.path.resolve()]
if codec_options.codec == "copy": if codec_options.codec == "copy":
output = output_paths[item.path.resolve()] encoded_items.append(
if audio_options.mode == "copy": _remux_item(
audio_source, source_has_audio = item.path, probe.audio_stream_count > 0 tools=tools,
else:
audio_source, source_has_audio = prepare_audio_source(
avs_runners=avs_runners,
root=root,
item=item, item=item,
original_has_audio=probe.audio_stream_count > 0, item_index=item_index,
item_count=len(inputs),
probe=probe,
timeline=timeline,
audio_options=audio_options, audio_options=audio_options,
output=output_paths[item.path.resolve()],
) )
result = remux_video(
ffmpeg=ffmpeg,
source=item.path,
audio_source=audio_source,
output=output,
audio_options=audio_options,
source_has_audio=source_has_audio,
frame_count=len(timeline.frames),
progress_label=f"script {item_index}/{len(inputs)}: {item.collapsed_relative}",
)
validate_encoded_video(
ffprobe,
result.output,
timeline,
expected_duration=timeline.duration_seconds,
expected_timestamps=timeline_frame_starts(timeline),
expected_audio=audio_options.mode != "none" and source_has_audio,
)
copy_video_metadata(exiftool, item.path, result.output)
write_video_end_timestamp(
exiftool, result.output, probe, timeline, timeline.duration_seconds
)
encoded_items.append(EncodedItem(input_item=item, output=result.output))
print(f" remuxed output: {result.output}")
continue
output_bit_depth = clip_info.bit_depth if clip_info is not None else 8
pixel_format = choose_video_pixel_format(
ffmpeg,
codec_options.codec,
output_bit_depth,
clip_info.chroma if clip_info is not None else None,
)
spec = _constraint_spec_for_item(item, timelines, clip_infos, probes_by_path)
encoded_spec = VideoConstraintSpec(
width=spec.width,
height=spec.height,
bit_depth=pixel_format_bit_depth(pixel_format),
chroma=spec.chroma,
fps=spec.fps,
)
effective_codec_options = _resolve_codec_options(codec_options, encoded_spec)
if effective_codec_options.profile or effective_codec_options.level:
print(
" constraints: "
f"profile {effective_codec_options.profile or 'none'}, "
f"level {effective_codec_options.level or 'none'}"
)
colorspace = ffmpeg_colorspace_from_matrix(timeline.matrix)
if output_bit_depth >= 10:
encoded_bit_depth = pixel_format_bit_depth(pixel_format)
print(
f" video: {output_bit_depth}-bit Avisynth output -> "
f"{encoded_bit_depth}-bit {pixel_format}"
)
if colorspace is not None:
print(f" color space: {matrix_name(timeline.matrix)} -> ffmpeg {colorspace}")
normal_deleted_frames = normal_deleted_frame_count(timeline)
preserve_video_timestamps = should_preserve_video_timestamps(timeline)
if not preserve_video_timestamps and timeline.timing_kind != "cfr":
print(" skipped encode: output timeline is not CFR")
continue
if not preserve_video_timestamps and timeline.dropped_frame_count and (
clip_info is not None and avisynth_duration_differs(clip_info, timeline)
):
print(
" skipped encode: drop_frame plus Avisynth speed change needs "
"timestamp-aware muxing"
) )
continue continue
output_duration = ( encoded_item = _encode_item(
timeline.duration_seconds tools=tools,
if preserve_video_timestamps
else encoded_duration_seconds(clip_info, timeline)
)
audio_tempo = audio_tempo_factor(timeline, output_duration)
output = output_paths[item.path.resolve()]
effective_audio_options = audio_options_for_probe(
audio_options,
probe,
timeline,
output_duration,
normal_deleted_frames=normal_deleted_frames,
)
audio_segments = (
timeline.audio_segments
if should_use_audio_segments(timeline, effective_audio_options)
else None
)
audio_source, source_has_audio = prepare_audio_source(
avs_runners=avs_runners,
root=root,
item=item, item=item,
original_has_audio=probe.audio_stream_count > 0, item_index=item_index,
audio_options=effective_audio_options, item_count=len(inputs),
timeline=timeline,
clip_info=clip_info,
probe=probe,
spec=_constraint_spec(specs_by_path[item.path.resolve()]),
codec_options=codec_options,
audio_options=audio_options,
output=output_paths[item.path.resolve()],
) )
encode_kwargs = dict( if encoded_item is not None:
y4m_command=y4m_command_for_timeline( encoded_items.append(encoded_item)
avs_runner,
timeline,
force_timeline_rate=preserve_video_timestamps,
),
ffmpeg=ffmpeg,
script=script,
audio_source=audio_source,
output=output,
options=effective_codec_options,
audio_options=effective_audio_options,
audio_start=timeline.frames[0].start,
audio_duration=timeline.duration_seconds,
audio_segments=audio_segments,
audio_tempo=audio_tempo,
audio_sample_rate=probe.audio_sample_rate,
source_has_audio=source_has_audio,
allow_audio_copy=effective_audio_options.mode == "copy",
pixel_format=pixel_format,
colorspace=colorspace,
frame_count=len(timeline.frames),
progress_label=(
f"script {item_index}/{len(inputs)}: {item.collapsed_relative}"
),
)
if preserve_video_timestamps:
assert mkvmerge is not None
result = encode_video_with_timestamps(
mkvmerge=mkvmerge,
x264=x264,
frame_durations=[frame.duration for frame in timeline.frames],
**encode_kwargs,
)
else:
result = encode_video_only_y4m(**encode_kwargs)
validate_encoded_video(
ffprobe,
result.output,
timeline,
expected_duration=output_duration,
expected_timestamps=(
timeline_frame_starts(timeline) if preserve_video_timestamps else None
),
expected_audio=(
effective_audio_options.mode != "none" and source_has_audio
),
)
print_encoder_statistics(result.encoder_log)
copy_video_metadata(exiftool, item.path, result.output)
write_video_end_timestamp(exiftool, result.output, probe, timeline, output_duration)
encoded_items.append(EncodedItem(input_item=item, output=result.output))
print(f" encoded output: {result.output}")
if not encoded_items: if not encoded_items:
print("\nNo outputs encoded.") print("\nNo outputs encoded.")
@@ -415,6 +332,265 @@ def encode_validated_video_only(
finalize_encoded_outputs(root, encoded_items) finalize_encoded_outputs(root, encoded_items)
def _remux_item(
*,
tools: EncodingTools,
item: VideoInput,
item_index: int,
item_count: int,
probe: VideoProbe,
timeline: OutputTimeline,
audio_options: AudioEncodeOptions,
output: Path,
) -> EncodedItem:
if audio_options.mode == "copy":
audio_source, source_has_audio = item.path, probe.audio_stream_count > 0
else:
audio_source, source_has_audio = prepare_audio_source(
avs_runners=tools.avs_runners,
root=tools.root,
item=item,
original_has_audio=probe.audio_stream_count > 0,
audio_options=audio_options,
)
result = remux_video(
ffmpeg=tools.ffmpeg,
source=item.path,
audio_source=audio_source,
output=output,
audio_options=audio_options,
source_has_audio=source_has_audio,
frame_count=len(timeline.frames),
progress_label=f"script {item_index}/{item_count}: {item.collapsed_relative}",
)
return _finalize_encoded_item(
ffprobe=tools.ffprobe,
exiftool=tools.exiftool,
item=item,
probe=probe,
timeline=timeline,
result=result,
expected_duration=timeline.duration_seconds,
expected_timestamps=timeline_frame_starts(timeline),
expected_audio=audio_options.mode != "none" and source_has_audio,
label="remuxed",
)
def _encode_item(
*,
tools: EncodingTools,
item: VideoInput,
item_index: int,
item_count: int,
timeline: OutputTimeline,
clip_info: AvisynthClipInfo | None,
probe: VideoProbe,
spec: VideoConstraintSpec,
codec_options: VideoCodecOptions,
audio_options: AudioEncodeOptions,
output: Path,
) -> EncodedItem | None:
if probe.hdr_dynamic and not probe.hdr10plus and timeline.transfer in {16, 18}:
raise RuntimeError(
f"{item.collapsed_relative} has dynamic HDR metadata that is not HDR10+. "
"Stream copy preserves it, but re-encoding it is not implemented yet."
)
if is_hdr_transfer(probe.color_transfer) and timeline.transfer is None:
raise RuntimeError(
f"{item.collapsed_relative} is HDR, but the Avisynth runner did not report "
"output color properties. Use the current bundled runner before encoding HDR."
)
output_bit_depth = clip_info.bit_depth if clip_info else 8
pixel_format = choose_video_pixel_format(
tools.ffmpeg,
codec_options.codec,
output_bit_depth,
clip_info.chroma if clip_info else None,
)
options = _resolve_codec_options(
codec_options, replace(spec, bit_depth=pixel_format_bit_depth(pixel_format))
)
if options.profile or options.level:
print(
f" constraints: profile {options.profile or 'none'}, "
f"level {options.level or 'none'}"
)
color = ColorMetadata(
timeline.matrix, timeline.primaries, timeline.transfer, timeline.color_range
)
static_hdr = (
(probe.mastering_display, probe.max_content_light, probe.max_frame_average_light, timeline.transfer == 16)
if timeline.transfer in {16, 18}
else None
)
if output_bit_depth >= 10:
print(
f" video: {output_bit_depth}-bit Avisynth output -> "
f"{pixel_format_bit_depth(pixel_format)}-bit {pixel_format}"
)
if color.colorspace:
print(f" color space: {matrix_name(timeline.matrix)} -> ffmpeg {color.colorspace}")
preserve_timestamps = should_preserve_video_timestamps(timeline)
if not preserve_timestamps and timeline.timing_kind != "cfr":
print(" skipped encode: output timeline is not CFR")
return None
if not preserve_timestamps and timeline.dropped_frame_count and (
clip_info is not None and avisynth_duration_differs(clip_info, timeline)
):
print(" skipped encode: drop_frame plus Avisynth speed change needs timestamp-aware muxing")
return None
output_duration = (
timeline.duration_seconds if preserve_timestamps else encoded_duration_seconds(clip_info, timeline)
)
effective_audio = audio_options_for_probe(
audio_options,
probe,
timeline,
output_duration,
normal_deleted_frames=normal_deleted_frame_count(timeline),
)
audio_source, source_has_audio = prepare_audio_source(
avs_runners=tools.avs_runners,
root=tools.root,
item=item,
original_has_audio=probe.audio_stream_count > 0,
audio_options=effective_audio,
)
dynamic_hdr, dynamic_hdr_temp = _dynamic_hdr_metadata(tools, item, probe, timeline, options)
script = visible_script_path(tools.root, item)
request = VideoEncodeRequest(
y4m_command=y4m_command_for_timeline(
tools.avs_runners.for_script(script), timeline, force_timeline_rate=preserve_timestamps
),
ffmpeg=tools.ffmpeg,
script=script,
audio_source=audio_source,
output=output,
options=options,
audio_options=effective_audio,
audio_start=timeline.frames[0].start,
audio_duration=timeline.duration_seconds,
audio_segments=timeline.audio_segments if should_use_audio_segments(timeline, effective_audio) else None,
audio_tempo=audio_tempo_factor(timeline, output_duration),
audio_sample_rate=probe.audio_sample_rate,
source_has_audio=source_has_audio,
allow_audio_copy=effective_audio.mode == "copy",
pixel_format=pixel_format,
colorspace=color.colorspace,
color_metadata=color,
static_hdr=static_hdr,
dynamic_hdr10plus=dynamic_hdr,
frame_count=len(timeline.frames),
progress_label=f"script {item_index}/{item_count}: {item.collapsed_relative}",
)
try:
if preserve_timestamps:
if tools.mkvmerge is None:
raise RuntimeError("mkvmerge is required for timestamp-aware encoding")
result = encode_video_with_timestamps(
request=request,
mkvmerge=tools.mkvmerge,
x264=tools.x264,
frame_durations=[frame.duration for frame in timeline.frames],
)
else:
result = encode_video_only_y4m(request)
finally:
if dynamic_hdr_temp is not None:
dynamic_hdr_temp.cleanup()
return _finalize_encoded_item(
ffprobe=tools.ffprobe,
exiftool=tools.exiftool,
item=item,
probe=probe,
timeline=timeline,
result=result,
expected_duration=output_duration,
expected_timestamps=timeline_frame_starts(timeline) if preserve_timestamps else None,
expected_audio=effective_audio.mode != "none" and source_has_audio,
label="encoded",
)
def _dynamic_hdr_metadata(
tools: EncodingTools,
item: VideoInput,
probe: VideoProbe,
timeline: OutputTimeline,
options: VideoCodecOptions,
) -> tuple[Path | None, tempfile.TemporaryDirectory[str] | None]:
if not (probe.hdr10plus and timeline.transfer in {16, 18}):
return None, None
if options.codec != "libx265":
raise RuntimeError(
f"{item.collapsed_relative} has HDR10+ metadata. HDR-preserving re-encoding "
"requires x265; use MBT_ToSDR for SDR output."
)
if probe.codec != "hevc":
raise RuntimeError(f"HDR10+ metadata extraction currently requires HEVC input: {item.path}")
temporary = tempfile.TemporaryDirectory(prefix="mbt-hdr10plus-")
root = Path(temporary.name)
source_json = root / "source.hdr10plus.json"
output_json = root / "selected.hdr10plus.json"
try:
extract_hdr10plus_metadata(
ffmpeg=tools.ffmpeg,
hdr10plus_tool=Path(require_tool("hdr10plus_tool")),
source=item.path,
output=source_json,
)
select_hdr10plus_frames(
source_json=source_json,
source_frames=[frame.source_frame for frame in timeline.frames],
output_json=output_json,
)
except Exception:
temporary.cleanup()
raise
print(f" HDR10+: preserving metadata for {len(timeline.frames)} output frame(s)")
return output_json, temporary
def _finalize_encoded_item(
*,
ffprobe: str,
exiftool: str,
item: VideoInput,
probe: VideoProbe,
timeline: OutputTimeline,
result: VideoEncodeResult,
expected_duration: float,
expected_timestamps: list[float] | None,
expected_audio: bool,
label: str,
) -> EncodedItem:
validate_encoded_video(
ffprobe,
result.output,
timeline,
expected_duration=expected_duration,
expected_timestamps=expected_timestamps,
expected_audio=expected_audio,
)
if result.encoder_log:
print_encoder_statistics(result.encoder_log)
copy_video_metadata(exiftool, item.path, result.output)
validate_output_color_metadata(
result.output,
probe_video(ffprobe, result.output, {}),
timeline,
expected_hdr10plus=probe.hdr10plus and timeline.transfer in {16, 18},
)
write_video_end_timestamp(exiftool, result.output, probe, timeline)
print(f" {label} output: {result.output}")
return EncodedItem(input_item=item, output=result.output)
def ask_interactive_encoding_settings( def ask_interactive_encoding_settings(
*, *,
inputs: list[VideoInput], inputs: list[VideoInput],
@@ -422,30 +598,19 @@ def ask_interactive_encoding_settings(
clip_infos: dict[Path, AvisynthClipInfo], clip_infos: dict[Path, AvisynthClipInfo],
probes_by_path: dict[Path, VideoProbe], probes_by_path: dict[Path, VideoProbe],
) -> EncodingSettings | None: ) -> EncodingSettings | None:
answers: dict[str, object] = { answers = EncodingAnswers(
"_timezone_default": _default_output_timezone( timezone_default=_default_output_timezone(
[probes_by_path[item.path.resolve()] for item in inputs] [probes_by_path[item.path.resolve()] for item in inputs]
) ),
} video_copy_available=_video_copy_available(
index = 0 inputs, timelines, clip_infos, probes_by_path
has_speed_changes = has_avisynth_speed_changes(timelines, clip_infos) ),
specs = _encoding_specs(inputs, timelines, clip_infos, probes_by_path)
answers["_video_copy_available"] = _video_copy_available(
inputs, timelines, clip_infos, probes_by_path
) )
index = 0
def answer(key: str, value: object) -> None: has_speed_changes = has_avisynth_speed_changes(
old_value = answers.get(key) timelines, clip_infos, probes_by_path
answers[key] = value )
if key == "codec" and old_value is not None and old_value != value: specs = _encoding_specs(inputs, timelines, clip_infos, probes_by_path)
_forget(answers, {"crf", "preset", "profile", "level", "threads"})
if key == "container" and old_value is not None and old_value != value:
_forget(answers, {"audio", "audio_bitrate"})
if key == "timestamp_names" and value is False:
_forget(answers, {"timezone"})
if key == "audio" and value not in {"aac", "opus"}:
_forget(answers, {"audio_bitrate", "audio_pitch"})
while True: while True:
steps = _encoding_steps(answers, has_speed_changes) steps = _encoding_steps(answers, has_speed_changes)
index = max(0, min(index, len(steps) - 1)) index = max(0, min(index, len(steps) - 1))
@@ -462,30 +627,28 @@ def ask_interactive_encoding_settings(
print(exc) print(exc)
prompt_input("Press Enter to retry...") prompt_input("Press Enter to retry...")
continue continue
answer(key, value) answers.set(key, value)
if key == "confirm" and value is False: if key == "confirm" and value is False:
return None return None
if index == len(steps) - 1: if index == len(steps) - 1:
break break
index += 1 index += 1
codec = answers["codec"] codec = answers.get("codec")
assert isinstance(codec, str) assert isinstance(codec, str)
audio_mode = answers.get("audio", "aac" if answers["container"] == ".mp4" else "opus") container = str(answers.get("container", ".mp4"))
audio_mode = answers.get("audio", "aac" if container == ".mp4" else "opus")
assert isinstance(audio_mode, str) assert isinstance(audio_mode, str)
return EncodingSettings( return EncodingSettings(
naming=OutputNamingOptions( naming=OutputNamingOptions(
use_timestamps=bool(answers["timestamp_names"]), use_timestamps=bool(answers.get("timestamp_names")),
timezone_value=answers.get( timezone_value=answers.get("timezone", answers.timezone_default),
"timezone",
answers["_timezone_default"],
),
), ),
codec_options=VideoCodecOptions( codec_options=VideoCodecOptions(
codec=codec, codec=codec,
crf=int(answers.get("crf", 21)), crf=int(answers.get("crf", 21)),
preset=str(answers.get("preset", "slow")), preset=str(answers.get("preset", "slow")),
extension=str(answers["container"]), extension=container,
profile=_none_if_none(str(answers.get("profile", "none"))), profile=_none_if_none(str(answers.get("profile", "none"))),
level=_none_if_none(str(answers.get("level", "none"))), level=_none_if_none(str(answers.get("level", "none"))),
threads=_threads_from_answer(answers.get("threads", "auto")), threads=_threads_from_answer(answers.get("threads", "auto")),
@@ -516,7 +679,103 @@ def _encoding_env_is_set() -> bool:
return any(os.environ.get(name) for name in names) return any(os.environ.get(name) for name in names)
def _encoding_steps(answers: dict[str, object], has_speed_changes: bool) -> list[str]: def _environment_encoding_settings(default_timezone: timezone) -> EncodingSettings:
timestamp_names = _env_is_truthy("MBT_VIDEO_TIMESTAMP_NAMES")
timezone_value = default_timezone
if timestamp_names and (raw_timezone := os.environ.get("MBT_VIDEO_TIMEZONE", "").strip()):
try:
timezone_value = parse_timezone_offset(raw_timezone)
except ValueError as exc:
raise RuntimeError(f"Invalid MBT_VIDEO_TIMEZONE: {exc}") from exc
container = _environment_choice(
"MBT_VIDEO_CONTAINER",
"mp4",
_ENV_CONTAINERS,
"mp4 or mkv",
)
codec = _environment_choice(
"MBT_VIDEO_CODEC",
"x265",
_ENV_CODECS,
"x264, x265, or copy",
)
crf = _environment_int("MBT_VIDEO_CRF", 16 if codec == "libx264" else 21)
preset = os.environ.get("MBT_VIDEO_PRESET", "slow").strip().lower() or "slow"
if preset not in ENCODER_PRESETS:
raise RuntimeError(f"MBT_VIDEO_PRESET must be one of: {'/'.join(ENCODER_PRESETS)}")
threads = _environment_threads()
audio_mode = _parse_audio(os.environ.get("MBT_VIDEO_AUDIO", "none"))
preserve_pitch = _environment_audio_pitch()
return EncodingSettings(
naming=OutputNamingOptions(timestamp_names, timezone_value),
codec_options=VideoCodecOptions(
codec=codec,
crf=crf,
preset=preset,
extension=container,
profile=_optional_environment("MBT_VIDEO_PROFILE"),
level=_optional_environment("MBT_VIDEO_LEVEL"),
threads=threads,
),
audio_options=AudioEncodeOptions(
mode=audio_mode,
bitrate=os.environ.get("MBT_VIDEO_AUDIO_BITRATE", "").strip()
or default_audio_bitrate(audio_mode),
preserve_pitch=preserve_pitch,
),
)
def _env_is_truthy(name: str) -> bool:
return os.environ.get(name, "").strip().lower() in {"1", "true", "yes", "y", "on"}
def _environment_choice(
name: str,
default: str,
values: dict[str, str],
allowed: str,
) -> str:
value = os.environ.get(name, default).strip().lower()
try:
return values[value]
except KeyError as exc:
raise RuntimeError(f"{name} must be {allowed}") from exc
def _environment_int(name: str, default: int) -> int:
value = os.environ.get(name, "").strip()
try:
return int(value) if value else default
except ValueError as exc:
raise RuntimeError(f"{name} must be an integer") from exc
def _environment_threads() -> int | None:
value = os.environ.get("MBT_VIDEO_THREADS", "").strip().lower()
if not value or value == "auto":
return None
threads = _environment_int("MBT_VIDEO_THREADS", 0)
if threads < 1:
raise RuntimeError("MBT_VIDEO_THREADS must be auto or a positive integer")
return threads
def _environment_audio_pitch() -> bool:
value = os.environ.get("MBT_VIDEO_AUDIO_PITCH", "").strip().lower()
if not value or value in {"preserve", "preserved", "keep", "same"}:
return True
if value in {"shift", "change", "changed", "speed"}:
return False
raise RuntimeError("MBT_VIDEO_AUDIO_PITCH must be preserve or shift")
def _optional_environment(name: str) -> str | None:
return os.environ.get(name, "").strip() or None
def _encoding_steps(answers: EncodingAnswers, has_speed_changes: bool) -> list[str]:
steps = ["timestamp_names"] steps = ["timestamp_names"]
if answers.get("timestamp_names", True): if answers.get("timestamp_names", True):
steps.append("timezone") steps.append("timezone")
@@ -533,7 +792,7 @@ def _encoding_steps(answers: dict[str, object], has_speed_changes: bool) -> list
def _render_encoding_screen( def _render_encoding_screen(
answers: dict[str, object], answers: EncodingAnswers,
current_key: str, current_key: str,
specs: list[EncodingOutputSpec], specs: list[EncodingOutputSpec],
has_speed_changes: bool, has_speed_changes: bool,
@@ -563,7 +822,7 @@ def _render_encoding_screen(
print(light_red("x264 is unavailable because this batch contains output above 10-bit.\n")) print(light_red("x264 is unavailable because this batch contains output above 10-bit.\n"))
def _encoding_table_keys(answers: dict[str, object], has_speed_changes: bool) -> list[str]: def _encoding_table_keys(answers: EncodingAnswers, has_speed_changes: bool) -> list[str]:
keys = [ keys = [
"timestamp_names", "timestamp_names",
"container", "container",
@@ -584,7 +843,7 @@ def _encoding_table_keys(answers: dict[str, object], has_speed_changes: bool) ->
def _question_for_key( def _question_for_key(
key: str, key: str,
answers: dict[str, object], answers: EncodingAnswers,
specs: list[EncodingOutputSpec], specs: list[EncodingOutputSpec],
) -> str: ) -> str:
if key == "timestamp_names": if key == "timestamp_names":
@@ -611,7 +870,7 @@ def _question_for_key(
return prompt + (light_red(" ".join(warnings)) if warnings else "") return prompt + (light_red(" ".join(warnings)) if warnings else "")
if key == "codec": if key == "codec":
options = ["x264", "x265"] options = ["x264", "x265"]
if answers.get("_video_copy_available"): if answers.video_copy_available:
options.append("copy") options.append("copy")
if _x264_unavailable(specs): if _x264_unavailable(specs):
options[0] = red_strikethrough(options[0]) options[0] = red_strikethrough(options[0])
@@ -624,20 +883,12 @@ def _question_for_key(
if key == "preset": if key == "preset":
default = _default_answer(key, answers, specs) default = _default_answer(key, answers, specs)
return f"Encoder preset [{default}] ({'/'.join(ENCODER_PRESETS)}): " return f"Encoder preset [{default}] ({'/'.join(ENCODER_PRESETS)}): "
if key == "profile": if key in {"profile", "level"}:
codec = str(answers["codec"]) codec = str(answers.get("codec"))
options = _profile_options(codec, specs) profile = key == "profile"
unsupported = _not_supported_by_all(codec, options, specs, profile=True) options = _profile_options(codec, specs) if profile else _level_options(codec, specs)
return "Profile " + _options_prompt( unsupported = _not_supported_by_all(codec, options, specs, profile=profile)
options, return key.title() + " " + _options_prompt(
str(_default_answer(key, answers, specs)),
unsupported,
) + ": "
if key == "level":
codec = str(answers["codec"])
options = _level_options(codec, specs)
unsupported = _not_supported_by_all(codec, options, specs, profile=False)
return "Level " + _options_prompt(
options, options,
str(_default_answer(key, answers, specs)), str(_default_answer(key, answers, specs)),
unsupported, unsupported,
@@ -647,7 +898,7 @@ def _question_for_key(
if key == "audio": if key == "audio":
return f"Audio [{_default_answer(key, answers, specs)}] (opus/aac/flac/copy/none): " return f"Audio [{_default_answer(key, answers, specs)}] (opus/aac/flac/copy/none): "
if key == "audio_bitrate": if key == "audio_bitrate":
return f"{str(answers['audio']).upper()} audio bitrate [{_default_answer(key, answers, specs)}]: " return f"{str(answers.get('audio')).upper()} audio bitrate [{_default_answer(key, answers, specs)}]: "
if key == "audio_pitch": if key == "audio_pitch":
return "Speed-changed audio pitch [preserve] (preserve/shift): " return "Speed-changed audio pitch [preserve] (preserve/shift): "
if key == "confirm": if key == "confirm":
@@ -658,7 +909,7 @@ def _question_for_key(
def _parse_answer( def _parse_answer(
key: str, key: str,
raw: str, raw: str,
answers: dict[str, object], answers: EncodingAnswers,
specs: list[EncodingOutputSpec], specs: list[EncodingOutputSpec],
) -> object: ) -> object:
default = _default_answer(key, answers, specs) default = _default_answer(key, answers, specs)
@@ -698,7 +949,7 @@ def _parse_answer(
if lowered in {"x265", "h265", "hevc", "libx265"}: if lowered in {"x265", "h265", "hevc", "libx265"}:
return "libx265" return "libx265"
if lowered in {"copy", "c"}: if lowered in {"copy", "c"}:
if not answers.get("_video_copy_available"): if not answers.video_copy_available:
raise ValueError( raise ValueError(
"Video copy is unavailable because a validated script changes " "Video copy is unavailable because a validated script changes "
"video frames or properties." "video frames or properties."
@@ -714,12 +965,13 @@ def _parse_answer(
if lowered not in ENCODER_PRESETS: if lowered not in ENCODER_PRESETS:
raise ValueError(f"Preset must be one of: {'/'.join(ENCODER_PRESETS)}.") raise ValueError(f"Preset must be one of: {'/'.join(ENCODER_PRESETS)}.")
return lowered return lowered
if key == "profile": if key in {"profile", "level"}:
options = _profile_options(str(answers["codec"]), specs) options = (
return _parse_option("profile", value, options) _profile_options(str(answers.get("codec")), specs)
if key == "level": if key == "profile"
options = _level_options(str(answers["codec"]), specs) else _level_options(str(answers.get("codec")), specs)
return _parse_option("level", value, options) )
return _parse_option(key, value, options)
if key == "threads": if key == "threads":
if lowered == "auto": if lowered == "auto":
return "auto" return "auto"
@@ -745,18 +997,19 @@ def _parse_answer(
def _default_answer( def _default_answer(
key: str, key: str,
answers: dict[str, object], answers: EncodingAnswers,
specs: list[EncodingOutputSpec], specs: list[EncodingOutputSpec],
) -> object: ) -> object:
if key in answers: value = answers.get(key)
return answers[key] if value is not None:
return value
codec = str(answers.get("codec", "libx265")) codec = str(answers.get("codec", "libx265"))
if key == "confirm": if key == "confirm":
return True return True
if key == "timestamp_names": if key == "timestamp_names":
return True return True
if key == "timezone": if key == "timezone":
return answers.get("_timezone_default", local_timezone()) return answers.timezone_default
if key == "container": if key == "container":
return ".mp4" return ".mp4"
if key == "codec": if key == "codec":
@@ -864,17 +1117,6 @@ def _constraint_spec(spec: EncodingOutputSpec) -> VideoConstraintSpec:
) )
def _constraint_spec_for_item(
item: VideoInput,
timelines: dict[Path, OutputTimeline],
clip_infos: dict[Path, AvisynthClipInfo],
probes_by_path: dict[Path, VideoProbe],
) -> VideoConstraintSpec:
return _constraint_spec(
_encoding_specs([item], timelines, clip_infos, probes_by_path)[0]
)
def _resolve_codec_options( def _resolve_codec_options(
options: VideoCodecOptions, options: VideoCodecOptions,
spec: VideoConstraintSpec, spec: VideoConstraintSpec,
@@ -971,15 +1213,15 @@ def _parse_option(name: str, value: str, options: list[str]) -> str:
def _parse_audio(value: str) -> str: def _parse_audio(value: str) -> str:
lowered = value.strip().lower() lowered = value.strip().lower()
if lowered in {"aac", "a"}: if lowered in {"aac", "a", "reencode", "re-encode"}:
return "aac" return "aac"
if lowered in {"copy", "c"}: if lowered in {"copy", "c", "streamcopy", "stream-copy"}:
return "copy" return "copy"
if lowered in {"none", "no", "n"}: if lowered in {"none", "no", "n", "off", "0"}:
return "none" return "none"
if lowered in {"opus", "o"}: if lowered in {"opus", "o", "libopus"}:
return "opus" return "opus"
if lowered in {"flac", "f"}: if lowered in {"flac", "f", "lossless"}:
return "flac" return "flac"
raise ValueError("Audio must be aac, opus, flac, copy, or none.") raise ValueError("Audio must be aac, opus, flac, copy, or none.")
@@ -988,11 +1230,6 @@ def _yes_no_suffix(default: bool) -> str:
return "Y/n" if default else "y/N" return "Y/n" if default else "y/N"
def _forget(answers: dict[str, object], keys: set[str]) -> None:
for key in keys:
answers.pop(key, None)
def _setting_label(key: str) -> str: def _setting_label(key: str) -> str:
return { return {
"timestamp_names": "timestamp names", "timestamp_names": "timestamp names",
@@ -1013,7 +1250,7 @@ def _setting_label(key: str) -> str:
def _setting_value( def _setting_value(
key: str, key: str,
value: object, value: object,
answers: dict[str, object], answers: EncodingAnswers,
specs: list[EncodingOutputSpec], specs: list[EncodingOutputSpec],
) -> str: ) -> str:
if key == "timestamp_names": if key == "timestamp_names":
@@ -1049,7 +1286,7 @@ def _setting_value(
def _filename_preview( def _filename_preview(
answers: dict[str, object], answers: EncodingAnswers,
specs: list[EncodingOutputSpec], specs: list[EncodingOutputSpec],
*, *,
limit: int, limit: int,
@@ -1152,14 +1389,9 @@ def prepare_audio_source(
def _format_audio_options(options: AudioEncodeOptions) -> str: def _format_audio_options(options: AudioEncodeOptions) -> str:
if options.mode == "none": if options.mode == "none":
return "none" return "none"
if options.mode == "aac": if options.mode in {"aac", "opus"}:
pitch = "preserve pitch" if options.preserve_pitch else "shift pitch with speed" pitch = "preserve pitch" if options.preserve_pitch else "shift pitch with speed"
return f"AAC {options.bitrate}, {pitch}" return f"{options.mode.upper()} {options.bitrate}, {pitch}"
if options.mode == "opus": return {"flac": "FLAC", "copy": "copy with best-effort beginning/end trim"}.get(
pitch = "preserve pitch" if options.preserve_pitch else "shift pitch with speed" options.mode, options.mode
return f"Opus {options.bitrate}, {pitch}" )
if options.mode == "flac":
return "FLAC"
if options.mode == "copy":
return "copy with best-effort beginning/end trim"
return options.mode
+75
View File
@@ -0,0 +1,75 @@
from __future__ import annotations
from dataclasses import dataclass
_MATRIX = {1: "bt709", 5: "smpte170m", 6: "smpte170m", 9: "bt2020nc", 10: "bt2020nc"}
_PRIMARIES = {1: "bt709", 9: "bt2020", 12: "smpte432"}
_TRANSFER = {1: "bt709", 16: "smpte2084", 18: "arib-std-b67"}
_RANGE = {0: "tv", 1: "pc"}
@dataclass(frozen=True)
class ColorMetadata:
matrix: int | None
primaries: int | None
transfer: int | None
color_range: int | None
@property
def colorspace(self) -> str | None:
return _MATRIX.get(self.matrix)
def ffmpeg_args(self) -> list[str]:
values = [
("-colorspace", self.colorspace),
("-color_primaries", _PRIMARIES.get(self.primaries)),
("-color_trc", _TRANSFER.get(self.transfer)),
("-color_range", _RANGE.get(self.color_range)),
]
return [part for option, value in values if value is not None for part in (option, value)]
def x264_args(self) -> list[str]:
values = [
("--colormatrix", self.colorspace),
("--colorprim", _PRIMARIES.get(self.primaries)),
("--transfer", _TRANSFER.get(self.transfer)),
("--range", _RANGE.get(self.color_range)),
]
return [part for option, value in values if value is not None for part in (option, value)]
def x265_params(self) -> str:
values = [
("colorprim", _PRIMARIES.get(self.primaries)),
("transfer", _TRANSFER.get(self.transfer)),
("colormatrix", self.colorspace),
("range", "full" if self.color_range == 1 else "limited" if self.color_range == 0 else None),
]
return ":".join(f"{name}={value}" for name, value in values if value is not None)
def mkvmerge_args(self) -> list[str]:
values = [
("--color-matrix-coefficients", self.matrix),
("--color-primaries", self.primaries),
("--color-transfer-characteristics", self.transfer),
("--color-range", {0: 1, 1: 2}.get(self.color_range)),
]
return [part for option, value in values if value is not None for part in (option, f"0:{value}")]
def hevc_bitstream_filter(self) -> str | None:
values = [
("colour_primaries", self.primaries),
("transfer_characteristics", self.transfer),
("matrix_coefficients", self.matrix),
("video_full_range_flag", self.color_range),
]
params = ":".join(f"{name}={value}" for name, value in values if value is not None)
return f"hevc_metadata={params}" if params else None
def expected_probe_values(self) -> dict[str, str | None]:
return {
"matrix": {1: "Rec709", 5: "Rec601", 6: "Rec601", 9: "Rec2020", 10: "Rec2020"}.get(self.matrix),
"primaries": _PRIMARIES.get(self.primaries),
"transfer": _TRANSFER.get(self.transfer),
"range": _RANGE.get(self.color_range),
}
+141 -106
View File
@@ -10,7 +10,9 @@ from pathlib import Path
from threading import Thread from threading import Thread
from typing import Callable from typing import Callable
from tools.avisynth_paths import avs_path
from tools.console import PipelineProgressView, ProgressView from tools.console import PipelineProgressView, ProgressView
from tools.video_color import ColorMetadata
from tools.video_formatting import chroma_family from tools.video_formatting import chroma_family
@@ -46,6 +48,31 @@ class AudioEncodeOptions:
preserve_pitch: bool = True preserve_pitch: bool = True
@dataclass(frozen=True)
class VideoEncodeRequest:
y4m_command: list[str]
ffmpeg: Path
script: Path
audio_source: Path
output: Path
options: VideoCodecOptions
audio_options: AudioEncodeOptions
audio_start: float
audio_duration: float | None
audio_segments: list[tuple[float, float]] | None
audio_tempo: float
audio_sample_rate: int | None
source_has_audio: bool
allow_audio_copy: bool
pixel_format: str
colorspace: str | None
color_metadata: ColorMetadata | None
static_hdr: tuple[str | None, int | None, int | None, bool] | None
dynamic_hdr10plus: Path | None
frame_count: int | None
progress_label: str
def default_audio_bitrate(mode: str) -> str: def default_audio_bitrate(mode: str) -> str:
return {"aac": "192k", "opus": "128k"}.get(mode, "") return {"aac": "192k", "opus": "128k"}.get(mode, "")
@@ -136,14 +163,24 @@ def supported_pixel_formats(ffmpeg: Path, codec: str, family: str) -> list[tuple
return [(8, f"yuv{family}p")] if family == "420" else [] return [(8, f"yuv{family}p")] if family == "420" else []
def ffmpeg_colorspace_from_matrix(matrix: int | None) -> str | None: def add_x265_color_metadata(command: list[str], color_metadata: ColorMetadata) -> None:
if matrix == 1: value = color_metadata.x265_params()
return "bt709" if value:
if matrix in {5, 6}: add_codec_params(command, "-x265-params", value)
return "smpte170m"
if matrix in {9, 10}:
return "bt2020nc" def x264_static_hdr_options(
return None static_hdr: tuple[str | None, int | None, int | None, bool] | None,
) -> list[str]:
if static_hdr is None:
return []
mastering_display, max_content, max_average, _ = static_hdr
options: list[str] = []
if mastering_display:
options.extend(["--mastering-display", mastering_display])
if max_content is not None:
options.extend(["--cll", f"{max_content},{max_average or 0}"])
return options
def render_audio_wav( def render_audio_wav(
@@ -197,6 +234,8 @@ def remux_video(
command.extend(["-c:a", "copy"]) command.extend(["-c:a", "copy"])
else: else:
add_audio_encoder_options(command, audio_options) add_audio_encoder_options(command, audio_options)
if output.suffix.lower() == ".mp4":
command.extend(["-movflags", "+write_colr"])
command.append(str(output)) command.append(str(output))
process = subprocess.Popen( process = subprocess.Popen(
command, command,
@@ -222,60 +261,20 @@ def remux_video(
) )
def encode_video_only_y4m( def encode_video_only_y4m(request: VideoEncodeRequest) -> VideoEncodeResult:
*, request.output.parent.mkdir(parents=True, exist_ok=True)
y4m_command: list[str], command = ffmpeg_y4m_command(request)
ffmpeg: Path,
script: Path,
audio_source: Path | None,
output: Path,
options: VideoCodecOptions | None = None,
audio_options: AudioEncodeOptions | None = None,
audio_start: float = 0.0,
audio_duration: float | None = None,
audio_segments: list[tuple[float, float]] | None = None,
audio_tempo: float = 1.0,
audio_sample_rate: int | None = None,
source_has_audio: bool = False,
allow_audio_copy: bool = False,
pixel_format: str = "yuv420p",
colorspace: str | None = None,
frame_count: int | None = None,
progress_label: str | None = None,
) -> VideoEncodeResult:
if options is None:
options = VideoCodecOptions(codec="libx264", crf=16, preset="slow")
if audio_options is None:
audio_options = AudioEncodeOptions(mode="none")
output.parent.mkdir(parents=True, exist_ok=True)
command = ffmpeg_y4m_command(
ffmpeg=ffmpeg,
audio_source=audio_source,
output=output,
options=options,
audio_options=audio_options,
audio_start=audio_start,
audio_duration=audio_duration,
audio_segments=audio_segments,
audio_tempo=audio_tempo,
audio_sample_rate=audio_sample_rate,
source_has_audio=source_has_audio,
allow_audio_copy=allow_audio_copy,
pixel_format=pixel_format,
colorspace=colorspace,
)
run = run_y4m_encoder( run = run_y4m_encoder(
y4m_command=y4m_command, y4m_command=request.y4m_command,
script=script, script=request.script,
encoder_command=command, encoder_command=command,
frame_count=frame_count, frame_count=request.frame_count,
progress_label=progress_label or output.name, progress_label=request.progress_label,
read_progress=read_ffmpeg_progress, read_progress=read_ffmpeg_progress,
) )
if run.encoder_returncode != 0: if run.encoder_returncode != 0:
raise RuntimeError( raise RuntimeError(
f"FFmpeg failed for {script} with exit {run.encoder_returncode}:\n" f"FFmpeg failed for {request.script} with exit {run.encoder_returncode}:\n"
+ run.encoder_log + run.encoder_log
+ ( + (
"\nAvisynth runner also exited with " "\nAvisynth runner also exited with "
@@ -287,71 +286,61 @@ def encode_video_only_y4m(
) )
if run.renderer_returncode != 0: if run.renderer_returncode != 0:
raise RuntimeError( raise RuntimeError(
f"Avisynth runner failed for {script} with exit {run.renderer_returncode}:\n" f"Avisynth runner failed for {request.script} with exit {run.renderer_returncode}:\n"
+ run.renderer_log + run.renderer_log
) )
return VideoEncodeResult( return VideoEncodeResult(
script=script, script=request.script,
output=output, output=request.output,
renderer_returncode=run.renderer_returncode, renderer_returncode=run.renderer_returncode,
ffmpeg_returncode=run.encoder_returncode, ffmpeg_returncode=run.encoder_returncode,
encoder_log=run.encoder_log, encoder_log=run.encoder_log,
) )
def ffmpeg_y4m_command( def ffmpeg_y4m_command(request: VideoEncodeRequest) -> list[str]:
*,
ffmpeg: Path,
audio_source: Path | None,
output: Path,
options: VideoCodecOptions,
audio_options: AudioEncodeOptions,
audio_start: float,
audio_duration: float | None,
audio_segments: list[tuple[float, float]] | None,
audio_tempo: float,
audio_sample_rate: int | None,
source_has_audio: bool,
allow_audio_copy: bool,
pixel_format: str,
colorspace: str | None,
) -> list[str]:
command = [ command = [
str(ffmpeg), "-y", "-hide_banner", "-loglevel", "info", "-nostats", str(request.ffmpeg), "-y", "-hide_banner", "-loglevel", "info", "-nostats",
"-progress", "pipe:2", "-f", "yuv4mpegpipe", "-i", "pipe:0", "-progress", "pipe:2", "-f", "yuv4mpegpipe", "-i", "pipe:0",
] ]
if audio_options.mode != "none" and source_has_audio: if request.audio_options.mode != "none" and request.source_has_audio:
if audio_source is None:
raise RuntimeError("An audio source is required when audio encoding is enabled")
add_audio_input_options( add_audio_input_options(
command, command,
audio_options=audio_options, audio_options=request.audio_options,
audio_start=audio_start, audio_start=request.audio_start,
audio_duration=audio_duration, audio_duration=request.audio_duration,
allow_audio_copy=allow_audio_copy, allow_audio_copy=request.allow_audio_copy,
) )
command.extend(["-i", str(audio_source)]) command.extend(["-i", str(request.audio_source)])
command.extend([ command.extend([
"-map", "0:v:0", "-c:v", options.codec, "-preset", options.preset, "-map", "0:v:0", "-c:v", request.options.codec, "-preset", request.options.preset,
"-crf", str(options.crf), "-pix_fmt", pixel_format, "-crf", str(request.options.crf), "-pix_fmt", request.pixel_format,
]) ])
add_video_profile_level(command, options) add_video_profile_level(command, request.options)
add_video_threads(command, options) add_video_threads(command, request.options)
if colorspace is not None: if request.options.codec == "libx265" and request.color_metadata:
command.extend(["-colorspace", colorspace]) add_x265_color_metadata(command, request.color_metadata)
add_static_hdr_metadata(command, request.options, request.static_hdr)
add_dynamic_hdr10plus_metadata(command, request.options, request.dynamic_hdr10plus)
if request.color_metadata is not None:
command.extend(request.color_metadata.ffmpeg_args())
elif request.colorspace is not None:
command.extend(["-colorspace", request.colorspace])
add_audio_options( add_audio_options(
command, command,
audio_options=audio_options, audio_options=request.audio_options,
audio_start=audio_start, audio_start=request.audio_start,
audio_duration=audio_duration, audio_duration=request.audio_duration,
audio_segments=audio_segments, audio_segments=request.audio_segments,
audio_tempo=audio_tempo, audio_tempo=request.audio_tempo,
audio_sample_rate=audio_sample_rate, audio_sample_rate=request.audio_sample_rate,
source_has_audio=source_has_audio, source_has_audio=request.source_has_audio,
allow_audio_copy=allow_audio_copy, allow_audio_copy=request.allow_audio_copy,
) )
command.append(str(output)) if request.output.suffix.lower() == ".mp4" and request.color_metadata and request.color_metadata.ffmpeg_args():
command.extend(["-movflags", "+write_colr"])
command.append(str(request.output))
return command return command
@@ -404,7 +393,7 @@ def video_only_script(script: Path) -> Iterator[Path]:
wrapper = script.with_name(f".{script.name}.video-only.avs") wrapper = script.with_name(f".{script.name}.video-only.avs")
wrapper.write_text( wrapper.write_text(
'mbt_video_only = true\n' 'mbt_video_only = true\n'
f'Import("{_avs_path(script.name)}")\n' f'Import("{avs_path(script.name)}")\n'
"last\n", "last\n",
encoding="utf-8", encoding="utf-8",
) )
@@ -414,10 +403,6 @@ def video_only_script(script: Path) -> Iterator[Path]:
wrapper.unlink(missing_ok=True) wrapper.unlink(missing_ok=True)
def _avs_path(path: str) -> str:
return path.replace("\\", "/").replace('"', '\\"')
def add_audio_input_options( def add_audio_input_options(
command: list[str], command: list[str],
*, *,
@@ -490,11 +475,61 @@ def add_video_threads(command: list[str], options: VideoCodecOptions) -> None:
if options.threads is None: if options.threads is None:
return return
if options.codec == "libx265": if options.codec == "libx265":
command.extend(["-x265-params", f"pools={options.threads}"]) add_codec_params(command, "-x265-params", f"pools={options.threads}")
else: else:
command.extend(["-threads", str(options.threads)]) command.extend(["-threads", str(options.threads)])
def add_static_hdr_metadata(
command: list[str],
options: VideoCodecOptions,
static_hdr: tuple[str | None, int | None, int | None, bool] | None,
) -> None:
if static_hdr is None:
return
mastering_display, max_content, max_average, is_pq = static_hdr
params: list[str] = ["hdr10=1"] if options.codec == "libx265" and is_pq else []
if mastering_display:
params.append(f"master-display={mastering_display}")
if max_content is not None:
params.append(f"max-cll={max_content},{max_average or 0}")
if not params:
return
option = "-x265-params" if options.codec == "libx265" else "-x264-params"
add_codec_params(command, option, ":".join(params))
def add_dynamic_hdr10plus_metadata(
command: list[str],
options: VideoCodecOptions,
metadata_json: Path | None,
) -> None:
if metadata_json is None:
return
if options.codec != "libx265":
raise RuntimeError("HDR10+ re-encoding requires x265")
# FFmpeg passes x265 options as a colon-separated string. Escaping keeps
# the Windows drive separator inside the JSON path value.
json_path = x265_parameter_path(metadata_json.resolve())
add_codec_params(
command,
"-x265-params",
f"dhdr10-info={json_path}:dhdr10-opt=1",
)
def x265_parameter_path(path: Path) -> str:
return str(path).replace("\\", "/").replace(":", r"\:")
def add_codec_params(command: list[str], option: str, value: str) -> None:
if option in command:
index = command.index(option) + 1
command[index] = f"{command[index]}:{value}"
else:
command.extend([option, value])
def print_encoder_statistics(encoder_log: str) -> None: def print_encoder_statistics(encoder_log: str) -> None:
statistics = encoder_statistics_lines(encoder_log) statistics = encoder_statistics_lines(encoder_log)
if not statistics: if not statistics:
+23 -5
View File
@@ -5,6 +5,7 @@ from pathlib import Path
from tools.avisynth_render import AvisynthClipInfo from tools.avisynth_render import AvisynthClipInfo
from tools.video_encode_output import AudioEncodeOptions from tools.video_encode_output import AudioEncodeOptions
from tools.video_formatting import rate_value
from tools.video_probe import VideoProbe from tools.video_probe import VideoProbe
from tools.video_timeline import OutputTimeline from tools.video_timeline import OutputTimeline
@@ -24,10 +25,6 @@ def preserves_entire_source_timeline(timeline: OutputTimeline) -> bool:
) )
def missing_source_frames_between_kept(timeline: OutputTimeline) -> int:
return missing_source_frames([frame.source_frame for frame in timeline.frames])
def missing_source_frames(source_frames: list[int]) -> int: def missing_source_frames(source_frames: list[int]) -> int:
missing = 0 missing = 0
for previous, current in zip(source_frames, source_frames[1:]): for previous, current in zip(source_frames, source_frames[1:]):
@@ -106,14 +103,35 @@ def encoded_duration_seconds(
def has_avisynth_speed_changes( def has_avisynth_speed_changes(
timelines: dict[Path, OutputTimeline], timelines: dict[Path, OutputTimeline],
clip_infos: dict[Path, AvisynthClipInfo], clip_infos: dict[Path, AvisynthClipInfo],
probes_by_path: dict[Path, VideoProbe],
) -> bool: ) -> bool:
for path, timeline in timelines.items(): for path, timeline in timelines.items():
clip_info = clip_infos.get(path) clip_info = clip_infos.get(path)
if clip_info is not None and avisynth_duration_differs(clip_info, timeline): if clip_info is None or not avisynth_duration_differs(clip_info, timeline):
continue
probe = probes_by_path.get(path)
if probe is not None and _matches_source_rate(clip_info, probe):
continue
if clip_info.duration_seconds is not None:
return True return True
return False return False
def _matches_source_rate(clip_info: AvisynthClipInfo, probe: VideoProbe) -> bool:
if clip_info.fps_num <= 0 or clip_info.fps_den <= 0:
return False
clip_rate = clip_info.fps_num / clip_info.fps_den
source_rates = [
rate_value(probe.avg_frame_rate),
rate_value(probe.r_frame_rate),
]
return any(
rate is not None and abs(clip_rate - rate) / rate <= 0.005
for rate in source_rates
if rate is not None and rate > 0
)
def audio_tempo_factor(timeline: OutputTimeline, output_duration: float) -> float: def audio_tempo_factor(timeline: OutputTimeline, output_duration: float) -> float:
if output_duration <= 0: if output_duration <= 0:
raise RuntimeError("Encoded output duration must be positive") raise RuntimeError("Encoded output duration must be positive")
+1 -342
View File
@@ -2,8 +2,6 @@ from __future__ import annotations
import os import os
import sys import sys
from dataclasses import dataclass
from datetime import timezone
from pathlib import Path from pathlib import Path
from tools.avisynth_runner import ( from tools.avisynth_runner import (
@@ -11,20 +9,7 @@ from tools.avisynth_runner import (
runner_set_from_env_or_bundled, runner_set_from_env_or_bundled,
validate_runner_path, validate_runner_path,
) )
from tools.console import prompt_input, prompt_yes_no from tools.console import prompt_input
from tools.timezones import local_timezone, parse_timezone_offset, timezone_to_string
from tools.video_encode_output import (
ENCODER_PRESETS,
AudioEncodeOptions,
VideoCodecOptions,
default_audio_bitrate,
)
@dataclass(frozen=True)
class OutputNamingOptions:
use_timestamps: bool
timezone_value: timezone
def ask_ffms2_plugin_path() -> Path | None: def ask_ffms2_plugin_path() -> Path | None:
@@ -79,329 +64,3 @@ def ask_avisynth_runners() -> AvisynthRunnerSet | None:
return None return None
runner = validate_runner_path(answer) runner = validate_runner_path(answer)
return AvisynthRunnerSet(default=runner, runner64=runner) return AvisynthRunnerSet(default=runner, runner64=runner)
def require_avisynth_runners_for_encode() -> AvisynthRunnerSet:
runners = runner_set_from_env_or_bundled()
if runners is not None:
return runners
if not sys.stdin.isatty():
raise RuntimeError(
"Build tools/avisynth_runner/mbt_avs_runner on Linux, provide "
"tools/avisynth_runner/mbt_avs_runner.exe on Windows, or set "
"MBT_AVS_RUNNER before enabling video encode. For 32-bit Windows "
"AviSynth, set MBT_AVS_RUNNER32."
)
print(
"\nEncoding needs media-batch-tools' AviSynth runner executable "
"(mbt_avs_runner or mbt_avs_runner.exe)."
)
answer = prompt_input("AviSynth runner executable path: ").strip()
if not answer:
raise RuntimeError("Avisynth runner path is required for encoding.")
runner = validate_runner_path(answer)
return AvisynthRunnerSet(default=runner, runner64=runner)
def ask_encode_video_only() -> bool:
env_value = os.environ.get("MBT_ENCODE_VIDEO")
if env_value is not None:
return env_value.strip().lower() in {"1", "true", "yes", "y", "on"}
if not sys.stdin.isatty():
return False
return prompt_yes_no("Encode validated outputs now?", default=False)
def ask_output_naming_options(
*,
default_timezone: timezone | None = None,
) -> OutputNamingOptions:
env_value = os.environ.get("MBT_VIDEO_TIMESTAMP_NAMES")
if env_value is not None:
use_timestamps = env_value.strip().lower() in {"1", "true", "yes", "y", "on"}
elif sys.stdin.isatty():
use_timestamps = prompt_yes_no(
"Name encoded files from their edited beginning timestamp?",
default=True,
)
else:
use_timestamps = False
default_timezone = default_timezone or local_timezone()
env_timezone = os.environ.get("MBT_VIDEO_TIMEZONE", "").strip()
if not use_timestamps:
timezone_value = default_timezone
elif env_timezone:
try:
timezone_value = parse_timezone_offset(env_timezone)
except ValueError as exc:
raise RuntimeError(f"Invalid MBT_VIDEO_TIMEZONE: {exc}") from exc
elif sys.stdin.isatty():
while True:
answer = prompt_input(
"Output filename timezone "
f"[{timezone_to_string(default_timezone)}]: "
).strip()
if not answer:
timezone_value = default_timezone
break
try:
timezone_value = parse_timezone_offset(answer)
break
except ValueError as exc:
print(exc)
else:
timezone_value = default_timezone
return OutputNamingOptions(
use_timestamps=use_timestamps,
timezone_value=timezone_value,
)
def ask_video_codec_options() -> VideoCodecOptions:
codec = _codec_from_env()
extension = _container_extension_from_env()
crf = _int_from_env("MBT_VIDEO_CRF")
preset = os.environ.get("MBT_VIDEO_PRESET", "").strip()
profile = _optional_env("MBT_VIDEO_PROFILE")
level = _optional_env("MBT_VIDEO_LEVEL")
threads = _threads_from_env()
if sys.stdin.isatty():
if extension is None:
extension = _prompt_container_extension()
if codec is None:
codec = _prompt_codec()
if codec != "copy" and crf is None:
default_crf = 16 if codec == "libx264" else 21
crf = _prompt_int(f"CRF [{default_crf}]: ", default=default_crf)
if codec != "copy" and not preset:
preset = _prompt_preset(default="slow")
if codec != "copy" and "MBT_VIDEO_THREADS" not in os.environ:
threads = _prompt_threads(default=None)
else:
extension = extension or ".mp4"
codec = codec or "libx265"
crf = crf if crf is not None else (16 if codec == "libx264" else 21)
preset = preset or "slow"
return VideoCodecOptions(
codec=codec,
crf=crf if crf is not None else 21,
preset=preset or "slow",
extension=extension,
profile=profile,
level=level,
threads=threads,
)
def _prompt_preset(*, default: str) -> str:
options = "/".join(ENCODER_PRESETS)
while True:
value = prompt_input(f"Encoder preset [{default}] ({options}): ").strip().lower()
if not value:
return default
if value in ENCODER_PRESETS:
return value
print(f"Preset must be one of: {options}.")
def _threads_from_env() -> int | None:
raw = os.environ.get("MBT_VIDEO_THREADS", "").strip().lower()
if not raw or raw == "auto":
return None
try:
value = int(raw)
except ValueError as exc:
raise RuntimeError("MBT_VIDEO_THREADS must be auto or a positive integer") from exc
if value < 1:
raise RuntimeError("MBT_VIDEO_THREADS must be auto or a positive integer")
return value
def _prompt_threads(*, default: int | None) -> int | None:
default_text = str(default) if default is not None else "auto"
while True:
value = prompt_input(f"Encoder threads [{default_text}] (auto or positive integer): ").strip().lower()
if not value or value == "auto":
return default
try:
threads = int(value)
except ValueError:
print("Threads must be auto or a positive integer.")
continue
if threads >= 1:
return threads
print("Threads must be auto or a positive integer.")
def ask_audio_options(
*,
has_speed_changes: bool,
container_extension: str,
) -> AudioEncodeOptions:
env_mode = os.environ.get("MBT_VIDEO_AUDIO", "").strip().lower()
env_bitrate = os.environ.get("MBT_VIDEO_AUDIO_BITRATE", "").strip()
preserve_pitch = ask_audio_pitch_option(has_speed_changes=has_speed_changes)
if env_mode:
mode = _normalize_audio_mode(env_mode)
return AudioEncodeOptions(
mode=mode,
bitrate=env_bitrate or default_audio_bitrate(mode),
preserve_pitch=preserve_pitch,
)
if not sys.stdin.isatty():
return AudioEncodeOptions(mode="none", preserve_pitch=preserve_pitch)
while True:
prompt = "Audio: Enter=" + (
"Opus" if container_extension == ".mkv" else "AAC"
) + ", a=AAC, o=Opus, f=FLAC, c=copy/trim audio, n=no audio: "
answer = prompt_input(prompt).strip().lower()
if not answer:
mode = "opus" if container_extension == ".mkv" else "aac"
break
if answer in {"a", "aac"}:
mode = "aac"
break
if answer in {"o", "opus"}:
mode = "opus"
break
if answer in {"f", "flac"}:
mode = "flac"
break
if answer in {"c", "copy"}:
mode = "copy"
break
if answer in {"n", "no", "none"}:
mode = "none"
break
print("Please choose Enter, c, or n.")
bitrate = default_audio_bitrate(mode)
if mode in {"aac", "opus"}:
label = "AAC" if mode == "aac" else "Opus"
bitrate = prompt_input(f"{label} audio bitrate [{bitrate}]: ").strip() or bitrate
return AudioEncodeOptions(mode=mode, bitrate=bitrate, preserve_pitch=preserve_pitch)
def ask_audio_pitch_option(*, has_speed_changes: bool) -> bool:
env_value = os.environ.get("MBT_VIDEO_AUDIO_PITCH", "").strip().lower()
if env_value:
if env_value in {"preserve", "preserved", "keep", "same"}:
return True
if env_value in {"shift", "change", "changed", "speed"}:
return False
raise RuntimeError("MBT_VIDEO_AUDIO_PITCH must be preserve or shift")
if not has_speed_changes or not sys.stdin.isatty():
return True
while True:
answer = prompt_input(
"Speed-changed audio pitch: Enter=preserve pitch, s=shift with speed: "
).strip().lower()
if not answer or answer in {"p", "preserve", "keep", "same"}:
return True
if answer in {"s", "shift", "change", "speed"}:
return False
print("Please choose Enter or s.")
def _normalize_audio_mode(value: str) -> str:
if value in {"aac", "reencode", "re-encode"}:
return "aac"
if value in {"opus", "libopus"}:
return "opus"
if value in {"flac", "lossless"}:
return "flac"
if value in {"copy", "streamcopy", "stream-copy"}:
return "copy"
if value in {"none", "no", "off", "0"}:
return "none"
raise RuntimeError("MBT_VIDEO_AUDIO must be aac, opus, flac, copy, or none")
def _container_extension_from_env() -> str | None:
raw = os.environ.get("MBT_VIDEO_CONTAINER", "").strip().lower()
if not raw:
return None
return _normalize_container_extension(raw)
def _prompt_container_extension() -> str:
while True:
answer = prompt_input("Output container: Enter=MP4, m=MKV: ").strip().lower()
if not answer:
return ".mp4"
try:
return _normalize_container_extension(answer)
except ValueError as exc:
print(exc)
def _normalize_container_extension(value: str) -> str:
if value in {"mp4", ".mp4"}:
return ".mp4"
if value in {"m", "mkv", ".mkv", "matroska"}:
return ".mkv"
raise ValueError("container must be mp4 or mkv")
def _codec_from_env() -> str | None:
raw = os.environ.get("MBT_VIDEO_CODEC", "").strip().lower()
if not raw:
return None
return _normalize_codec(raw)
def _prompt_codec() -> str:
while True:
answer = prompt_input("Video codec: x264, Enter=x265, c=copy: ").strip().lower()
if not answer:
return "libx265"
try:
return _normalize_codec(answer)
except ValueError as exc:
print(exc)
def _normalize_codec(value: str) -> str:
if value in {"x264", "h264", "libx264"}:
return "libx264"
if value in {"x265", "h265", "hevc", "libx265"}:
return "libx265"
if value in {"c", "copy", "streamcopy", "stream-copy"}:
return "copy"
raise ValueError("codec must be x264, x265, or copy")
def _int_from_env(name: str) -> int | None:
raw = os.environ.get(name, "").strip()
if not raw:
return None
try:
return int(raw)
except ValueError as exc:
raise RuntimeError(f"{name} must be an integer") from exc
def _optional_env(name: str) -> str | None:
value = os.environ.get(name, "").strip()
return value or None
def _prompt_int(prompt: str, *, default: int) -> int:
while True:
answer = prompt_input(prompt).strip()
if not answer:
return default
try:
return int(answer)
except ValueError:
print("Please enter an integer.")
+103 -13
View File
@@ -8,11 +8,12 @@ from datetime import datetime, timedelta, timezone
from pathlib import Path from pathlib import Path
from tools.console import prompt_input, prompt_yes_no from tools.console import prompt_input, prompt_yes_no
from tools.video_color import ColorMetadata
from tools.exiftool import run_exiftool_command from tools.exiftool import run_exiftool_command
from tools.filesystem import unique_path
from tools.filenames import format_rounded_filename_stem, round_datetime_to_second from tools.filenames import format_rounded_filename_stem, round_datetime_to_second
from tools.metadata_copy import copy_meaningful_metadata from tools.metadata_copy import copy_meaningful_metadata
from tools.video_inputs import VideoInput from tools.video_inputs import VideoInput
from tools.video_options import OutputNamingOptions
from tools.video_probe import VideoProbe, probe_video from tools.video_probe import VideoProbe, probe_video
from tools.video_timeline import OutputTimeline from tools.video_timeline import OutputTimeline
from tools.video_formatting import join_parts, video_parts from tools.video_formatting import join_parts, video_parts
@@ -24,6 +25,12 @@ class EncodedItem:
output: Path output: Path
@dataclass(frozen=True)
class OutputNamingOptions:
use_timestamps: bool
timezone_value: timezone
def planned_output_path( def planned_output_path(
*, *,
script: Path, script: Path,
@@ -65,12 +72,11 @@ def output_begin_timestamp(
if probe.duration_seconds is None: if probe.duration_seconds is None:
raise RuntimeError(f"No source duration found for {probe.path}") raise RuntimeError(f"No source duration found for {probe.path}")
source_end = probe.metadata_end source_begin = _stretched_source_begin_timestamp(probe, timeline)
if source_end.tzinfo is None:
source_end = source_end.replace(tzinfo=timezone.utc)
source_begin = source_end - timedelta(seconds=probe.duration_seconds)
first_frame_start = timeline.frames[0].start first_frame_start = timeline.frames[0].start
return (source_begin + timedelta(seconds=first_frame_start)).astimezone( return (source_begin + timedelta(
seconds=first_frame_start * timeline.absolute_timestretch
)).astimezone(
output_timezone output_timezone
) )
@@ -78,10 +84,47 @@ def output_begin_timestamp(
def output_end_timestamp( def output_end_timestamp(
probe: VideoProbe, probe: VideoProbe,
timeline: OutputTimeline, timeline: OutputTimeline,
output_duration: float,
) -> datetime: ) -> datetime:
begin = output_begin_timestamp(probe, timeline, timezone.utc) source_begin = _stretched_source_begin_timestamp(probe, timeline)
return begin + timedelta(seconds=output_duration) return source_begin + timedelta(
seconds=_last_consumed_source_end(probe, timeline)
* timeline.absolute_timestretch
)
def _stretched_source_begin_timestamp(
probe: VideoProbe,
timeline: OutputTimeline,
) -> datetime:
if probe.metadata_end is None:
raise RuntimeError(f"No metadata end timestamp found for {probe.path}")
if probe.duration_seconds is None:
raise RuntimeError(f"No source duration found for {probe.path}")
source_end = probe.metadata_end
if source_end.tzinfo is None:
source_end = source_end.replace(tzinfo=timezone.utc)
return source_end - timedelta(
seconds=probe.duration_seconds * timeline.absolute_timestretch
)
def _last_consumed_source_end(probe: VideoProbe, timeline: OutputTimeline) -> float:
source_frames = timeline.consumed_source_frames or [
frame.source_frame for frame in timeline.frames
]
if not source_frames:
raise RuntimeError(f"No source frames were consumed for {probe.path}")
source_frame = max(source_frames)
if source_frame >= len(probe.frame_timestamps):
raise RuntimeError(
f"source frame {source_frame} is outside probed frame table for {probe.path}"
)
start = probe.frame_timestamps[source_frame]
if source_frame + 1 < len(probe.frame_timestamps):
return probe.frame_timestamps[source_frame + 1]
if probe.duration_seconds is None:
raise RuntimeError(f"No source duration found for {probe.path}")
return max(start, probe.duration_seconds)
def write_video_end_timestamp( def write_video_end_timestamp(
@@ -89,13 +132,15 @@ def write_video_end_timestamp(
output: Path, output: Path,
probe: VideoProbe, probe: VideoProbe,
timeline: OutputTimeline, timeline: OutputTimeline,
output_duration: float,
) -> None: ) -> None:
if output.suffix.lower() not in {".mp4", ".mov"}: if output.suffix.lower() not in {".mp4", ".mov"}:
print(f" skipped container timestamp write for {output.suffix}") print(f" skipped container timestamp write for {output.suffix}")
return return
end_time = round_datetime_to_second( end_time = round_datetime_to_second(
output_end_timestamp(probe, timeline, output_duration).astimezone(timezone.utc) output_end_timestamp(
probe,
timeline,
).astimezone(timezone.utc)
) )
timestamp = end_time.strftime("%Y:%m:%d %H:%M:%S") timestamp = end_time.strftime("%Y:%m:%d %H:%M:%S")
args = [ args = [
@@ -115,11 +160,26 @@ def write_video_end_timestamp(
capture_output=True, capture_output=True,
text=True, text=True,
) )
print(f" wrote video end timestamp: {timestamp} UTC") speed_note = (
""
if timeline.absolute_timestretch == 1
else f" ({timeline.absolute_timestretch:g}x real-time)"
)
print(f" wrote video end timestamp: {timestamp} UTC{speed_note}")
def copy_video_metadata(exiftool: str, source: Path, output: Path) -> None: def copy_video_metadata(exiftool: str, source: Path, output: Path) -> None:
copy_meaningful_metadata(exiftool, source, output) copy_meaningful_metadata(
exiftool,
source,
output,
extra_excluded_tags=[
"--QuickTime:ColorRepresentation",
"--QuickTime:ColorPrimaries",
"--QuickTime:TransferCharacteristics",
"--QuickTime:MatrixCoefficients",
],
)
if output.suffix.lower() in {".mp4", ".mov"}: if output.suffix.lower() in {".mp4", ".mov"}:
print(" copied source metadata") print(" copied source metadata")
else: else:
@@ -189,6 +249,36 @@ def validate_encoded_video(
return probe return probe
def validate_output_color_metadata(
output: Path,
probe: VideoProbe,
timeline: OutputTimeline,
*,
expected_hdr10plus: bool = False,
) -> None:
expected = ColorMetadata(
timeline.matrix, timeline.primaries, timeline.transfer, timeline.color_range
).expected_probe_values()
actual = {
"matrix": probe.color_matrix,
"primaries": probe.color_primaries,
"transfer": probe.color_transfer,
"range": probe.color_range,
}
mismatches = [
f"{name} {actual[name] or 'missing'} (expected {value})"
for name, value in expected.items()
if value is not None and actual[name] != value
]
if mismatches:
raise RuntimeError(
f"Encoded output lost color metadata for {output}: "
+ "; ".join(mismatches)
)
if expected_hdr10plus and not probe.hdr10plus:
raise RuntimeError(f"Encoded output lost HDR10+ metadata for {output}")
def validate_encoded_timestamps( def validate_encoded_timestamps(
output: Path, output: Path,
actual: list[float], actual: list[float],
+153 -33
View File
@@ -116,6 +116,15 @@ class VideoProbe:
gps_latitude: float | None gps_latitude: float | None
gps_longitude: float | None gps_longitude: float | None
warnings: list[str] warnings: list[str]
color_range: str | None = None
color_transfer: str | None = None
color_primaries: str | None = None
mastering_display: str | None = None
max_content_light: int | None = None
max_frame_average_light: int | None = None
hdr_peak_luminance: float | None = None
hdr_dynamic: bool = False
hdr10plus: bool = False
def require_tool(name: str) -> str: def require_tool(name: str) -> str:
@@ -169,6 +178,19 @@ def probe_video(ffprobe: str, path: Path, metadata: dict) -> VideoProbe:
if timing.timing_kind == "unknown": if timing.timing_kind == "unknown":
warnings.append("Not enough frame timestamps to classify CFR/VFR timing.") warnings.append("Not enough frame timestamps to classify CFR/VFR timing.")
color_space = _as_str(stream.get("color_space"))
color_range = _as_str(stream.get("color_range"))
color_transfer = _as_str(stream.get("color_transfer"))
color_primaries = _as_str(stream.get("color_primaries"))
side_data = list(stream.get("side_data_list") or [])
if is_hdr_transfer(color_transfer):
# HEVC commonly stores HDR10 static and HDR10+ dynamic metadata on
# decoded frames, not the stream header.
side_data.extend(_probe_first_frame_side_data(ffprobe, path))
mastering_display, max_content_light, max_frame_average_light, hdr_peak_luminance, hdr_dynamic, hdr10plus = (
_hdr_side_data(side_data)
)
return VideoProbe( return VideoProbe(
path=path, path=path,
codec=_as_str(stream.get("codec_name")), codec=_as_str(stream.get("codec_name")),
@@ -177,8 +199,8 @@ def probe_video(ffprobe: str, path: Path, metadata: dict) -> VideoProbe:
pixel_format=_as_str(stream.get("pix_fmt")), pixel_format=_as_str(stream.get("pix_fmt")),
bit_depth=_video_bit_depth(stream), bit_depth=_video_bit_depth(stream),
video_bit_rate=video_bit_rate, video_bit_rate=video_bit_rate,
color_space=_as_str(stream.get("color_space")), color_space=color_space,
color_matrix=_avisynth_matrix_from_color_space(_as_str(stream.get("color_space"))), color_matrix=_avisynth_matrix_from_color_space(color_space),
expected_color_matrix=_expected_color_matrix( expected_color_matrix=_expected_color_matrix(
_to_int(stream.get("width")), _to_int(stream.get("width")),
_to_int(stream.get("height")), _to_int(stream.get("height")),
@@ -201,26 +223,29 @@ def probe_video(ffprobe: str, path: Path, metadata: dict) -> VideoProbe:
gps_latitude=gps_latitude, gps_latitude=gps_latitude,
gps_longitude=gps_longitude, gps_longitude=gps_longitude,
warnings=warnings, warnings=warnings,
color_range=color_range,
color_transfer=color_transfer,
color_primaries=color_primaries,
mastering_display=mastering_display,
max_content_light=max_content_light,
max_frame_average_light=max_frame_average_light,
hdr_peak_luminance=hdr_peak_luminance,
hdr_dynamic=hdr_dynamic,
hdr10plus=hdr10plus,
) )
def _probe_stream(ffprobe: str, path: Path) -> tuple[dict, dict]: def _probe_stream(ffprobe: str, path: Path) -> tuple[dict, dict]:
command = [ data = _ffprobe_json(
ffprobe, ffprobe,
"-v", path,
"error",
"-select_streams", "-select_streams",
"v:0", "v:0",
"-show_entries", "-show_entries",
"stream=codec_name,width,height,pix_fmt,bits_per_raw_sample,bits_per_sample,bit_rate,color_space,color_range,color_transfer,color_primaries,avg_frame_rate,r_frame_rate,time_base,duration,nb_frames", "stream=codec_name,width,height,pix_fmt,bits_per_raw_sample,bits_per_sample,bit_rate,color_space,color_range,color_transfer,color_primaries,avg_frame_rate,r_frame_rate,time_base,duration,nb_frames:stream_side_data=side_data_type,red_x,red_y,green_x,green_y,blue_x,blue_y,white_point_x,white_point_y,min_luminance,max_luminance,max_content,max_average",
"-show_entries", "-show_entries",
"format=duration", "format=duration",
"-of", )
"json",
str(path),
]
completed = subprocess.run(command, check=True, capture_output=True, text=True)
data = json.loads(completed.stdout or "{}")
streams = data.get("streams") or [] streams = data.get("streams") or []
if not streams: if not streams:
raise RuntimeError(f"No video stream found in {path}") raise RuntimeError(f"No video stream found in {path}")
@@ -232,22 +257,16 @@ def _probe_audio_streams(
path: Path, path: Path,
fallback_duration: float | None, fallback_duration: float | None,
) -> list[AudioStreamProbe]: ) -> list[AudioStreamProbe]:
command = [ data = _ffprobe_json(
ffprobe, ffprobe,
"-v", path,
"error",
"-select_streams", "-select_streams",
"a", "a",
"-show_entries", "-show_entries",
"stream=index,codec_name,channels,sample_rate,bit_rate,duration", "stream=index,codec_name,channels,sample_rate,bit_rate,duration",
"-show_entries", "-show_entries",
"packet=stream_index,size", "packet=stream_index,size",
"-of", )
"json",
str(path),
]
completed = subprocess.run(command, check=True, capture_output=True, text=True)
data = json.loads(completed.stdout or "{}")
streams = data.get("streams") or [] streams = data.get("streams") or []
packet_bytes: Counter[int] = Counter() packet_bytes: Counter[int] = Counter()
for packet in data.get("packets") or []: for packet in data.get("packets") or []:
@@ -272,21 +291,30 @@ def _probe_audio_streams(
] ]
def _probe_video_packets(ffprobe: str, path: Path) -> tuple[list[float], int]: def _probe_first_frame_side_data(ffprobe: str, path: Path) -> list[dict]:
command = [ data = _ffprobe_json(
ffprobe, ffprobe,
"-v", path,
"error", "-select_streams",
"v:0",
"-read_intervals",
"%+#1",
"-show_entries",
"frame_side_data=side_data_type,red_x,red_y,green_x,green_y,blue_x,blue_y,white_point_x,white_point_y,min_luminance,max_luminance,max_content,max_average",
)
frames = data.get("frames") or []
return list((frames[0] if frames else {}).get("side_data_list") or [])
def _probe_video_packets(ffprobe: str, path: Path) -> tuple[list[float], int]:
data = _ffprobe_json(
ffprobe,
path,
"-select_streams", "-select_streams",
"v:0", "v:0",
"-show_entries", "-show_entries",
"packet=pts_time,size", "packet=pts_time,size",
"-of", )
"json",
str(path),
]
completed = subprocess.run(command, check=True, capture_output=True, text=True)
data = json.loads(completed.stdout or "{}")
timestamps: list[float] = [] timestamps: list[float] = []
total_bytes = 0 total_bytes = 0
for packet in data.get("packets") or []: for packet in data.get("packets") or []:
@@ -297,6 +325,16 @@ def _probe_video_packets(ffprobe: str, path: Path) -> tuple[list[float], int]:
return sorted(timestamps), total_bytes return sorted(timestamps), total_bytes
def _ffprobe_json(ffprobe: str, path: Path, *args: str) -> dict:
completed = subprocess.run(
[ffprobe, "-v", "error", *args, "-of", "json", str(path)],
check=True,
capture_output=True,
text=True,
)
return json.loads(completed.stdout or "{}")
def _average_bit_rate(total_bytes: int, duration: float | None) -> int | None: def _average_bit_rate(total_bytes: int, duration: float | None) -> int | None:
if total_bytes <= 0 or duration is None or duration <= 0: if total_bytes <= 0 or duration is None or duration <= 0:
return None return None
@@ -465,7 +503,9 @@ def _to_float(value: object) -> float | None:
if value is None or value == "N/A": if value is None or value == "N/A":
return None return None
try: try:
return float(value) text = str(value)
numerator, separator, denominator = text.partition("/")
return float(numerator) / float(denominator) if separator else float(text)
except (TypeError, ValueError): except (TypeError, ValueError):
return None return None
@@ -474,7 +514,8 @@ def _to_int(value: object) -> int | None:
if value is None or value == "N/A": if value is None or value == "N/A":
return None return None
try: try:
return int(value) numeric = _to_float(value)
return int(numeric) if numeric is not None else None
except (TypeError, ValueError): except (TypeError, ValueError):
return None return None
@@ -507,6 +548,85 @@ def _avisynth_matrix_from_color_space(color_space: str | None) -> str | None:
return None return None
def avisynth_primaries_from_color_primaries(value: str | None) -> int:
normalized = (value or "").lower()
if normalized in {"bt709", "smpte170m", "bt470bg", "smpte240m"}:
return 1
if normalized in {"bt2020", "bt2020nc", "bt2020ncl"}:
return 9
if normalized in {"smpte432", "displayp3", "display-p3"}:
return 12
return 2
def avisynth_transfer_from_color_transfer(value: str | None) -> int:
normalized = (value or "").lower()
if normalized in {"bt709", "smpte170m", "bt470bg", "bt601"}:
return 1
if normalized in {"smpte2084", "pq"}:
return 16
if normalized in {"arib-std-b67", "hlg"}:
return 18
return 2
def avisynth_range_from_color_range(value: str | None) -> int:
return 1 if (value or "").lower() in {"pc", "jpeg", "full"} else 0
def is_hdr_transfer(value: str | None) -> bool:
return (value or "").lower() in {"smpte2084", "pq", "arib-std-b67", "hlg"}
def hdr_kind(value: str | None, *, dynamic: bool = False) -> str | None:
if not is_hdr_transfer(value):
return None
if dynamic:
return "HDR10+"
return "HDR10" if (value or "").lower() in {"smpte2084", "pq"} else "HLG"
def _hdr_side_data(
side_data: list[dict],
) -> tuple[str | None, int | None, int | None, float | None, bool, bool]:
mastering: str | None = None
max_content: int | None = None
max_average: int | None = None
peak_luminance: float | None = None
dynamic = False
hdr10plus = False
for entry in side_data:
kind = str(entry.get("side_data_type") or "").lower()
if "mastering display" in kind:
mastering = _format_mastering_display(entry)
peak_luminance = _to_float(entry.get("max_luminance"))
elif "content light" in kind:
max_content = _to_int(entry.get("max_content"))
max_average = _to_int(entry.get("max_average"))
elif "dynamic hdr" in kind or "hdr10+" in kind or "dovi" in kind:
dynamic = True
hdr10plus = hdr10plus or "smpte2094-40" in kind or "hdr10+" in kind
return mastering, max_content, max_average, peak_luminance, dynamic, hdr10plus
def _format_mastering_display(entry: dict) -> str | None:
keys = (
"green_x", "green_y", "blue_x", "blue_y", "red_x", "red_y",
"white_point_x", "white_point_y", "max_luminance", "min_luminance",
)
values = [_to_float(entry.get(key)) for key in keys]
if any(value is None for value in values):
return None
green_x, green_y, blue_x, blue_y, red_x, red_y, white_x, white_y, maximum, minimum = values
return (
f"G({round(green_x * 50000)},{round(green_y * 50000)})"
f"B({round(blue_x * 50000)},{round(blue_y * 50000)})"
f"R({round(red_x * 50000)},{round(red_y * 50000)})"
f"WP({round(white_x * 50000)},{round(white_y * 50000)})"
f"L({round(maximum * 10000)},{round(minimum * 10000)})"
)
def _expected_color_matrix(width: int | None, height: int | None) -> str | None: def _expected_color_matrix(width: int | None, height: int | None) -> str | None:
if width is None or height is None: if width is None or height is None:
return None return None
+22 -4
View File
@@ -20,7 +20,7 @@ from tools.video_formatting import (
) )
from tools.video_inputs import VideoInput from tools.video_inputs import VideoInput
from tools.video_outputs import output_begin_timestamp from tools.video_outputs import output_begin_timestamp
from tools.video_probe import VideoProbe from tools.video_probe import VideoProbe, hdr_kind
from tools.video_timeline import OutputTimeline from tools.video_timeline import OutputTimeline
@@ -252,6 +252,9 @@ def print_probe_summary(inputs: list[VideoInput], probes: list[VideoProbe]) -> N
f"{format_frame_count(probe.stream_frame_count, probe.timing.frame_count)}" f"{format_frame_count(probe.stream_frame_count, probe.timing.frame_count)}"
) )
print(f" video: {_format_probe_video_line(probe)}") print(f" video: {_format_probe_video_line(probe)}")
hdr_line = _format_hdr_line(probe)
if hdr_line:
print(f" HDR: {hdr_line}")
if probe.audio_streams: if probe.audio_streams:
for index, stream in enumerate(probe.audio_streams, start=1): for index, stream in enumerate(probe.audio_streams, start=1):
print(f" audio {index}: {_format_audio_stream(stream)}") print(f" audio {index}: {_format_audio_stream(stream)}")
@@ -272,8 +275,7 @@ def print_probe_summary(inputs: list[VideoInput], probes: list[VideoProbe]) -> N
def _format_probe_video_line(probe: VideoProbe) -> str: def _format_probe_video_line(probe: VideoProbe) -> str:
return join_parts( parts = video_parts(
video_parts(
codec=probe.codec or "unknown codec", codec=probe.codec or "unknown codec",
width=probe.width, width=probe.width,
height=probe.height, height=probe.height,
@@ -282,7 +284,23 @@ def _format_probe_video_line(probe: VideoProbe) -> str:
matrix=probe.color_matrix or "unknown", matrix=probe.color_matrix or "unknown",
bit_rate=probe.video_bit_rate, bit_rate=probe.video_bit_rate,
) )
) kind = hdr_kind(probe.color_transfer, dynamic=probe.hdr10plus)
if kind:
parts.append(kind)
return join_parts(parts)
def _format_hdr_line(probe: VideoProbe) -> str | None:
kind = hdr_kind(probe.color_transfer, dynamic=probe.hdr10plus)
if kind is None:
return None
parts = ["PQ" if (probe.color_transfer or "").lower() == "smpte2084" else "HLG"]
parts.append("full range" if (probe.color_range or "").lower() == "pc" else "limited range")
if probe.max_content_light is not None:
parts.append(f"MaxCLL {probe.max_content_light}")
if probe.max_frame_average_light is not None:
parts.append(f"MaxFALL {probe.max_frame_average_light}")
return join_parts(parts)
def _format_audio_stream(stream) -> str: def _format_audio_stream(stream) -> str:
+42
View File
@@ -28,6 +28,10 @@ class OutputTimeline:
dropped_frame_count: int dropped_frame_count: int
source_frame_count: int source_frame_count: int
matrix: int | None matrix: int | None
absolute_timestretch: float = 1.0
primaries: int | None = None
transfer: int | None = None
color_range: int | None = None
def build_output_timeline( def build_output_timeline(
@@ -40,6 +44,10 @@ def build_output_timeline(
audio_segments: list[tuple[float, float]] = [] audio_segments: list[tuple[float, float]] = []
consumed_source_frames: list[int] = [] consumed_source_frames: list[int] = []
matrices: set[int] = set() matrices: set[int] = set()
primaries_values: set[int] = set()
transfers: set[int] = set()
ranges: set[int] = set()
timestretches: set[float] = set()
dropped_count = 0 dropped_count = 0
leading_trimmed = False leading_trimmed = False
last_consumed_source_frame: int | None = None last_consumed_source_frame: int | None = None
@@ -79,6 +87,14 @@ def build_output_timeline(
consumed_source_frames.append(frame.source_frame) consumed_source_frames.append(frame.source_frame)
if frame.matrix is not None: if frame.matrix is not None:
matrices.add(frame.matrix) matrices.add(frame.matrix)
if frame.primaries is not None:
primaries_values.add(frame.primaries)
if frame.transfer is not None:
transfers.add(frame.transfer)
if frame.color_range is not None:
ranges.add(frame.color_range)
if frame.absolute_timestretch is not None:
timestretches.add(frame.absolute_timestretch)
_append_audio_segment( _append_audio_segment(
audio_segments, audio_segments,
start=probe.frame_timestamps[frame.source_frame], start=probe.frame_timestamps[frame.source_frame],
@@ -102,6 +118,20 @@ def build_output_timeline(
f"Output frames have mixed _Matrix values for {probe.path}: " f"Output frames have mixed _Matrix values for {probe.path}: "
+ ", ".join(str(value) for value in sorted(matrices)) + ", ".join(str(value) for value in sorted(matrices))
) )
_require_one_color_value("_Primaries", primaries_values, probe)
_require_one_color_value("_Transfer", transfers, probe)
_require_one_color_value("_ColorRange", ranges, probe)
if len(timestretches) > 1:
raise RuntimeError(
f"Output frames have mixed absolute timestretch values for {probe.path}: "
+ ", ".join(str(value) for value in sorted(timestretches))
)
absolute_timestretch = next(iter(timestretches), 1.0)
if absolute_timestretch <= 0:
raise RuntimeError(
f"Output frames have an invalid absolute timestretch for {probe.path}: "
f"{absolute_timestretch}"
)
source_count = len(probe.frame_timestamps) source_count = len(probe.frame_timestamps)
beginning_trimmed = leading_trimmed or kept_frames[0].source_frame > 0 beginning_trimmed = leading_trimmed or kept_frames[0].source_frame > 0
@@ -120,9 +150,21 @@ def build_output_timeline(
dropped_frame_count=dropped_count, dropped_frame_count=dropped_count,
source_frame_count=source_count, source_frame_count=source_count,
matrix=next(iter(matrices)) if matrices else None, matrix=next(iter(matrices)) if matrices else None,
absolute_timestretch=absolute_timestretch,
primaries=next(iter(primaries_values)) if primaries_values else None,
transfer=next(iter(transfers)) if transfers else None,
color_range=next(iter(ranges)) if ranges else None,
) )
def _require_one_color_value(name: str, values: set[int], probe: VideoProbe) -> None:
if len(values) > 1:
raise RuntimeError(
f"Output frames have mixed {name} values for {probe.path}: "
+ ", ".join(str(value) for value in sorted(values))
)
def _append_audio_segment( def _append_audio_segment(
segments: list[tuple[float, float]], segments: list[tuple[float, float]],
*, *,
+70 -122
View File
@@ -4,12 +4,14 @@ import re
import subprocess import subprocess
import sys import sys
import tempfile import tempfile
from dataclasses import replace
from pathlib import Path from pathlib import Path
from tools.console import PipelineProgressView, ProgressView from tools.console import PipelineProgressView, ProgressView
from tools.video_color import ColorMetadata
from tools.video_encode_output import ( from tools.video_encode_output import (
AudioEncodeOptions, AudioEncodeOptions,
VideoCodecOptions, VideoEncodeRequest,
VideoEncodeResult, VideoEncodeResult,
add_audio_input_options, add_audio_input_options,
add_audio_options, add_audio_options,
@@ -17,78 +19,46 @@ from tools.video_encode_output import (
pixel_format_bit_depth, pixel_format_bit_depth,
read_ffmpeg_progress, read_ffmpeg_progress,
run_y4m_encoder, run_y4m_encoder,
x264_static_hdr_options,
) )
def encode_video_with_timestamps( def encode_video_with_timestamps(
*, *,
y4m_command: list[str], request: VideoEncodeRequest,
ffmpeg: Path,
mkvmerge: Path, mkvmerge: Path,
x264: Path | None, x264: Path | None,
script: Path,
frame_durations: list[float], frame_durations: list[float],
audio_source: Path,
output: Path,
options: VideoCodecOptions,
audio_options: AudioEncodeOptions,
audio_start: float,
audio_duration: float | None,
audio_segments: list[tuple[float, float]] | None,
audio_tempo: float,
audio_sample_rate: int | None,
source_has_audio: bool,
allow_audio_copy: bool,
pixel_format: str,
colorspace: str | None,
frame_count: int | None = None,
progress_label: str | None = None,
) -> VideoEncodeResult: ) -> VideoEncodeResult:
if not frame_durations: if not frame_durations:
raise RuntimeError("Cannot encode VFR output with no frame durations") raise RuntimeError("Cannot encode VFR output with no frame durations")
output.parent.mkdir(parents=True, exist_ok=True) request.output.parent.mkdir(parents=True, exist_ok=True)
# Network shares may allow the final output but deny creating temporary folders. # Network shares may allow the final output but deny creating temporary folders.
with tempfile.TemporaryDirectory(prefix="mbt-vfr-") as temp_dir: with tempfile.TemporaryDirectory(prefix="mbt-vfr-") as temp_dir:
temp_root = Path(temp_dir) temp_root = Path(temp_dir)
timecodes = temp_root / "frames.timecodes.txt" timecodes = temp_root / "frames.timecodes.txt"
write_timecode_v2(timecodes, frame_durations) write_timecode_v2(timecodes, frame_durations)
intermediate = temp_root / ( intermediate = temp_root / (
"encoded.h264" if options.codec == "libx264" else "encoded.mkv" "encoded.h264" if request.options.codec == "libx264" else "encoded.mkv"
) )
if options.codec == "libx264": if request.options.codec == "libx264":
if x264 is None: if x264 is None:
raise RuntimeError("x264 CLI is required for timestamp-aware x264 encoding") raise RuntimeError("x264 CLI is required for timestamp-aware x264 encoding")
renderer_returncode, encoder_log = encode_x264_y4m( renderer_returncode, encoder_log = encode_x264_y4m(
y4m_command=y4m_command, request=request,
x264=x264, x264=x264,
script=script,
timecodes=timecodes, timecodes=timecodes,
output=intermediate, output=intermediate,
options=options,
pixel_format=pixel_format,
colorspace=colorspace,
frame_count=len(frame_durations),
progress_label=progress_label,
) )
elif options.codec == "libx265": elif request.options.codec == "libx265":
result = encode_video_only_y4m( result = encode_video_only_y4m(
y4m_command=y4m_command, replace(request, output=intermediate, audio_options=AudioEncodeOptions(mode="none"))
ffmpeg=ffmpeg,
script=script,
audio_source=None,
output=intermediate,
options=options,
audio_options=AudioEncodeOptions(mode="none"),
pixel_format=pixel_format,
colorspace=colorspace,
frame_count=len(frame_durations),
progress_label=progress_label,
) )
renderer_returncode = result.renderer_returncode renderer_returncode = result.renderer_returncode
encoder_log = result.encoder_log encoder_log = result.encoder_log
else: else:
raise RuntimeError(f"Unsupported timestamp-aware codec: {options.codec}") raise RuntimeError(f"Unsupported timestamp-aware codec: {request.options.codec}")
timed_video = temp_root / "timed.mkv" timed_video = temp_root / "timed.mkv"
apply_video_timecodes( apply_video_timecodes(
@@ -96,27 +66,16 @@ def encode_video_with_timestamps(
source=intermediate, source=intermediate,
timecodes=timecodes, timecodes=timecodes,
output=timed_video, output=timed_video,
color_metadata=request.color_metadata,
) )
ffmpeg_returncode = mux_timed_video_with_audio( ffmpeg_returncode = mux_timed_video_with_audio(
ffmpeg=ffmpeg, request=request,
timed_video=timed_video, timed_video=timed_video,
audio_source=audio_source,
output=output,
audio_options=audio_options,
audio_start=audio_start,
audio_duration=audio_duration,
audio_segments=audio_segments,
audio_tempo=audio_tempo,
audio_sample_rate=audio_sample_rate,
source_has_audio=source_has_audio,
allow_audio_copy=allow_audio_copy,
frame_count=len(frame_durations),
progress_label=progress_label,
) )
return VideoEncodeResult( return VideoEncodeResult(
script=script, script=request.script,
output=output, output=request.output,
renderer_returncode=renderer_returncode, renderer_returncode=renderer_returncode,
ffmpeg_returncode=ffmpeg_returncode, ffmpeg_returncode=ffmpeg_returncode,
encoder_log=encoder_log, encoder_log=encoder_log,
@@ -136,30 +95,22 @@ def write_timecode_v2(path: Path, frame_durations: list[float]) -> None:
def encode_x264_y4m( def encode_x264_y4m(
*, *,
y4m_command: list[str], request: VideoEncodeRequest,
x264: Path, x264: Path,
script: Path,
timecodes: Path, timecodes: Path,
output: Path, output: Path,
options: VideoCodecOptions,
pixel_format: str,
colorspace: str | None,
frame_count: int | None,
progress_label: str | None,
) -> tuple[int, str]: ) -> tuple[int, str]:
run = run_y4m_encoder( run = run_y4m_encoder(
y4m_command=y4m_command, y4m_command=request.y4m_command,
script=script, script=request.script,
encoder_command=x264_command( encoder_command=x264_command(
request=request,
x264=x264, x264=x264,
timecodes=timecodes, timecodes=timecodes,
output=output, output=output,
options=options,
pixel_format=pixel_format,
colorspace=colorspace,
), ),
frame_count=frame_count, frame_count=request.frame_count,
progress_label=progress_label or script.name, progress_label=request.progress_label,
read_progress=read_x264_progress, read_progress=read_x264_progress,
) )
if run.encoder_returncode != 0: if run.encoder_returncode != 0:
@@ -167,12 +118,12 @@ def encode_x264_y4m(
if run.renderer_returncode != 0: if run.renderer_returncode != 0:
details += "\nAvisynth runner:\n" + run.renderer_log details += "\nAvisynth runner:\n" + run.renderer_log
raise RuntimeError( raise RuntimeError(
f"x264 failed for {script} with exit {run.encoder_returncode}:\n" f"x264 failed for {request.script} with exit {run.encoder_returncode}:\n"
+ details.strip() + details.strip()
) )
if run.renderer_returncode != 0: if run.renderer_returncode != 0:
raise RuntimeError( raise RuntimeError(
f"Avisynth runner failed for {script} with exit {run.renderer_returncode}:\n" f"Avisynth runner failed for {request.script} with exit {run.renderer_returncode}:\n"
+ run.renderer_log + run.renderer_log
) )
return run.renderer_returncode, run.encoder_log return run.renderer_returncode, run.encoder_log
@@ -180,12 +131,10 @@ def encode_x264_y4m(
def x264_command( def x264_command(
*, *,
request: VideoEncodeRequest,
x264: Path, x264: Path,
timecodes: Path, timecodes: Path,
output: Path, output: Path,
options: VideoCodecOptions,
pixel_format: str,
colorspace: str | None,
) -> list[str]: ) -> list[str]:
command = [ command = [
str(x264), str(x264),
@@ -194,24 +143,27 @@ def x264_command(
"--tcfile-in", "--tcfile-in",
str(timecodes), str(timecodes),
"--crf", "--crf",
str(options.crf), str(request.options.crf),
"--preset", "--preset",
options.preset, request.options.preset,
"--output-depth", "--output-depth",
str(pixel_format_bit_depth(pixel_format)), str(pixel_format_bit_depth(request.pixel_format)),
"--output-csp", "--output-csp",
_x264_output_csp(pixel_format), _x264_output_csp(request.pixel_format),
"--verbose", "--verbose",
"--no-progress", "--no-progress",
] ]
if options.profile: if request.options.profile:
command.extend(["--profile", options.profile]) command.extend(["--profile", request.options.profile])
if options.level: if request.options.level:
command.extend(["--level", options.level]) command.extend(["--level", request.options.level])
if options.threads is not None: if request.options.threads is not None:
command.extend(["--threads", str(options.threads)]) command.extend(["--threads", str(request.options.threads)])
if colorspace: if request.color_metadata is not None:
command.extend(["--colormatrix", colorspace]) command.extend(request.color_metadata.x264_args())
elif request.colorspace:
command.extend(["--colormatrix", request.colorspace])
command.extend(x264_static_hdr_options(request.static_hdr))
command.extend(["--output", str(output), "-"]) command.extend(["--output", str(output), "-"])
return command return command
@@ -230,6 +182,7 @@ def apply_video_timecodes(
source: Path, source: Path,
timecodes: Path, timecodes: Path,
output: Path, output: Path,
color_metadata: ColorMetadata | None = None,
) -> None: ) -> None:
completed = subprocess.run( completed = subprocess.run(
[ [
@@ -241,6 +194,7 @@ def apply_video_timecodes(
str(output), str(output),
"--timestamps", "--timestamps",
f"0:{timecodes}", f"0:{timecodes}",
*(color_metadata.mkvmerge_args() if color_metadata else []),
str(source), str(source),
], ],
capture_output=True, capture_output=True,
@@ -256,23 +210,11 @@ def apply_video_timecodes(
def mux_timed_video_with_audio( def mux_timed_video_with_audio(
*, *,
ffmpeg: Path, request: VideoEncodeRequest,
timed_video: Path, timed_video: Path,
audio_source: Path,
output: Path,
audio_options: AudioEncodeOptions,
audio_start: float,
audio_duration: float | None,
audio_segments: list[tuple[float, float]] | None,
audio_tempo: float,
audio_sample_rate: int | None,
source_has_audio: bool,
allow_audio_copy: bool,
frame_count: int | None,
progress_label: str | None,
) -> int: ) -> int:
command = [ command = [
str(ffmpeg), str(request.ffmpeg),
"-y", "-y",
"-hide_banner", "-hide_banner",
"-loglevel", "-loglevel",
@@ -283,31 +225,37 @@ def mux_timed_video_with_audio(
"-i", "-i",
str(timed_video), str(timed_video),
] ]
if audio_options.mode != "none" and source_has_audio: if request.audio_options.mode != "none" and request.source_has_audio:
add_audio_input_options( add_audio_input_options(
command, command,
audio_options=audio_options, audio_options=request.audio_options,
audio_start=audio_start, audio_start=request.audio_start,
audio_duration=audio_duration, audio_duration=request.audio_duration,
allow_audio_copy=allow_audio_copy, allow_audio_copy=request.allow_audio_copy,
) )
command.extend(["-i", str(audio_source)]) command.extend(["-i", str(request.audio_source)])
command.extend(["-map", "0:v:0", "-c:v", "copy"]) command.extend(["-map", "0:v:0", "-c:v", "copy"])
if output.suffix.lower() == ".mp4": if request.color_metadata:
command.extend(["-video_track_timescale", "90000"]) command.extend(request.color_metadata.ffmpeg_args())
if request.options.codec == "libx265":
bitstream_filter = request.color_metadata.hevc_bitstream_filter() if request.color_metadata else None
if bitstream_filter:
command.extend(["-bsf:v", bitstream_filter])
if request.output.suffix.lower() == ".mp4":
command.extend(["-video_track_timescale", "90000", "-movflags", "+write_colr"])
add_audio_options( add_audio_options(
command, command,
audio_options=audio_options, audio_options=request.audio_options,
audio_start=audio_start, audio_start=request.audio_start,
audio_duration=audio_duration, audio_duration=request.audio_duration,
audio_segments=audio_segments, audio_segments=request.audio_segments,
audio_tempo=audio_tempo, audio_tempo=request.audio_tempo,
audio_sample_rate=audio_sample_rate, audio_sample_rate=request.audio_sample_rate,
source_has_audio=source_has_audio, source_has_audio=request.source_has_audio,
allow_audio_copy=allow_audio_copy, allow_audio_copy=request.allow_audio_copy,
) )
command.append(str(output)) command.append(str(request.output))
process = subprocess.Popen( process = subprocess.Popen(
command, command,
@@ -317,12 +265,12 @@ def mux_timed_video_with_audio(
) )
ffmpeg_stderr = read_ffmpeg_progress( ffmpeg_stderr = read_ffmpeg_progress(
process, process,
total_frames=frame_count, total_frames=request.frame_count,
label=f"Muxing {progress_label or output.name}", label=f"Muxing {request.progress_label}",
) )
if process.returncode != 0: if process.returncode != 0:
raise RuntimeError( raise RuntimeError(
f"FFmpeg failed while muxing timestamped output {output} with exit " f"FFmpeg failed while muxing timestamped output {request.output} with exit "
f"{process.returncode}:\n{ffmpeg_stderr}" f"{process.returncode}:\n{ffmpeg_stderr}"
) )
return process.returncode return process.returncode
+8 -11
View File
@@ -4,6 +4,7 @@ import subprocess
import sys import sys
from pathlib import Path from pathlib import Path
from tools.avisynth_paths import avs_path
from tools.console import clear_screen from tools.console import clear_screen
from tools.avisynth_runner import AvisynthRunnerSet from tools.avisynth_runner import AvisynthRunnerSet
from tools.avisynth_render import AvisynthClipInfo, run_validation_script from tools.avisynth_render import AvisynthClipInfo, run_validation_script
@@ -161,7 +162,7 @@ def _run_import_diagnostics(
), ),
( (
"import-helper", "import-helper",
f'Import("{_avs_path(str(helper.resolve()))}")\n' f'Import("{avs_path(str(helper.resolve()))}")\n'
'BlankClip(length=1, width=16, height=16, pixel_type="YV12")\n', 'BlankClip(length=1, width=16, height=16, pixel_type="YV12")\n',
), ),
] ]
@@ -169,7 +170,7 @@ def _run_import_diagnostics(
cases.append( cases.append(
( (
"load-ffms2", "load-ffms2",
f'LoadPlugin("{_avs_path(str(Path(plugin_path).resolve()))}")\n' f'LoadPlugin("{avs_path(str(Path(plugin_path).resolve()))}")\n'
'BlankClip(length=1, width=16, height=16, pixel_type="YV12")\n', 'BlankClip(length=1, width=16, height=16, pixel_type="YV12")\n',
) )
) )
@@ -178,19 +179,19 @@ def _run_import_diagnostics(
( (
"ffvideo", "ffvideo",
_optional_load_plugin(plugin_path) _optional_load_plugin(plugin_path)
+ f'FFVideoSource("{_avs_path(str(Path(source_path).resolve()))}", ' + f'FFVideoSource("{avs_path(str(Path(source_path).resolve()))}", '
+ f'cachefile="{_avs_path(str((source_dir / (stem + ".diag.ffindex")).resolve()))}")\n', + f'cachefile="{avs_path(str((source_dir / (stem + ".diag.ffindex")).resolve()))}")\n',
), ),
( (
"hidden-source", "hidden-source",
"mbt_video_only = true\n" "mbt_video_only = true\n"
f'Import("{_avs_path(str(hidden_source.resolve()))}")\n' f'Import("{avs_path(str(hidden_source.resolve()))}")\n'
"last\n", "last\n",
), ),
( (
"editable", "editable",
"mbt_video_only = true\n" "mbt_video_only = true\n"
f'Import("{_avs_path(str(visible.resolve()))}")\n' f'Import("{avs_path(str(visible.resolve()))}")\n'
"last\n", "last\n",
), ),
] ]
@@ -228,11 +229,7 @@ def _remove_validation_artifacts(csv_path: Path) -> None:
def _optional_load_plugin(plugin_path: str | None) -> str: def _optional_load_plugin(plugin_path: str | None) -> str:
if not plugin_path: if not plugin_path:
return "" return ""
return f'LoadPlugin("{_avs_path(str(Path(plugin_path).resolve()))}")\n' return f'LoadPlugin("{avs_path(str(Path(plugin_path).resolve()))}")\n'
def _avs_path(path: str) -> str:
return path.replace("\\", "/").replace('"', '\\"')
def _indent(text: str) -> str: def _indent(text: str) -> str:
+450 -29
View File
@@ -1,16 +1,22 @@
from __future__ import annotations from __future__ import annotations
import unittest import json
import os
import tempfile import tempfile
import unittest
from contextlib import redirect_stdout from contextlib import redirect_stdout
from dataclasses import replace
from datetime import datetime, timedelta, timezone from datetime import datetime, timedelta, timezone
from io import StringIO from io import StringIO
from pathlib import Path from pathlib import Path
from unittest.mock import patch from unittest.mock import patch
from tools.avisynth_render import AvisynthClipInfo, run_validation_script from tools.avisynth_render import AvisynthClipInfo, run_validation_script
from tools.avisynth_runner import script_requests_32bit, validate_runner_path from tools.avisynth_runner import (
from tools.avisynth_validate import FrameIdentity script_requests_32bit,
validate_runner_path,
)
from tools.avisynth_validate import FrameIdentity, read_frame_identity_csv
from tools.avisynth_workspace import ( from tools.avisynth_workspace import (
_clear_source_root, _clear_source_root,
_format_relative_timestamp, _format_relative_timestamp,
@@ -20,16 +26,22 @@ from tools.executables import require_executable
from tools.video_encode_output import ( from tools.video_encode_output import (
AudioEncodeOptions, AudioEncodeOptions,
VideoCodecOptions, VideoCodecOptions,
VideoEncodeRequest,
add_dynamic_hdr10plus_metadata,
add_x265_color_metadata,
add_video_threads, add_video_threads,
atempo_filters, atempo_filters,
choose_video_pixel_format, choose_video_pixel_format,
encoder_statistics_lines, encoder_statistics_lines,
read_ffmpeg_progress, read_ffmpeg_progress,
speed_change_filters, speed_change_filters,
x265_parameter_path,
) )
from tools.hdr10plus import select_hdr10plus_frames
from tools.video_color import ColorMetadata
from tools.video_timestamp_encode import write_timecode_v2, x264_command from tools.video_timestamp_encode import write_timecode_v2, x264_command
from tools.video_encode_plan import ( from tools.video_encode_plan import (
missing_source_frames_between_kept, has_avisynth_speed_changes,
normal_deleted_frame_count, normal_deleted_frame_count,
preserves_entire_source_timeline, preserves_entire_source_timeline,
y4m_command_for_timeline, y4m_command_for_timeline,
@@ -44,25 +56,45 @@ from tools.video_codec_constraints import (
resolve_profile, resolve_profile,
) )
from tools.video_inputs import VideoInput from tools.video_inputs import VideoInput
from tools.video_options import OutputNamingOptions from tools.video_outputs import (
from tools.video_outputs import output_begin_timestamp, planned_output_path EncodedItem,
OutputNamingOptions,
move_encoded_output_to_source_dir,
output_begin_timestamp,
output_end_timestamp,
planned_output_path,
)
from tools.video_batch_encode import ( from tools.video_batch_encode import (
EncodingAnswers,
EncodingOutputSpec, EncodingOutputSpec,
_default_output_timezone, _default_output_timezone,
_default_answer, _default_answer,
_environment_encoding_settings,
_encoding_steps, _encoding_steps,
_parse_audio, _parse_audio,
_parse_answer, _parse_answer,
_question_for_key, _question_for_key,
_stream_copy_warnings, _stream_copy_warnings,
) )
from tools.video_probe import AudioStreamProbe, VideoProbe, _video_timezone, summarize_frame_timing from tools.video_probe import (
AudioStreamProbe,
VideoProbe,
_hdr_side_data,
_format_mastering_display,
_video_timezone,
hdr_kind,
summarize_frame_timing,
)
from tools.video_reporting import _format_probe_timestamp, _format_vfr_timing from tools.video_reporting import _format_probe_timestamp, _format_vfr_timing
from tools.video_reporting import _format_source_span, format_validation_summary_lines from tools.video_reporting import _format_source_span, format_validation_summary_lines
from tools.console import PipelineProgressView, ProgressView, _progress_bar from tools.console import PipelineProgressView, ProgressView, _progress_bar
from tools.video_encode_output import _update_renderer_progress from tools.video_encode_output import _update_renderer_progress
from tools.video_timeline import OutputFrameTiming, OutputTimeline, build_output_timeline from tools.video_timeline import OutputFrameTiming, OutputTimeline, build_output_timeline
from video_encode import print_workspace_summary, wait_for_script_edits from video_encode import (
print_workspace_summary,
stale_workspace_manifests,
wait_for_script_edits,
)
def make_probe( def make_probe(
@@ -112,6 +144,13 @@ def make_probe(
) )
def make_answers(**values: object) -> EncodingAnswers:
answers = EncodingAnswers(timezone.utc, True)
for key, value in values.items():
answers.set(key, value)
return answers
class VideoCodecConstraintTests(unittest.TestCase): class VideoCodecConstraintTests(unittest.TestCase):
def test_video_copy_is_only_available_for_a_full_source_timeline(self) -> None: def test_video_copy_is_only_available_for_a_full_source_timeline(self) -> None:
source = OutputTimeline( source = OutputTimeline(
@@ -215,6 +254,39 @@ class VideoCodecConstraintTests(unittest.TestCase):
self.assertEqual(selected, "yuv420p10le") self.assertEqual(selected, "yuv420p10le")
def test_x265_color_metadata_sets_hdr_vui_parameters(self) -> None:
command: list[str] = []
add_x265_color_metadata(
command,
ColorMetadata(9, 9, 16, 0),
)
self.assertEqual(
command,
[
"-x265-params",
"colorprim=bt2020:transfer=smpte2084:colormatrix=bt2020nc:range=limited",
],
)
def test_mkvmerge_color_options_carry_hdr_signal(self) -> None:
self.assertEqual(
ColorMetadata(9, 9, 16, 0).mkvmerge_args(),
[
"--color-matrix-coefficients", "0:9",
"--color-primaries", "0:9",
"--color-transfer-characteristics", "0:16",
"--color-range", "0:1",
],
)
def test_hevc_color_bitstream_filter_carries_hdr_signal(self) -> None:
self.assertEqual(
ColorMetadata(9, 9, 16, 0).hevc_bitstream_filter(),
"hevc_metadata=colour_primaries=9:transfer_characteristics=16:matrix_coefficients=9:video_full_range_flag=0",
)
def test_x265_minimum_profile_tracks_depth_and_chroma(self) -> None: def test_x265_minimum_profile_tracks_depth_and_chroma(self) -> None:
self.assertEqual( self.assertEqual(
minimum_profile("libx265", VideoConstraintSpec(1920, 1080, 12, "420", 60)), minimum_profile("libx265", VideoConstraintSpec(1920, 1080, 12, "420", 60)),
@@ -265,7 +337,53 @@ class VideoCodecConstraintTests(unittest.TestCase):
self.assertIsNone(resolve_level("libx265", None, clip)) self.assertIsNone(resolve_level("libx265", None, clip))
class HdrToneMapTests(unittest.TestCase):
def test_mpc_hable_mapping_matches_hdrtools_parameters(self) -> None:
def hable(value: float) -> float:
return (
(value * (0.15 * value + 0.10 * 0.50) + 0.20 * 0.02)
/ (value * (0.15 * value + 0.50) + 0.20 * 0.30)
- 0.02 / 0.30
)
display_nits = 125.0
exposure = 10_000.0 / display_nits
hdrtools_white_scale = 1.0 / hable(4.8)
for linear_pq in (0.0, 0.001, 0.01, 0.1, 0.5, 1.0):
mpc_value = hable(linear_pq * exposure) / hable(4.8)
hdrtools_value = hable(linear_pq * exposure) * hdrtools_white_scale
self.assertAlmostEqual(mpc_value, hdrtools_value, places=12)
def test_luminance_gamut_compression_preserves_luma_and_chroma_direction(self) -> None:
weights = (0.2126, 0.7152, 0.0722)
source = (1.3, 0.2, 0.1)
luma = sum(value * weight for value, weight in zip(source, weights))
minimum = min(source)
maximum = max(source)
saturation = min(
1.0,
(1.0 - luma) / (maximum - luma) if maximum > 1.0 else 1.0,
luma / (luma - minimum) if minimum < 0.0 else 1.0,
)
mapped = tuple(luma + (value - luma) * saturation for value in source)
self.assertTrue(all(0.0 <= value <= 1.0 for value in mapped))
self.assertAlmostEqual(sum(value * weight for value, weight in zip(mapped, weights)), luma)
for original, result in zip(source, mapped):
self.assertAlmostEqual((result - luma) / (original - luma), saturation)
class WorkspaceSummaryTests(unittest.TestCase): class WorkspaceSummaryTests(unittest.TestCase):
def test_stale_workspace_manifests_require_missing_video(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
root = Path(temp_dir)
(root / "kept.mp4").write_bytes(b"")
(root / "kept.mp4.manifest.json").write_text("{}", encoding="utf-8")
missing = root / "missing.mp4.manifest.json"
missing.write_text("{}", encoding="utf-8")
self.assertEqual(stale_workspace_manifests(root), [missing])
def test_source_cleanup_retries_transient_windows_lock(self) -> None: def test_source_cleanup_retries_transient_windows_lock(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir: with tempfile.TemporaryDirectory() as temp_dir:
source_root = Path(temp_dir) / ".source" source_root = Path(temp_dir) / ".source"
@@ -358,20 +476,41 @@ class VideoTimelineTests(unittest.TestCase):
) )
def test_x264_vfr_command_uses_timecodes_and_output_format(self) -> None: def test_x264_vfr_command_uses_timecodes_and_output_format(self) -> None:
options = VideoCodecOptions(
codec="libx264",
crf=18,
preset="slow",
profile="high10",
level="4.1",
threads=3,
)
command = x264_command( command = x264_command(
request=VideoEncodeRequest(
y4m_command=[],
ffmpeg=Path("ffmpeg"),
script=Path("video.avs"),
audio_source=Path("audio.mp4"),
output=Path("video.mp4"),
options=options,
audio_options=AudioEncodeOptions(mode="none"),
audio_start=0,
audio_duration=None,
audio_segments=None,
audio_tempo=1,
audio_sample_rate=None,
source_has_audio=False,
allow_audio_copy=False,
pixel_format="yuv422p10le",
colorspace="bt709",
color_metadata=None,
static_hdr=None,
dynamic_hdr10plus=None,
frame_count=None,
progress_label="video",
),
x264=Path("x264"), x264=Path("x264"),
timecodes=Path("frames.txt"), timecodes=Path("frames.txt"),
output=Path("video.h264"), output=Path("video.h264"),
options=VideoCodecOptions(
codec="libx264",
crf=18,
preset="slow",
profile="high10",
level="4.1",
threads=3,
),
pixel_format="yuv422p10le",
colorspace="bt709",
) )
self.assertIn("--tcfile-in", command) self.assertIn("--tcfile-in", command)
@@ -390,6 +529,91 @@ class VideoTimelineTests(unittest.TestCase):
self.assertEqual(command, ["-x265-params", "pools=3"]) self.assertEqual(command, ["-x265-params", "pools=3"])
def test_color_metadata_carries_hdr_properties(self) -> None:
self.assertEqual(
ColorMetadata(9, 9, 16, 0).ffmpeg_args(),
[
"-colorspace", "bt2020nc",
"-color_primaries", "bt2020",
"-color_trc", "smpte2084",
"-color_range", "tv",
],
)
def test_hdr10_and_mastering_display_detection(self) -> None:
self.assertEqual(hdr_kind("smpte2084"), "HDR10")
self.assertEqual(hdr_kind("smpte2084", dynamic=True), "HDR10+")
self.assertTrue(
_hdr_side_data(
[{"side_data_type": "HDR Dynamic Metadata SMPTE2094-40 (HDR10+)"}]
)[-1]
)
self.assertEqual(
_format_mastering_display(
{
"green_x": "13250/50000",
"green_y": "34500/50000",
"blue_x": "7500/50000",
"blue_y": "3000/50000",
"red_x": "34000/50000",
"red_y": "16000/50000",
"white_point_x": "15635/50000",
"white_point_y": "16450/50000",
"max_luminance": "40000000/10000",
"min_luminance": "50/10000",
}
),
"G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(40000000,50)",
)
def test_hdr10plus_selection_reindexes_scenes(self) -> None:
entries = [
{"LuminanceParameters": {"AverageRGB": value}, "NumberOfWindows": 1,
"TargetedSystemDisplayMaximumLuminance": 1000,
"SceneFrameIndex": index, "SceneId": 0, "SequenceFrameIndex": index}
for index, value in enumerate((10, 10, 20, 20, 30))
]
with tempfile.TemporaryDirectory() as temp_dir:
source = Path(temp_dir) / "source.json"
output = Path(temp_dir) / "selected.json"
source.write_text(json.dumps({"SceneInfo": entries}), encoding="utf-8")
select_hdr10plus_frames(
source_json=source,
source_frames=[1, 2, 4],
output_json=output,
)
selected = json.loads(output.read_text(encoding="utf-8"))
self.assertEqual(
[entry["SequenceFrameIndex"] for entry in selected["SceneInfo"]], [0, 1, 2]
)
self.assertEqual(
[entry["SceneFrameIndex"] for entry in selected["SceneInfo"]], [0, 0, 0]
)
self.assertEqual(selected["SceneInfoSummary"], {
"SceneFirstFrameIndex": [0, 1, 2],
"SceneFrameNumbers": [1, 1, 1],
})
def test_dynamic_hdr10plus_is_passed_to_x265(self) -> None:
command: list[str] = []
add_dynamic_hdr10plus_metadata(
command,
VideoCodecOptions(codec="libx265", crf=21, preset="slow"),
Path("metadata.json"),
)
self.assertEqual(
command,
[
"-x265-params",
f"dhdr10-info={Path('metadata.json').resolve()}:dhdr10-opt=1",
],
)
self.assertEqual(
x265_parameter_path(Path(r"C:\work\metadata.json")),
r"C\:/work/metadata.json",
)
def test_encoder_statistics_keep_frame_and_reference_lines(self) -> None: def test_encoder_statistics_keep_frame_and_reference_lines(self) -> None:
statistics = encoder_statistics_lines( statistics = encoder_statistics_lines(
"x264 [info]: frame I:2 Avg QP:18.00\n" "x264 [info]: frame I:2 Avg QP:18.00\n"
@@ -545,6 +769,83 @@ class VideoTimelineTests(unittest.TestCase):
self.assertEqual([round(frame.duration, 3) for frame in timeline.frames], [0.04, 0.06, 0.06]) self.assertEqual([round(frame.duration, 3) for frame in timeline.frames], [0.04, 0.06, 0.06])
self.assertEqual(timeline.timing_kind, "vfr") self.assertEqual(timeline.timing_kind, "vfr")
def test_timeline_uses_one_validated_absolute_timestretch(self) -> None:
probe = make_probe(timestamps=[0.0, 1.0], duration=2.0)
timeline = build_output_timeline(
[
FrameIdentity(0, 1, 0, absolute_timestretch=0.125),
FrameIdentity(1, 1, 1, absolute_timestretch=0.125),
],
probe=probe,
)
self.assertEqual(timeline.absolute_timestretch, 0.125)
def test_metadata_timestretch_is_not_an_avisynth_speed_change(self) -> None:
probe = make_probe(timestamps=[0.0, 1.0], duration=2.0)
probe = replace(probe, avg_frame_rate="60/1", r_frame_rate="60/1")
timeline = OutputTimeline(
frames=[
OutputFrameTiming(0, 1, 0, 0.0, 1.0),
OutputFrameTiming(1, 1, 1, 1.0, 1.0),
],
audio_segments=[(0.0, 2.0)],
consumed_source_frames=[0, 1],
duration_seconds=2.0,
timing_kind="cfr",
beginning_trimmed=False,
ending_trimmed=False,
dropped_frame_count=0,
source_frame_count=2,
matrix=None,
absolute_timestretch=0.125,
)
clip_info = AvisynthClipInfo(
width=1920,
height=1080,
chroma="420",
bit_depth=8,
frames=123,
fps_num=60,
fps_den=1,
has_audio=True,
audio_rate=48000,
audio_channels=2,
audio_samples=0,
)
self.assertFalse(
has_avisynth_speed_changes(
{probe.path: timeline},
{probe.path: clip_info},
{probe.path: probe},
)
)
def test_validation_csv_carries_output_color_properties(self) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
csv_path = Path(temp_dir) / "frames.csv"
csv_path.write_text(
"output_frame,source_id,source_frame,drop_frame,matrix,primaries,transfer,color_range,absolute_timestretch\n"
"0,1,0,0,9,9,16,0,1\n",
encoding="utf-8",
)
frame = read_frame_identity_csv(csv_path)[0]
self.assertEqual((frame.matrix, frame.primaries, frame.transfer, frame.color_range), (9, 9, 16, 0))
def test_timeline_rejects_mixed_absolute_timestretches(self) -> None:
probe = make_probe(timestamps=[0.0, 1.0], duration=2.0)
with self.assertRaisesRegex(RuntimeError, "mixed absolute timestretch"):
build_output_timeline(
[
FrameIdentity(0, 1, 0, absolute_timestretch=1.0),
FrameIdentity(1, 1, 1, absolute_timestretch=0.5),
],
probe=probe,
)
def test_complex_source_span_is_summarized(self) -> None: def test_complex_source_span_is_summarized(self) -> None:
probe = make_probe( probe = make_probe(
timestamps=[float(index) for index in range(10)], timestamps=[float(index) for index in range(10)],
@@ -661,32 +962,66 @@ class VideoTimelineTests(unittest.TestCase):
timedelta(hours=9), timedelta(hours=9),
) )
def test_environment_encoding_settings_preserve_automation_aliases(self) -> None:
environment = {
"MBT_VIDEO_TIMESTAMP_NAMES": "yes",
"MBT_VIDEO_TIMEZONE": "+02:00",
"MBT_VIDEO_CONTAINER": "matroska",
"MBT_VIDEO_CODEC": "stream-copy",
"MBT_VIDEO_CRF": "18",
"MBT_VIDEO_PRESET": "slower",
"MBT_VIDEO_PROFILE": "main10",
"MBT_VIDEO_LEVEL": "4.1",
"MBT_VIDEO_THREADS": "3",
"MBT_VIDEO_AUDIO": "streamcopy",
"MBT_VIDEO_AUDIO_PITCH": "speed",
}
with patch.dict(os.environ, environment, clear=True):
settings = _environment_encoding_settings(timezone.utc)
self.assertTrue(settings.naming.use_timestamps)
self.assertEqual(settings.naming.timezone_value.utcoffset(None), timedelta(hours=2))
self.assertEqual(settings.codec_options, VideoCodecOptions(
codec="copy", crf=18, preset="slower", extension=".mkv",
profile="main10", level="4.1", threads=3,
))
self.assertEqual(settings.audio_options, AudioEncodeOptions(
mode="copy", bitrate="", preserve_pitch=False,
))
def test_timestamp_naming_off_skips_timezone_step(self) -> None: def test_timestamp_naming_off_skips_timezone_step(self) -> None:
self.assertNotIn( self.assertNotIn(
"timezone", "timezone",
_encoding_steps({"timestamp_names": False}, has_speed_changes=False), _encoding_steps(make_answers(timestamp_names=False), has_speed_changes=False),
) )
def test_threads_setting_follows_level_and_requires_positive_integer(self) -> None: def test_threads_setting_follows_level_and_requires_positive_integer(self) -> None:
steps = _encoding_steps({}, has_speed_changes=False) steps = _encoding_steps(make_answers(), has_speed_changes=False)
self.assertEqual(steps[steps.index("level") + 1], "threads") self.assertEqual(steps[steps.index("level") + 1], "threads")
self.assertEqual(_parse_answer("threads", "auto", {}, []), "auto") self.assertEqual(_parse_answer("threads", "auto", make_answers(), []), "auto")
self.assertEqual(_parse_answer("threads", "2", {}, []), 2) self.assertEqual(_parse_answer("threads", "2", make_answers(), []), 2)
with self.assertRaisesRegex(ValueError, "positive integer"): with self.assertRaisesRegex(ValueError, "positive integer"):
_parse_answer("threads", "0", {}, []) _parse_answer("threads", "0", make_answers(), [])
def test_default_crf_depends_on_codec(self) -> None: def test_default_crf_depends_on_codec(self) -> None:
self.assertEqual(_default_answer("crf", {"codec": "libx264"}, []), 16) self.assertEqual(_default_answer("crf", make_answers(codec="libx264"), []), 16)
self.assertEqual(_default_answer("crf", {"codec": "libx265"}, []), 21) self.assertEqual(_default_answer("crf", make_answers(codec="libx265"), []), 21)
def test_changed_codec_resets_dependent_answers(self) -> None:
answers = make_answers(codec="libx264", crf=16, preset="slow", threads=2)
answers.set("codec", "libx265")
self.assertEqual(_default_answer("crf", answers, []), 21)
self.assertEqual(_default_answer("threads", answers, []), "auto")
def test_container_question_lists_available_options(self) -> None: def test_container_question_lists_available_options(self) -> None:
question = _question_for_key("container", {}, []) question = _question_for_key("container", make_answers(), [])
self.assertIn("(mp4/mkv)", question) self.assertIn("(mp4/mkv)", question)
def test_preset_question_lists_available_options(self) -> None: def test_preset_question_lists_available_options(self) -> None:
question = _question_for_key("preset", {}, []) question = _question_for_key("preset", make_answers(), [])
self.assertIn("ultrafast/superfast/veryfast", question) self.assertIn("ultrafast/superfast/veryfast", question)
self.assertIn("slow/slower/veryslow/placebo", question) self.assertIn("slow/slower/veryslow/placebo", question)
@@ -721,7 +1056,6 @@ class VideoTimelineTests(unittest.TestCase):
matrix=None, matrix=None,
) )
self.assertEqual(missing_source_frames_between_kept(timeline), 1)
self.assertEqual(normal_deleted_frame_count(timeline), 0) self.assertEqual(normal_deleted_frame_count(timeline), 0)
def test_normal_frame_deletion_requires_audio_cut_segments(self) -> None: def test_normal_frame_deletion_requires_audio_cut_segments(self) -> None:
@@ -744,7 +1078,6 @@ class VideoTimelineTests(unittest.TestCase):
matrix=None, matrix=None,
) )
self.assertEqual(missing_source_frames_between_kept(timeline), 2)
self.assertEqual(normal_deleted_frame_count(timeline), 2) self.assertEqual(normal_deleted_frame_count(timeline), 2)
self.assertEqual(timeline.audio_segments, [(0.0, 2.0), (3.0, 1.0), (5.0, 2.0)]) self.assertEqual(timeline.audio_segments, [(0.0, 2.0), (3.0, 1.0), (5.0, 2.0)])
@@ -791,6 +1124,25 @@ class AudioFilterTests(unittest.TestCase):
class OutputNamingTests(unittest.TestCase): class OutputNamingTests(unittest.TestCase):
def test_backup_move_uses_a_unique_workspace_path(self) -> None:
with tempfile.TemporaryDirectory() as temporary:
root = Path(temporary)
source = root / "source" / "original.mp4"
output = root / "encoded.mp4"
source.parent.mkdir()
source.write_bytes(b"source")
output.write_bytes(b"output")
item = EncodedItem(
VideoInput(source, Path("original.mp4"), Path("original.mp4")),
output,
)
backup = move_encoded_output_to_source_dir(root, item, "backup")
self.assertEqual(backup, root / "originals" / "original.mp4")
self.assertEqual(backup.read_bytes(), b"source")
self.assertEqual((source.parent / "encoded.mp4").read_bytes(), b"output")
def test_output_begin_uses_source_end_minus_source_duration_plus_first_frame(self) -> None: def test_output_begin_uses_source_end_minus_source_duration_plus_first_frame(self) -> None:
probe = make_probe( probe = make_probe(
timestamps=[0.0, 1.0, 2.0], timestamps=[0.0, 1.0, 2.0],
@@ -823,6 +1175,55 @@ class OutputNamingTests(unittest.TestCase):
datetime(2026, 6, 2, 12, 0, 2, tzinfo=timezone.utc), datetime(2026, 6, 2, 12, 0, 2, tzinfo=timezone.utc),
) )
def test_absolute_timestretch_changes_metadata_timestamps(self) -> None:
probe = make_probe(
timestamps=[0.0, 27.0],
duration=27.64,
metadata_end=datetime(2026, 7, 23, 16, 57, 11, tzinfo=timezone.utc),
)
timeline = OutputTimeline(
frames=[
OutputFrameTiming(0, 1, 0, 0.0, 27.0),
OutputFrameTiming(1, 1, 1, 27.0, 0.64),
],
audio_segments=[(0.0, 27.64)],
consumed_source_frames=[0, 1],
duration_seconds=27.64,
timing_kind="cfr",
beginning_trimmed=False,
ending_trimmed=False,
dropped_frame_count=0,
source_frame_count=2,
matrix=None,
absolute_timestretch=0.125,
)
self.assertEqual(
output_begin_timestamp(probe, timeline, timezone.utc),
datetime(2026, 7, 23, 16, 57, 7, 545000, tzinfo=timezone.utc),
)
self.assertEqual(
output_end_timestamp(probe, timeline),
probe.metadata_end,
)
trimmed_timeline = replace(
timeline,
frames=[OutputFrameTiming(0, 1, 1, 27.0, 0.64)],
audio_segments=[(27.0, 0.64)],
consumed_source_frames=[1],
duration_seconds=0.64,
beginning_trimmed=True,
)
self.assertEqual(
output_begin_timestamp(probe, trimmed_timeline, timezone.utc),
datetime(2026, 7, 23, 16, 57, 10, 920000, tzinfo=timezone.utc),
)
self.assertEqual(
output_end_timestamp(probe, trimmed_timeline),
probe.metadata_end,
)
def test_planned_output_path_uses_unique_timestamp_name(self) -> None: def test_planned_output_path_uses_unique_timestamp_name(self) -> None:
probe = make_probe( probe = make_probe(
timestamps=[0.0], timestamps=[0.0],
@@ -1071,8 +1472,28 @@ class AvisynthWorkspaceTests(unittest.TestCase):
self.assertIn("} catch (err_msg) {\n mbt_video_only = false", text) self.assertIn("} catch (err_msg) {\n mbt_video_only = false", text)
self.assertIn("mbt_validation_blank = mbt_validation_blank", text) self.assertIn("mbt_validation_blank = mbt_validation_blank", text)
self.assertIn("BlankClip(video, color=$000000)", text) self.assertIn("BlankClip(video, color=$000000)", text)
self.assertIn("global mbt_hdr_peak = 0.000000", text)
self.assertNotIn('Defined("mbt_video_only")', text) self.assertNotIn('Defined("mbt_video_only")', text)
def test_hidden_source_marks_timing_after_conditional_readers(self) -> None:
probe = make_probe(timestamps=[0.0, 0.04], duration=0.08)
text = _hidden_source_script(
VideoInput(Path("source.mp4"), Path("source.mp4"), Path("source.mp4")),
probe,
0,
Path("workspace/.source/source.mp4.source.avs"),
Path("tools/avisynth_helpers.avs"),
None,
Path("workspace/.source/source.mp4.timestamps.txt"),
Path("workspace/.source/source.mp4.frame-times.txt"),
Path("workspace/.source/source.mp4.frame-time-labels.txt"),
Path("workspace/.source/source.mp4.durations.txt"),
Path("workspace/.source/source.mp4.duration-labels.txt"),
)
self.assertLess(text.index("MBT_MarkSource"), text.index("ConditionalReader"))
self.assertGreater(text.index("MBT_MarkTiming(last)"), text.rindex("ConditionalReader"))
def test_validation_wrapper_keeps_audio_for_clip_summary(self) -> None: def test_validation_wrapper_keeps_audio_for_clip_summary(self) -> None:
from tools.avisynth_workspace import _validation_script from tools.avisynth_workspace import _validation_script
+30 -7
View File
@@ -34,7 +34,6 @@ from tools.video_batch_encode import encode_validated_video_only
from tools.video_inputs import VideoInput, collect_video_inputs from tools.video_inputs import VideoInput, collect_video_inputs
from tools.video_options import ( from tools.video_options import (
ask_avisynth_runners, ask_avisynth_runners,
ask_encode_video_only,
ask_ffms2_plugin_path, ask_ffms2_plugin_path,
) )
from tools.video_probe import probe_video, require_tool from tools.video_probe import probe_video, require_tool
@@ -206,7 +205,7 @@ def run(argv: list[str]) -> int:
avs_runners, avs_runners,
ffprobe, ffprobe,
) )
elif ask_encode_video_only(): elif _env_truthy("MBT_ENCODE_VIDEO"):
encode_validated_video_only( encode_validated_video_only(
root, root,
inputs, inputs,
@@ -229,6 +228,10 @@ def run(argv: list[str]) -> int:
return 0 return 0
def _env_truthy(name: str) -> bool:
return os.environ.get(name, "").strip().lower() in {"1", "true", "yes", "y", "on"}
def print_workspace_summary( def print_workspace_summary(
root: Path, root: Path,
inputs: list[VideoInput], inputs: list[VideoInput],
@@ -257,19 +260,27 @@ def print_workspace_summary(
def delete_stale_visible_scripts(root: Path, inputs: list[VideoInput]) -> None: def delete_stale_visible_scripts(root: Path, inputs: list[VideoInput]) -> None:
stale = stale_visible_scripts(root, inputs) stale = stale_visible_scripts(root, inputs)
if not stale: stale_manifests = stale_workspace_manifests(root)
if not stale and not stale_manifests:
return return
print("\nOld editable Avisynth scripts not used by this batch:") if stale:
for script in stale: print("\nOld editable Avisynth scripts not used by this batch:")
print(f" - {script}") for script in stale:
print(f" - {script}")
if stale_manifests:
print("\nStale video manifest files:")
for manifest in stale_manifests:
print(f" - {manifest}")
if sys.stdin.isatty(): if sys.stdin.isatty():
delete = prompt_yes_no("Delete old editable .avs scripts?", default=True) delete = prompt_yes_no("Delete listed stale workspace files?", default=True)
else: else:
delete = False delete = False
if not delete: if not delete:
return return
for script in stale: for script in stale:
script.unlink() script.unlink()
for manifest in stale_manifests:
manifest.unlink()
def stale_visible_scripts(root: Path, inputs: list[VideoInput]) -> list[Path]: def stale_visible_scripts(root: Path, inputs: list[VideoInput]) -> list[Path]:
@@ -286,6 +297,18 @@ def stale_visible_scripts(root: Path, inputs: list[VideoInput]) -> list[Path]:
return sorted(stale) return sorted(stale)
def stale_workspace_manifests(root: Path) -> list[Path]:
if not root.exists():
return []
return sorted(
manifest
for manifest in root.rglob("*.manifest.json")
if not manifest.with_name(
manifest.name.removesuffix(".manifest.json")
).is_file()
)
def preflight_kept_visible_scripts( def preflight_kept_visible_scripts(
*, *,
avs_runners, avs_runners,