- Generated by
1.18.0
Directories | |
| app | |
| models | |
Current ACMX project version: 2.136.0.
ACMXVK is an in-progress Vulkan port of the ACMX2 real-time video shader engine. The goal is to preserve ACMX2's workflow and behavior while replacing the MX2/OpenGL rendering path with the installed MXVK engine and Vulkan SPIR-V shaders.
The port is currently at Increment 10A (HDR Increment 6 of 6). It is usable for video, camera, and still-image shader processing, but it is not yet a complete replacement for ACMX2.
| Area | Status | Notes |
|---|---|---|
| Standalone CMake project | Complete | Builds against installed MXVK and MXWrite and provides an uninstall target. |
| Runtime resource paths | Implemented | ACMX2-compatible -p/--path, ACMXVK_PATH, and build/install fallbacks resolve data, internal shaders, shader libraries, playlists, and MIDI examples. ACMXVK_SHADER_PATH supplies a default SPIR-V library. |
| Window and Vulkan lifecycle | Complete | MXVK owns the window, device, swapchain, rendering, screenshots, and validation integration. |
| Video, camera, and image input | Complete | Video files prefer MXVK's FFmpeg capture with CUDA/NVDEC when available and fall back to OpenCV; an MXVK CUDA installation sends NVDEC frames directly to Vulkan independently of the acidcam-gpu build option. --use-source-fps provides real-time effects playback on the source-reported video clock, waiting when early and skipping decode work when late. Camera devices use ACMX2-compatible resolution, pixel-format, buffer, and FPS negotiation and report both the negotiated mode and measured delivery rate. --enumerate-device probes native V4L2 modes on Linux, AVFoundation modes on macOS, and DirectShow modes on Windows. Camera recordings use real-time PTS so expensive 4K processing preserves wall-clock duration even below the nominal FPS. --maximize-fps decouples camera acquisition from Vulkan presentation. Encoded output follows --resolution when supplied and otherwise uses the negotiated source dimensions, including a width/height swap for 90-degree input rotation. In either case, oversized previews are fitted to the usable display without changing the render or output dimensions. Still images remain OpenCV-backed. |
| HDR video | Implemented (6/6) | Detects 10-bit BT.2020/PQ/HLG input, captures mastering-display/content-light metadata, decodes HDR files to native RGBA16, uploads an R16G16B16A16_UNORM source texture, and preserves HDR precision through RGBA16F scene, multipass, compute, history, crossfade, and 3D final targets. PQ and HLG are decoded to linear BT.2020 before effects and encoded afterward. Video recording sends normalized RGBA16 directly to MXWrite for HEVC Main10/YUV420P10LE encoding with source color and luminance metadata, timestamps, and the existing audio muxing behavior. Windowed preview and ordinary image output use an isolated BT.2020-to-BT.709 SDR tone map; TIFF and raw snapshots retain RGBA16 samples. |
| Basic shader playback | Complete | Loads Vulkan fragment or compute shaders compiled to .spv; --fragment and --compute validate the SPIR-V stage. |
| Shader libraries | Complete | Prefers library.json and falls back to index.txt; supports nested paths and object or string entries. Offline --build mode incrementally compiles source manifests containing .frag, .comp, or .spv entries into a validated runtime library. |
| Shader selection | Complete | Supports selection by index or filename, keyboard switching, and live selection from the ACMX Qt interface through --interface-shm. |
| Shader crossfades | Implemented | Shader, playlist, multipass, and bypass changes crossfade from a snapshot of the preceding rendered result. All 35 ACMX2 transition styles are bundled as Vulkan SPIR-V shaders, with selectable duration/style and optional random transition selection during autopilot. |
| Multipass and playlists | Implemented | Includes named playlist nodes, mixed fragment/compute chains, sequential autopilot, and random autopilot. Shader stages are detected from SPIR-V entry points rather than filenames. |
| Frame history/texture cache | Implemented | Uses one shared Vulkan sampler2DArray ring buffer with configurable size and write delay. Fragment and compute post-processing passes can sample it at binding 2, and SPIR-V reflection enables it automatically for history-capable libraries. CUDA-filter builds place the post-filter image in history through direct CUDA/Vulkan layered-image interop. |
| Custom library uniforms | Implemented | Up to 64 validated floats from library.json, with repeatable --uniform name=value overrides and live updates from the ACMX Qt interface. |
| Video recording | Implemented | MXWrite supports software or hardware encoders, encoder options, no-drop mode, duration and size limits, optional audio copying or audio-free --mute-output recording, source-timeline PTS, audio-clock synchronization, and pipelined Vulkan readback. |
| Snapshot and PNG output | Implemented | Supports full PNG sequences, periodic generated frames, ACMX2-compatible one-shot Z PNG snapshots, optional lossless TIFF snapshots on 4, optional lossless WebP snapshots on 5, and headerless raw snapshots on 6. HDR PNG/WebP output is SDR tone-mapped; HDR TIFF/raw output retains normalized RGBA16 samples. |
| Text overlays and watermark | Implemented | Provides an ACMX2-compatible preview HUD with shader, multipass chain, decoded video position/source duration, processing elapsed time, measured FPS, audio track, CUDA filter, and autopilot status. The native title bar identifies graphics, video, or capture mode; distinguishes preview from recording; and reports recording time, frame count, and current encoded file size. Slow video processing advances the video timer by decoded frames rather than wall time. --disable-counter, a configured watermark, or F9 hides the HUD; F9 can show it again when a watermark selected the hidden default. When both are visible, the HUD starts below the watermark. The HUD and title are excluded from readback, snapshots, and recordings; explicit filter/watermark overlays remain included in output. |
| Rotation and final-output flip | Implemented | Applies input rotation and optional final display/recording flip. |
| Runtime playback controls | Implemented | Supports video pause, rendering freeze, shader locking, wall-clock or audio-reactive shader time, time stepping/speed, and fullscreen switching. |
| Input validation | Implemented | Centralized allowlists validate CLI and environment strings, paths, URLs, identifiers, encoder fields, manifests, playlists, MIDI maps, device names, and bounded live MIDI messages before use. Configuration files, lines, entry counts, numeric ranges, image dimensions, and SPIR-V binaries have explicit limits. |
| ACMX2 GLSL compatibility | Partial | Existing GLSL effects must be translated to the MXVK Vulkan descriptor ABI and compiled to SPIR-V. |
| Audio-reactive shader data | Implemented | RtAudio capture, an FFmpeg-decoded media file, an M3U/M3U8 playlist, or --use-source-audio with real-time source-FPS video playback can drive amplitude, frequency, peak, RMS, smoothed amplitude, low/mid/high bands, a current-frame FFT, configurable FFT history, audio-reactive shader time, and optional delta/sensitivity scaling. Source-video analysis follows the media clock even when late video frames are skipped. Live and file audio support configurable shader warmup, adjustable-gain output pass-through, and AAC muxing; live input can also be recorded independently as PCM16 WAV with adjustable gain, while file audio supports repeat and stop-at-EOF behavior. |
| MIDI controls | Partial | Optional RtMidi support handles input enumeration, a bounded callback queue, live monitoring, ACMX2 MIDI Map .midi_cfg files, Slider 1–4 custom uniforms, ACMXVK-equivalent playback actions, PNG/TIFF/WebP/raw snapshots, HUD and watermark toggling, audio-time/delta/FFT sensitivity actions, and direct three-axis 3D model rotation/scale controls. Paired knobs use ACMX2's centered, velocity-sensitive repeat behavior. |
| CUDA filters | Partial | Optional acidcam-gpu integration accepts filter chains and temporal-buffer sizes, keeps NVDEC video frames, camera RGBA, and input rotation resident on the GPU through filtering and Vulkan upload/history, and supports ACMX2-compatible Left/Right selection from the keyboard or MIDI maps. |
| DNN effects | Implemented | Optional -DWITH_OPENCV_DNN=ON builds support ACMX2-compatible DexiNed edge detection, PP-HumanSeg foreground isolation/background composition, and generic YAML-configured image-to-image ONNX processing before the Vulkan shader chain. |
| Deep Dream | Implemented | Optional -DWITH_DEEP_DREAM=ON builds use CUDA LibTorch with exported VGG16 or Inception V3 feature models. Pixel-gradient ascent can process camera, video, or image frames before the existing fragment/compute shader chain, with temporal feedback, original independent-frame modes, random animation, feature-layer/channel targeting, progressive octaves, jitter, gradient smoothing, FP16, optional acidcam-gpu-first ordering, interface live control, and a CUDA-resident capture-to-Vulkan path. |
| 3D model pipeline | Initial support | --enable-3d maps live video, camera, or still-image input onto MXVK's OBJ/MXMOD model renderer. Compatible fragments execute directly on model UVs; compute, history/spectrum, multipass, and playlist chains use a pre-model offscreen target whose result becomes the model texture. The camera starts at the normalized model center as a 120-degree skybox view with automatic rotation disabled. OBJ, MXMOD, and compressed MXMOD files are supported, with a bundled textured cube as the default. Mouse look/movement, automatic rotation, scale/speed controls, ACMX2-compatible camera oscillation and three-axis wave deformation, 2D/3D switching, recording, snapshots, and compatible MIDI-map actions are implemented. |
| Qt interface integration | Initial integration | The ACMX Qt launcher selects ACMX2 or ACMXVK libraries, builds ACMXVK source manifests into an incremental hidden SPIR-V library, launches that output, and streams renderer output into its log. Live shader selection and source recompilation, custom uniforms, multipass chains, Repeat, Normalized Time, overlays, CUDA filter chains, Deep Dream configuration, and file-audio replacement are integrated into the ACMXVK workflow. |
acmx.cpp is the compact process entry point. main_window.hpp and main_window.cpp now own the render-window class boundary. Its application implementation is grouped under app/ by responsibility: command-line options, shader-library building, media helpers, lifecycle/state, audio and MIDI, shaders and interface control, overlays, input/output, and rendering. See app/README.md for the file map. Options, shader-library building, playlist parsing, resource discovery, output-path generation, media utilities, asynchronous camera capture, interface shared-memory access, and background snapshot encoding use normal .hpp/.cpp pairs. The class declaration and complete runtime state live in main_window.hpp. Lifecycle, event, frame-processing, audio, MIDI, custom-control, media-clock, shader, interface IPC, playlist, resource, HUD, watermark, DNN-overlay, input, output, snapshot, frame-readback, 3D, crossfade, history, capture, and frame-upload methods are standard out-of-class definitions in main_window.cpp. The former ordered .ipp implementation sections have been fully retired.
Ensure the selected Vulkan SDK's bin directory is on PATH so CMake can find tools such as glslc. If the SDK is installed outside the platform's standard search paths, set VULKAN_SDK or add its prefix to CMAKE_PREFIX_PATH using the setup instructions supplied with that SDK.
From the acidcam-gpu repository root:
To install or uninstall using the selected CMake prefix:
Audio and MIDI support are optional and remain disabled when their CMake options are omitted.
Deep Dream performs gradient ascent on the input pixels instead of training or changing the neural network. A selected feature activation becomes the objective, LibTorch computes its gradient with respect to the current image, and ACMXVK moves the pixels in the direction that strengthens that activation. The dreamed result then enters the normal Vulkan fragment/compute shader chain. The model weights remain frozen.
This feature currently requires an NVIDIA CUDA GPU. It is independent of the acidcam-gpu filter option: -DWITH_DEEP_DREAM=ON -DWITH_CUDA=OFF enables Deep Dream without linking libacidcam-gpu, while adding -DWITH_CUDA=ON also enables the optional CUDA filter chain and --gpu-filter-before-dream.
The complete Doxygen guide is available on the Deep Dream related page and covers architecture, every runtime control, the Qt interface, model export, performance, error recovery, and troubleshooting.
Install a CUDA toolkit, cuDNN, a CUDA-enabled OpenCV containing cudaarithm/cudawarping, and CUDA-enabled LibTorch. For an Arch Linux system using the CUDA PyTorch packages:
MXVK must be installed with OpenCV capture support. Build it with CUDA enabled to obtain the direct CUDA/Vulkan path:
Then configure ACMXVK. This example uses the CUDA LibTorch distribution in /opt/libtorch; omit Torch_DIR when the system PyTorch package already installs TorchConfig.cmake in CMake's default search path:
Change TORCH_CUDA_ARCH_LIST to the compute capability of the target GPU or omit it to let PyTorch autodetect the device. Use -DWITH_CUDA=ON in the ACMXVK command only when the separately installed acidcam-gpu package is also required. A successful configure prints Deep Dream: ENABLED; a successful probe reports CPU autograd, the CUDA device count, CUDA autograd, cuDNN, and whether direct CUDA/Vulkan interop was compiled in.
The probe always exercises CPU autograd. With --dream-model, it additionally loads the model on the selected GPU, validates all metadata and feature outputs, and runs real forward/backward gradient-ascent and feedback steps:
Use --cuda-device N to select a device. A container reporting zero CUDA devices must be launched with NVIDIA device access. If an installed executable cannot locate LibTorch shared libraries, add /opt/libtorch/lib to the system dynamic-loader configuration or to LD_LIBRARY_PATH for that invocation.
Deep Dream is not built by the current Pcons path; use CMake. CUDA and LibTorch make this feature unavailable on macOS. Linux with an NVIDIA GPU is the primary tested configuration.
The exporter creates a frozen TorchScript feature model, embeds the authoritative ACMXVK metadata, and writes a matching .pt.json sidecar used by the interface. Create the standard 13 VGG16 ReLU feature layers with:
The exporter also supports pretrained Inception V3 feature blocks. Its default layer is Mixed_6c, a useful middle-to-late activation for complex Deep Dream structures:
The Inception export exposes Mixed_5b through Mixed_7c and writes the same embedded metadata and .pt.json interface sidecar as the VGG16 export. Select one at runtime with, for example, --dream-layer Mixed_6c.
The default weights may be downloaded into PyTorch's user cache on the first run. --weights none avoids a download for structural testing but does not produce a useful dream model. Use --layers to export a subset and --default-layer to choose the initial target. The exporter writes a readable .pt.json sidecar and embeds the authoritative typed metadata in the TorchScript module. Only load TorchScript models from sources you trust.
| Option | Purpose | Range/default |
|---|---|---|
| --dream-model file.pt | Select an exported TorchScript feature model. | Required |
| --dream-layer name | Choose a named or numbered feature endpoint. | Model default |
| --dream-iterations N | Gradient-ascent steps at each octave per frame. | 1–100; 1 |
| --dream-strength N | Pixel-gradient step size. | >0–10; 0.05 |
| --dream-feedback N | Blend the previous dreamed frame into the next input. | 0–0.99; 0.9 |
| --dream-zoom N | Scale the feedback image per frame. | 0.9–1.1; 1.01 |
| --dream-rotation N | Rotate feedback per frame in degrees. | -5–5; 0.1 |
| --dream-size N | Bound the longest neural working dimension; 0 uses native size. | 0 or 64–4096; 512 |
| --dream-fp16 | Use half-precision model/input tensors. | Off |
| --dream-channel N\|all | Target one zero-based feature channel or all channels. | all/-1 |
| --dream-octaves N | Process progressively larger scales. | 1–8; 1 |
| --dream-octave-scale N | Size ratio between adjacent octaves. | 1.1–3; 1.4 |
| --dream-jitter N | Deterministically shift each ascent input. | 0–64 pixels; 0 |
| --dream-smoothing N | Smooth the input gradient spatially. | 0–16 pixels; 0 |
| --random-dream seconds | Randomize safe animation controls on the media clock. | Positive interval |
| --gpu-filter-before-dream | Run an enabled acidcam-gpu chain before Deep Dream. | Off |
| --dream-headless | Traditional independent-frame dreaming in a headless recording. | Off |
| --deep-orig | Independent-frame dreaming with a preview window. | Off |
--dream-channel all and --dream-channel -1 are equivalent. All-channel mode is the recommended starting point for Inception because an individual ReLU channel can be inactive for a particular image. A finite zero-gradient frame is handled as a no-op iteration and does not disable Deep Dream.
The following sections describe the implementation and tuning behavior in more detail.
Inspect the model and select a layer by name or zero-based output index:
The loader bounds the model file size, validates the ACMXVK format version, normalization and layer metadata, loads directly onto the selected CUDA device, freezes model parameters, runs a small forward pass, and verifies every feature output before accepting the model.
Increment 3 adds the reusable RGBA8 gradient-ascent processor. It converts input pixels to normalized NCHW tensors, optimizes the pixels against the selected activation while model weights remain frozen, normalizes and validates each gradient, clamps values to the model's valid input range, and preserves the original alpha channel. Model inspection now runs one real gradient-ascent step on a synthetic image and reports its loss, mean gradient, and mean pixel change. This verifies the complete forward/backward pixel path. Only load TorchScript models from sources you trust.
Increment 4 integrates that processor into camera, video, and still-image input. Deep Dream runs before acidcam-gpu filters and the existing Vulkan fragment/compute multipass chain, so existing shaders, history, playlists, crossfades, MIDI controls, audio reactivity, recording, and 3D texture mapping remain available. One iteration per source frame is the real-time-oriented default:
--dream-iterations accepts 1–100 and --dream-strength accepts values greater than zero through 10. Higher values can be substantially slower or visually unstable. CUDA-enabled MXVK builds keep compatible capture and NVDEC frames resident through LibTorch and MXVK's imported Vulkan texture. Other configurations use the host-compatible path.
Increment 5 adds a temporal feedback loop. Each new source frame is blended with the previous dreamed result after a small centered zoom and rotation, then the combined image receives the next gradient-ascent step. This allows the default single iteration to accumulate structure over time while fresh camera or video detail continuously anchors the image. The controls are:
Feedback accepts 0–0.99; zero restores independent per-frame processing. Zoom accepts 0.9–1.1, and rotation accepts -5 through 5 degrees per source frame. The transformed feedback uses reflected borders to avoid introducing black edges. --check-deep-dream exercises a second synthetic frame and reports that the feedback path is ready. Existing Vulkan shaders still process the completed dreamed frame.
Increment 6 bounds neural processing independently from output resolution. By default, an input larger than 512 pixels on its longest axis is reduced with an area filter for gradient ascent, then restored to the exact source dimensions before Vulkan shaders and recording. This makes 1080p and 4K sources practical without changing shader resolution. Use --dream-size 0 for native-resolution processing or choose 64–4096 for a different maximum.
--dream-fp16 converts the model and working tensors to half precision on CUDA. This can improve throughput and reduce VRAM use on NVIDIA GPUs with efficient FP16 support, including the RTX 2070. Loss and gradient normalization remain in FP32 for numerical stability. FP32 remains the default compatibility mode.
Increment 7 adds individual feature-channel targeting. The default --dream-channel all maximizes the complete selected activation as before. Choose a zero-based channel to isolate a narrower learned visual feature:
The selected layer's channel count is printed by --check-deep-dream, and an out-of-range channel is rejected before media processing begins. VGG16 was trained for ImageNet classification and does not label a particular channel as "faces"; channel targeting provides a controlled way to explore which channels produce face-like, eye-like, architectural, or textural structures. Earlier layers tend toward edges and textures, while later layers produce larger and more semantic forms.
Increment 8 adds progressive multi-octave processing. One octave remains the default and follows the previous real-time path. Increasing the count starts at a smaller image, performs the configured gradient iterations, then advances toward the full neural working size. At each transition ACMXVK restores the source detail that would otherwise be lost during upscaling:
--dream-octaves accepts 1–8. --dream-octave-scale accepts 1.1–3.0 and controls the size ratio between adjacent scales. Very small duplicate scales are collapsed when the model's minimum input size is reached. Each octave runs the selected number of gradient iterations, so three octaves are approximately three times as expensive as one. Multi-octave processing still completes before the existing Vulkan fragment/compute shader chain.
Increment 9 adds deterministic spatial jitter to gradient ascent. Jitter rolls the neural input before each forward pass and lets autograd map the gradient back to the unshifted image. This reduces persistent border bias and fixed-grid patterns, especially when several iterations or octaves are used:
The value is the maximum horizontal and vertical displacement in pixels and accepts 0–64; zero preserves the previous behavior. Offsets are derived from the source-frame, octave, and iteration sequence rather than the system clock, so identical inputs and settings are independent of processing speed. Jitter does not add another model pass and can be combined with channel targeting, FP16, temporal feedback, and the existing Vulkan shader chain.
Increment 10 adds optional CUDA gradient smoothing. It applies a centered box filter to the input gradient before its magnitude is measured and normalized, encouraging wider coherent structures instead of isolated pixel noise:
The value is a pixel radius from 0–16; zero preserves the previous gradient path. Radius 1 or 2 is a useful starting point, while large radii produce broader and softer structures and add more GPU work. Smoothing operates on the existing gradient and does not introduce another VGG forward/backward pass. It can be combined with jitter, octaves, individual channels, feedback, and FP16.
For continuously changing visuals, --random-dream <seconds> randomizes the strength, temporal feedback, zoom, rotation, octave count, and octave scale at the requested interval. --random_dream is accepted as an alias. Video files use their decoded media timeline, so headless and offline output is independent of processing speed; cameras use elapsed capture time. For example:
The randomizer deliberately leaves the model, feature layer, iteration count, channel, working resolution, jitter, and smoothing unchanged. Octaves range from 1 through 8; higher random values require proportionally more neural processing.
--dream-headless provides a traditional offline frame-by-frame Deep Dream video render. It is restricted to headless video recording and requires --headless, --input, --output, and --dream-model. The mode disables temporal feedback, forced zoom, and feedback rotation, then enables no-drop encoding so every decoded source frame is processed independently:
This mode cannot be combined with --random-dream. It does not affect the windowed Deep Dream path or ordinary headless processing.
Use --deep-orig for the same independent-frame processing with the preview window visible. It still requires video input, recorded output, and a Deep Dream model, and it retains no-drop encoding:
--deep-orig and --dream-headless are mutually exclusive because they select windowed and headless forms of the same processing mode.
Increment 11 adds the CUDA/Vulkan interop path for Deep Dream. When MXVK was built with CUDA support, compatible camera and video frames remain on the GPU across capture or NVDEC, LibTorch preprocessing, optional input rotation, optional acidcam-gpu filters, MXVK texture/history upload, and Vulkan shader processing. LibTorch wraps the pitched CUDA RGBA allocation without copying it to the CPU, and the dreamed RGBA result is copied device-to-device into MXVK's Vulkan-imported image.
The handoff waits for the producing OpenCV CUDA stream before LibTorch reads the frame and completes LibTorch work before publishing the result. Temporal feedback, octaves, channel targeting, jitter, smoothing, alpha preservation, history, 3D textures, recording, and fragment/compute chains remain supported. HDR compatibility conversion, active OpenCV DNN effects, asynchronous maximize-FPS camera capture, or an MXVK installation without CUDA use the established host path. --check-deep-dream reports whether CUDA/Vulkan interop was compiled into the executable.
Use --gpu-filter-before-dream with --gpu-filter to run the selected acidcam-gpu chain on the source CUDA frame before LibTorch processes it. Without this option, the established order remains Deep Dream followed by acidcam-gpu. Both orders remain GPU-resident and publish their final combined result to the Vulkan texture, model texture, and history cache. The initial implementation is for SDR camera and video input and cannot be combined with --maximize-fps, a still graphic, HDR input, or an OpenCV DNN input effect.
When the ACMX Qt interface detects a Deep Dream-enabled ACMXVK executable, its Session > Deep Dream Settings dialog exposes the model, feature layer, gradient-ascent, temporal-feedback, detail, and performance controls. The settings persist between sessions and apply to Run Selected, Run All, and Edit Command. While ACMXVK is running, Apply publishes the complete Deep Dream configuration through the versioned interface-control block. Scalar controls take effect between frames; changing the model, layer, or precision safely rebuilds the LibTorch model before replacing the active instance. Invalid live settings are rejected without disturbing the current effect. The optional acidcam-gpu-first order is available when a CUDA filter chain is enabled.
The interface reads the exporter-generated <model>.pt.json sidecar when a model is selected. Valid ACMXVK metadata replaces the generic VGG16 layer choices with the exact layer list embedded by the exporter, reports the architecture and minimum input size, and preserves a previously selected layer when the new model supports it. Missing sidecars retain editable manual layer entry; malformed, oversized, duplicate, or unsupported metadata is reported without loading it into the interface.
pcons-build.py is a native Pcons alternative to the CMake build. Build and install MXVK first, then pass its installation prefix through PREFIX:
PREFIX is only the MXVK dependency prefix; the two PCONS_*_PREFIX values choose ACMXVK's staged and final install location. The Pcons build bundles the repository MXWrite source and installs the executable, SPIR-V shaders, crossfades, playlists, MIDI examples, models, and overlay font under that prefix. Its CUDA filter path is intentionally unsupported for now; use the existing CMake build with -DWITH_CUDA=ON for a matching CUDA-enabled MXVK and acidcam-gpu installation.
Build with OpenCV's DNN module to enable the first ported ACMX2 DNN effect:
WITH_OPENCV_DNN is independent of ACMXVK's WITH_CUDA acidcam-gpu filter option. The configuration above does not find or link libacidcam-gpu.so. OpenCV DNN can still select its own CUDA backend when the installed OpenCV was built with that backend; otherwise it uses OpenCV's CPU backend.
Pass a DexiNed-compatible ONNX model with --edge. The generated grayscale edge image becomes the input to the selected fragment/compute pipeline:
The first frame can take longer because ACMXVK warms and benchmarks CPU and available CUDA DNN targets. Later frames retain the faster backend. If an inference error occurs, edge processing is disabled once and the original input continues through the Vulkan pipeline.
Pass a PP-HumanSeg-compatible ONNX model with --human to isolate the detected person before the shader chain. The default mode applies shaders to the isolated foreground on black:
Add --background to process only the background and composite the original person over the final Vulkan result. This composition is currently supported in 2D mode. --black and --white tune the mask's alpha thresholds; both accept values from 0.0 to 1.0, default to 0.35 and 0.75, and require the black point to remain lower than the white point:
Human segmentation and DexiNed can be combined. ACMXVK segments first and then applies edge detection, matching ACMX2's input-effect order.
Use --onnx <config.yaml> for ACMX2-compatible generic image-to-image models. The ONNX file named by model.path is resolved relative to the configuration file. Generic processing runs after human segmentation and edge detection and before rotation, acidcam-gpu filters, history, 3D texture mapping, and Vulkan shaders:
The accepted YAML schema is:
One-channel outputs become grayscale BGR frames. Three-or-more-channel NCHW outputs use their first three RGB planes. Models with multiple output tensors retain ACMX2's fused-plane handling. Results are normalized to 8-bit color and resized back to the source frame. Dynamic configurations at 256 pixels or less enable edge-preserving bilateral smoothing by default unless the postprocessing block overrides it. Configuration size, supported keys, strings, tensor names, dimensions, numeric values, and the referenced ONNX file are validated before inference.
Increment 5H added the MXVK spectrum-history descriptor and UBO suffix, so that matching MXVK version must be installed before compiling ACMXVK with -DAUDIO=ON. Increment 5I changes only ACMXVK and does not require another MXVK reinstall. Increments 5J through 5R, 6A through 6C, and 7A through 7B also change only ACMXVK. Increment 7A requires the existing MXVK and acidcam-gpu installations to have CUDA support. Increment 7C adds MXVK's layered CUDA history upload API, so MXVK must be rebuilt and reinstalled before building ACMXVK 7C with -DWITH_CUDA=ON. Increment 7D adds a CUDA GpuMat input overload to acidcam-gpu's temporal frame buffer, so acidcam-gpu must be rebuilt and reinstalled before building ACMXVK 7D with -DWITH_CUDA=ON; MXVK does not need another reinstall for 7D. Increment 7E changes only ACMXVK, so neither MXVK nor acidcam-gpu needs another reinstall. Increment 7F also changes only ACMXVK and uses MXVK's existing FFmpeg-capture API when that feature is present. Increment 7G changes only ACMXVK as well. Increment 7H also changes only ACMXVK and does not require either dependency to be reinstalled. Increment 7I adds MXVK's FFmpeg capture device-selection API, so MXVK must be rebuilt and reinstalled before ACMXVK is rebuilt; acidcam-gpu does not need another reinstall. Increment 7J adds MXVK's in-place FFmpeg seek API, so MXVK must again be rebuilt and reinstalled before rebuilding ACMXVK; acidcam-gpu remains unchanged. Increment 7K changes MXVK's internal NVDEC surface synchronization, so MXVK must be rebuilt and reinstalled once more; acidcam-gpu remains unchanged. Increment 7L changes only ACMXVK and does not require either dependency to be rebuilt or reinstalled. Increments 7M through 7P also change only ACMXVK. Increment 7Q adds MXVK's efficient capture-frame skip APIs and raises MXVK to 0.26.0, so MXVK must be rebuilt and reinstalled before rebuilding ACMXVK; acidcam-gpu remains unchanged. Increment 7R changes only ACMXVK and does not require MXVK or acidcam-gpu to be rebuilt. Increment 7S adds MXVK's preview-only text queue and raises MXVK to 0.27.0, so MXVK must be rebuilt and reinstalled before ACMXVK; acidcam-gpu remains unchanged. Increment 7T changes only ACMXVK and does not require MXVK or acidcam-gpu to be rebuilt. Increment 7U also changes only ACMXVK. Increment 7V pipelines rendered-frame readback and raises MXVK to 0.28.0, so MXVK must be rebuilt and reinstalled before ACMXVK; acidcam-gpu remains unchanged. Increment 7W fixes discrete-GPU readback memory selection and moves hardware-encoder upload work off the render thread. It raises MXVK to 0.28.1, so MXVK must be rebuilt and reinstalled before ACMXVK. Increment 7X adds compute post-processing pipelines and raises MXVK to 0.29.0, so MXVK must again be rebuilt and reinstalled before ACMXVK; acidcam-gpu is unchanged. Increment 7Y changes only ACMXVK. It selects the video, negotiated camera, or still-image dimensions when --resolution is omitted and applies input rotation before choosing the automatic width and height. Neither MXVK nor acidcam-gpu needs to be rebuilt or reinstalled. Increment 7Z changes only ACMXVK. It prevents the desktop compositor from maximizing an oversized source window to a different aspect ratio by fitting the preview within the usable display and locking its aspect. Recording and generated output retain the automatic source dimensions. Neither dependency needs to be rebuilt or reinstalled. Increment 8A adds MXVK 0.30.0's fixed native render extent. Shader passes, watermark/output text, snapshots, and recording readback remain at the source dimensions while a separate aspect-preserving copy is fitted into the preview swapchain. Output text and preview-only HUD fonts are sized independently. MXVK must be rebuilt and reinstalled; acidcam-gpu remains unchanged. Increment 8B changes only ACMXVK. All overlay and HUD font sizing now follows the preview window height at a smaller 1/60 scale instead of using the source image or video height. MXVK does not need another rebuild or reinstall. Increment 8C adds MXVK 0.31.1's non-owning shared history descriptor for fragment and compute post-processing passes. ACMXVK reflects binding 2 from SPIR-V, enables one input-frame history ring automatically, and supplies its live head/layer metadata to every pass. Converted history shaders are now included in library.json. Bindings 3 and 4 are reflected as well, allowing spectrum shaders to receive zero-initialized safe descriptors even without explicit audio-buffer flags. MXVK must be rebuilt and reinstalled; acidcam-gpu remains unchanged. Increment 8D changes only ACMXVK. --use-source-audio selects the input video's embedded audio track for shader reactivity during --use-source-fps playback. Optional pass-through uses the selected RtAudio output as the A/V master clock; silent analysis follows the video media clock directly. Neither MXVK nor acidcam-gpu needs to be rebuilt or reinstalled. Increment 8E also changes only ACMXVK. A video without a decodable audio track now continues with zero-valued audio-reactive inputs instead of aborting; requested pass-through is disabled with a warning. An explicitly selected --audio-file remains a hard error when it cannot be decoded. Increment 8F changes only ACMXVK. Its native title bar now follows ACMX2's graphics/video/capture mode format, explicitly reports preview or recording state, and refreshes recording time, frame count, and MXWrite's encoded file size twice per second. PNG sequences report recording state without a single-file size. Increment 8G changes only ACMXVK. Shader bypass now retains a compiled identity post-processing pass instead of removing MXVK's source-sized offscreen chain. This keeps its Vulkan image barriers and presentation target valid when Space disables the selected shader, fixing the MoltenVK crash in video and camera modes. The identity pass is also used when no visual shader is active. Increment 8H changes only ACMXVK. K now toggles ACMX2-compatible shader locking. While locked, manual shader and playlist-node selection and both autopilot modes retain the current pipeline; autopilot resumes its existing countdown after unlocking. The runtime HUD marks the shader as locked, and ACMX2 MIDI-map action code 75 can toggle the same state. Increment 8I changes only ACMXVK. It uses MXVK's VKAbstractModel and MXModel loader to render OBJ, MXMOD, and compressed MXMOD models. Each input frame replaces texture slot zero across the complete model draw, and the result enters the same ordered fragment/compute pipeline used by 2D input. The bundled cube.obj is selected when --model is omitted. MXVK 0.31.1 already provides the required public model API, so it does not need to be rebuilt or reinstalled for this increment. Increment 8J changes the 3D render order and camera defaults to match ACMX2. The active compatible fragment shader is now the model fragment shader, so the effect evaluates against the source texture in model UV space. The camera starts at the normalized object center with yaw 270, pitch 0, distance 0, a 120-degree field of view, and automatic view rotation disabled. MXVK's model fragment UBO now includes ACMXVK's 64 custom-uniform slots and audio-band fields, so MXVK must be rebuilt and reinstalled for this increment. Increment 8K adds MXVK 0.33.0's post-processing texture-consumer stage. Compute shaders, history/spectrum shaders, --shader-pass, and playlist chains now finish in an offscreen image that is sampled by the model. They no longer appear as a flat screen-space layer over the completed 3D object. Increment 8L changes only ACMXVK. Its 3D camera now accepts ACMX2's continuous W/A/S/D look controls, plus/minus movement along the view direction, and 1/2 movement-sensitivity controls. MXVK and acidcam-gpu do not need to be rebuilt or reinstalled. Increment 8M ports ACMX2's complete 35-effect crossfade set to Vulkan. Shader, playlist-node, multipass, and bypass changes snapshot the preceding rendered result and blend it into the new result for the requested wall-clock duration. The selected transition is shown in the HUD; brackets select its style, and N optionally randomizes it for autopilot changes. MXVK 0.33.1 changes the synchronous snapshot path to read MXVK's owned source-sized offscreen image, avoiding an invalid post-presentation swapchain transition. MXVK must be rebuilt and reinstalled before rebuilding ACMXVK; acidcam-gpu is unchanged. Increment 8N changes only ACMXVK. C now toggles ACMX2's three-axis model wave, with the same 0.005-per-frame amplitude ramp, 0.0-to-0.5 reflection, frequency 2.0, phase offsets, and optional audio-reactive phase advancement. The deformation runs in the model vertex shader and analytically transforms normals through the sequential X/Y/Z displacement, avoiding per-frame CPU mesh uploads. The HUD reports [wave], and ACMX2 MIDI-map action 67 toggles the same state. MXVK and acidcam-gpu do not need another rebuild or reinstall. Increment 8O also changes only ACMXVK. O toggles ACMX2's historically named "scale oscillation," which advances by 0.016 per rendered 3D frame and replaces the active camera offset with 0.3 * sin(phase). Manual zoom speed, zoom, mouse-wheel movement, and model-scale changes are suspended while oscillation is active; disabling it restores the stored manual camera distance. The HUD reports [oscillate], and ACMX2 MIDI-map action 79 toggles the same state. Increment 8P separates model scaling from crossfade selection. Plain [ and ] now change only the crossfade style; Shift+- and Shift++ decrease or increase model scale, while unmodified minus/plus retain camera zoom. MIDI-map actions 91 and 93 are routed directly to the shifted scale controls and no longer change the selected crossfade. MXVK and acidcam-gpu are unchanged. Increment 8Q adds optional lossless WebP snapshots. Configure ACMXVK with -DWEBP=ON to link libwebp, then press 5 to save the final processed frame as WebP through the same bounded background queue and --prefix destination used by Z PNG snapshots. WebP remains disabled by default, and this increment changes only ACMXVK. Increment 8R adds --mute-output. Audio input, source/file decoding, shader reactivity, pass-through monitoring, audio-clock synchronization, and optional standalone --record-audio WAV output remain active, while copying or muxing audio into the encoded video is suppressed. This increment changes only ACMXVK. Increment 8S adds optional lossless TIFF snapshots. Configure ACMXVK with -DTIFF=ON to link libtiff, then press 4 to save the final processed frame as an LZW-compressed 8-bit RGBA TIFF through the same background queue and --prefix destination used by PNG and WebP snapshots. TIFF remains disabled by default, and this increment changes only ACMXVK. Increment 8T corrects live camera recording timestamps. Camera frames now use the active audio clock when available and otherwise use elapsed capture time to derive explicit output PTS. When 4K rendering or encoding delivers fewer frames than the nominal camera rate, the resulting video retains its real capture duration instead of playing fast. This increment changes only ACMXVK. Increment 8U completes ACMX2's core snapshot-key set. Press 6 to save the final processed frame as tightly packed, top-to-bottom, 8-bit RGBA pixels in a headerless .raw file. Raw capture uses the same rendered-resolution naming, --prefix destination, bounded background queue, and shutdown draining as the encoded snapshot formats. It requires no optional build dependency and changes only ACMXVK. ACMX2 MIDI-map action code 54 triggers the same capture. Increment 8V begins the DNN port with ACMX2-compatible DexiNed edge detection. Configure with -DWITH_OPENCV_DNN=ON, then select an ONNX edge model with --edge. Each source frame is converted to the edge map before rotation, acidcam-gpu filtering, history insertion, 3D texture mapping, and the Vulkan shader chain. ACMXVK benchmarks the available OpenCV CPU and CUDA DNN backends on the first frame and retains the faster backend. This increment changes only ACMXVK; MXVK and acidcam-gpu do not need to be reinstalled. DNN-only builds use -DWITH_OPENCV_DNN=ON -DWITH_CUDA=OFF and have no libacidcam-gpu.so dependency. Increment 8W adds PP-HumanSeg foreground isolation and ACMX2-compatible background-only shader composition. --human selects the model, --background preserves the person above the completed 2D shader/crossfade chain, and --black/--white tune the hardened alpha mask. DNN remains independent of libacidcam-gpu.so; this increment changes only ACMXVK and requires no MXVK or acidcam-gpu reinstall. Increment 8X completes the core ACMX2 DNN input-effect port with --onnx and the existing generic model YAML schema. It supports fixed and dynamic input shapes, optional tensor names, scale/mean/channel preprocessing, output-shape conversion, and bilateral smoothing. Generic inference uses the same automatic OpenCV CPU/CUDA backend selection and remains independent of libacidcam-gpu.so. This increment changes only ACMXVK. Increment 8Y makes video shader animation deterministic with respect to the decoded media timeline. Shader delta, shader time advancement, compatibility iTime, reported shader FPS, and the exposed frame index now come from source video frames rather than processing wall time. A video therefore produces the same shader animation whether it is processed faster or slower than real time; --normalized remains available for deterministic camera and still-image rendering. This increment changes only ACMXVK. Increment 8Z extends decoded-video timing to crossfades and built-in 3D model animation. Crossfade duration, automatic model rotation, vertex wave motion, and scale oscillation now advance from decoded source frames in video mode, so their appearance does not depend on processing speed. Pausing a video pauses these animations, skipped source frames advance them by the corresponding source duration, and a repeated video resets their autonomous animation state. Camera and still-image modes retain live render-clock behavior. This increment changes only ACMXVK. Increment 9A hides the preview-only runtime HUD by default whenever --use-watermark configures an initially enabled watermark. F9 can still show the HUD, and when both overlays are visible the HUD starts one line below the watermark so the text does not overlap. The watermark remains included in snapshots and encoded output, while the HUD remains preview-only. This increment changes only ACMXVK. Increment 9B moves the playlist-autopilot countdown onto decoded source-frame progress for video input. Repeated render loops and video pauses no longer consume the countdown, while frames skipped during real-time catch-up still advance it. Camera and still-image modes retain their rendered-frame countdown, and shader locking continues to pause autopilot without losing its remaining interval. This increment changes only ACMXVK. Increment 9C adds the active playlist node to the preview-only runtime HUD, immediately above its multipass shader chain. Playlist load, runtime enable, manual node selection, and autopilot changes now all print the selected node's index, name, and pass count to the terminal. This increment changes only ACMXVK. Increment 9D makes multipass chains readable in the runtime HUD. Playlist and standalone multipass shaders are shown as individually numbered pass lines instead of one clipped comma-separated line. The HUD displays up to eight passes and summarizes any additional entries, keeping malformed or unusually large playlists from filling the entire preview. Encoded filter text retains its compact single-line form. This increment changes only ACMXVK. Increment 9E labels the selected library shader as Post-shader whenever it follows an active playlist or standalone multipass chain. The runtime HUD, encoded --display-filter overlay, startup message, and Shift+Up/Down terminal log now use the same terminology, making the final pass distinct from the node's numbered shader passes. Single-shader mode retains the Shader label. This increment changes only ACMXVK. Increment 9F corrects and expands ACMX2 MIDI-map compatibility. Action code 78 now dispatches N to toggle random autopilot XFade selection instead of incorrectly duplicating J. Existing maps can also use 298 for the runtime HUD, 52/53 for optional TIFF/WebP snapshots, 70 for fullscreen, and 73/85 for direct I/U shader-time stepping. This increment changes only ACMXVK. Increment 9G adds an ACMX2/ACMXVK target selector to the Qt MIDI-map editor. Both profiles write the existing .midi_cfg format, but each shows only its supported actions with target-correct descriptions. Compatible captured mappings are retained when switching profiles, while codes whose meanings differ between the applications are deliberately kept separate. This increment changes the MIDI-map editor and ACMXVK documentation; MXVK and acidcam-gpu are unchanged. Increment 9H ports ACMX2's remaining virtual 3D MIDI controls. Paired knob codes 506:507, 508:509, and 512:513 rotate the model on its X, Y, and Z axes; 514:515 changes model scale; and button codes 510/511 adjust the manual rotation sensitivity. Rotation knobs retain ACMX2's delta-sensitive direction handling and velocity-sensitive repeat rate. The ACMXVK profile in the Qt MIDI-map editor exposes all of these mappings. This increment changes only ACMXVK and the MIDI-map editor; MXVK and acidcam-gpu are unchanged. Increment 9I adds a windowless shader-library build mode for command-line and interface use. --build <library.json> --builddir <directory> accepts source entries ending in .frag, .comp, or .spv, invokes glslc only for missing, outdated, or invalid compiled modules, and atomically writes a runtime-ready library.json containing the resulting .spv paths. Existing SPIR-V modules are validated and copied when needed. Custom-uniform ranges and defaults are preserved in declaration order. This increment changes only ACMXVK and does not require MXVK or acidcam-gpu to be rebuilt. Increment 9J adds the tolerant --fix <directory> variant of library building. It continues after individual shader compilation, validation, or path failures, removes any stale output module for each failed entry, and writes only successful entries to the generated library.json. Both strict and fix builds now print completion at every 5-percent boundary. Structural failures involving the input manifest, output directory, or final manifest remain fatal. This increment changes only ACMXVK and does not require MXVK or acidcam-gpu to be rebuilt. Increment 9K adds the explicit destructive --prune option for archival cleanup. When combined with --fix, ACMXVK deletes a .frag or .comp source only when an executed glslc process returns a compilation error. Missing tools, interrupted compiler processes, unsafe paths, and input/output filesystem errors never prune source files. This increment changes only ACMXVK and does not require MXVK or acidcam-gpu to be rebuilt. Increment 9L adds a deletion confirmation guard to archival pruning. A --prune request now stops before reading or writing the library and warns that source deletion is permanent unless --force is also present. This increment changes only ACMXVK and does not require MXVK or acidcam-gpu to be rebuilt. Increment 9M adds a complex kaleidoscopic mandala compute shader as a complete four-custom-uniform example. The bundled shader manifest now declares explicit slots for slider1 through slider4, allowing command-line overrides, MIDI maps, and direct MIDI CC mappings to control symmetry, fractal detail, motion, and source/effect blending. This increment changes only ACMXVK and does not require MXVK or acidcam-gpu to be rebuilt. Increment 9N decouples explicit output geometry from preview-window geometry. --resolution now fixes the native shader, snapshot, and encoded-output extent while an oversized preview is uniformly fitted within 90 percent of the usable display and centered with its aspect ratio locked. This matches automatic source-resolution behavior and changes only ACMXVK. Increment 9O adds --unbuffered for launchers and other supervising processes. When enabled, ACMXVK flushes standard output and standard error after every write so startup, build-progress, validation, capture, and teardown messages can be displayed immediately by the ACMX interface. Normal terminal behavior is unchanged when the option is omitted. Increment 9P adds live shader selection from the ACMX Qt interface. Runs launched with --interface-shm consume versioned, semaphore-protected shader selection messages and resolve them strictly through the active manifest. Source-library names such as effect.comp map to effect.comp.spv in the generated runtime library. A live selection uses ACMXVK's normal crossfade and pipeline rebuild path. Later interface increments extend the same channel to source reload and additional runtime settings. Increment 9Q extends ACMX interface control to named custom uniforms. Slider and value changes from the interface are read atomically with the selection sequence, matched against the active runtime manifest, clamped to each uniform's declared range, and uploaded to fragment, compute, multipass, and 3D rendering paths without restarting ACMXVK. Unknown names and non-finite values are ignored safely. Increment 9S adds true surface-free Vulkan batch rendering. --headless and its ACMX2-compatible alias --silent skip SDL video initialization, window creation, the Vulkan presentation surface, swapchain acquisition, and presentation. MXVK-owned RGBA8 targets feed ACMXVK's existing asynchronous recording readback path. Headless mode accepts video or still-image input and requires --output; still images also require a positive --duration. Ctrl+C is handled as a graceful stop request at a frame boundary: pending GPU readbacks are drained before MXWrite closes the output. This increment requires an MXVK installation containing VK_Window::RuntimeMode::Headless.
Process a video without a graphical desktop or preview window:
--silent is an exact alias, so existing ACMX2-style command lines can use:
The host still needs a working Vulkan loader, ICD, and GPU/device access, but it does not need X11, Wayland, Quartz, or another graphical session.
Increment 9T adds ACMX2-style headless completion progress. Progress lines report the percentage, processed and expected frame counts, written frame count, media time, and current encoded size. They are emitted whenever the percentage advances or after 500 ms and finish at 100% after a normal source or duration boundary. Headless --repeat now requires a positive --duration so a looping batch job always has a defined completion point. Ctrl+C remains an early graceful stop and therefore does not misreport the interrupted job as 100% complete.
Increment 9U prevents initial headless target creation from running ACMXVK's sprite/input initialization twice. Video frame zero now remains the first submitted encoder frame at PTS zero instead of being consumed during setup and leaving a synthetic black frame at the beginning of headless output. Windowed initialization and later swapchain recreation behavior are unchanged.
HDR support is planned as six independently testable increments:
All six increments are complete. ACMXVK automatically reports detected HDR video metadata when opening a file, or it can inspect a file without initializing Vulkan:
Increment 3 preserves that precision through RGBA16F scene and effect targets, mixed fragment/compute chains, texture history, crossfades, and 3D texture prepasses. Fragment shaders require no format declaration changes. HDR compute shaders must declare their binding-5 output as layout(rgba16f); MXVK rejects an incompatible rgba8 compute shader with a clear error instead of binding an invalid storage image. Increment 4 wraps the user effect chain in the matching PQ or HLG transfer passes, so shader inputs, intermediate values, texture history, and crossfade history are linear BT.2020. It transfer-encodes the result and reads the final RGBA16F target back as normalized RGBA16. Increment 5 sends that buffer directly to MXWrite, which converts encoded BT.2020 RGB to limited-range YUV420P10LE and writes HEVC Main10 with the source PQ/HLG, primaries, matrix, range, mastering-display, and content-light metadata. Explicit and source-timeline timestamps use the same writer queue as SDR, and the existing post-encode audio copy/mux paths are unchanged. HDR recording selects software libx265 automatically and requires even output dimensions. Increment 6 renders the 3D texture consumer into an RGBA16F final target, so 3D output uses that same Main10 path. A presentation-only PQ/HLG tone mapper converts BT.2020 to BT.709/sRGB for an SDR window without changing the encoded frame. PNG sequences, generated PNGs, and PNG/WebP snapshots use the equivalent CPU tone map. TIFF and raw snapshots retain normalized 16-bit RGBA data. MXVK checks the required sampled/color-attachment/storage format features before creating HDR targets and ACMXVK reports unsupported transfer functions, encoders, or odd Main10 dimensions explicitly.
Increment 7R routes user-controlled strings through one validation module. Paths and visible labels accept well-formed printable UTF-8, including spaces and international filenames, while rejecting embedded controls, malformed encoding, bidirectional overrides, noncharacters, and oversized values. Identifiers, encoder tokens, MIDI expressions, uniform overrides, and FFmpeg option strings use narrower ASCII allowlists appropriate to their grammar. Playlist URLs are restricted to http, https, file, rtsp, and rtmp with basic authority and percent-escape validation.
Text configuration files are limited to 4 MiB; line-oriented index, playlist, and MIDI-map formats also have 4096-byte line limits. Shader, playlist, audio-playlist, and MIDI-map entry counts are bounded. User SPIR-V files must have a valid aligned size and SPIR-V magic word before Vulkan loads them. Command-line dimensions, frame rates, device indices, buffer counts, durations, colors, and other allocation-sensitive numbers also have explicit ranges. Build the regression test with the default BUILD_TESTING=ON and run:
-p/--path accepts an ACMXVK resource root with this layout:
When none of --shaders, --fragment, or --compute is supplied, ACMXVK automatically uses shaders/library.json (or index.txt) from the selected resource root. Relative playlist and MIDI-map names are also searched beneath their matching resource subdirectories. Media input and output arguments remain relative to the current working directory.
Resource precedence is:
An explicit --shaders, --fragment, or --compute always wins. Otherwise, ACMXVK_SHADER_PATH can name a SPIR-V library directory or its library.json or index.txt file. ACMX2_SHADER_PATH is intentionally not consumed because ACMX2 libraries contain OpenGL GLSL rather than MXVK SPIR-V.
For example:
The equivalent environment setup is:
On an Apple Silicon Mac, configure the Vulkan SDK environment according to its installation instructions, then configure an arm64 build with MoltenVK enabled:
ACMXVK_USE_MOLTENVK defaults to ON on Apple platforms. The CMake project uses the Vulkan SDK loader, which discovers MoltenVK as its macOS Vulkan driver. It intentionally does not link libMoltenVK directly because SDL, volk, and validation layers must use the same Vulkan loader dispatch path. Explicitly setting CMAKE_OSX_ARCHITECTURES remains useful when CMake is run from a translated shell or when switching between arm64 and universal builds.
Process a video with a SPIR-V shader library:
Video shaders automatically use decoded source-frame time. The same input frame therefore receives the same iTime, shader-time delta, compatibility frame index, and animated alpha regardless of whether processing is faster or slower than real time. --normalized is not required for video files.
Play a video with effects at its reported source rate instead of processing it as quickly as possible:
This is a real-time playback mode. Frames wait when processing is ahead of the source clock. If an effect cannot render fast enough, ACMXVK skips late source frames so the displayed video position does not drift into slow motion. The mode cannot be combined with --fps, because the reported source rate is the requested clock. P pause and L freeze suspend this clock and resume without jumping over the paused interval. Add --use-source-audio with an AUDIO=ON build to use the video's embedded audio track for shader reactivity. Add --pass-through to hear it; --audio-output <index> selects the output device, which then becomes the more accurate A/V master clock. If the video has no decodable audio track, ACMXVK warns once, continues with zero-valued audio inputs, and disables pass-through.
Render a still image for five seconds and encode it with a software encoder:
Select target-bitrate VBR instead of CRF/CQ with --bitrate (also available as --video-bitrate or --encode-bitrate). Values are bits per second and accept K, M, or G suffixes. This is equivalent to FFmpeg's -b:v 10M setting. The short -b remains an alias for --encode-crf for ACMX2 command compatibility:
Target bitrate controls the encoder's average rate rather than imposing a hard per-second cap, so short clips and keyframe-heavy sections can exceed the requested rate. Omitting this option retains --encode-crf quality mode.
Take a processed PNG snapshot by pressing Z. The directory is created on the first request:
Snapshots include the timestamp, rendered resolution, and a collision-safe sequence number in their filename. Readback is enabled for only the requested frame unless video or PNG output already requires continuous readback. PNG compression and disk writes run on a bounded background queue so the render loop can continue while the snapshot is saved; queued work is drained during shutdown.
To enable lossless WebP snapshots, install the libwebp development package and configure ACMXVK with -DWEBP=ON. Press 5 to capture WebP instead of Z; it uses the same processed frame, naming scheme, --prefix directory, background queue, and shutdown draining behavior as PNG:
To enable lossless TIFF snapshots, install the libtiff development package and configure with -DTIFF=ON. Press 4 to capture an LZW-compressed RGBA TIFF using the same processed frame, naming scheme, --prefix directory, and background queue. SDR captures are 8-bit; HDR captures retain 16-bit samples:
Press 6 to save the same final processed frame as a headerless .raw file. SDR captures are RGBA8; HDR captures are native-endian normalized RGBA16. Its dimensions are embedded in the filename. Supply those dimensions when opening an SDR capture with a raw-video tool, for example:
Show the active shader/filter information with a yellow watermark. Configuring a watermark starts the preview-only runtime HUD hidden; press F9 to show it. Press E to toggle only the watermark while the filter information remains visible:
The source overlay font defaults to ACMX2/data/font.ttf, is copied into the build resource tree, and is installed with ACMXVK. It can be replaced at configure time with -DACMXVK_OVERLAY_FONT=/path/to/font.ttf.
Preview the included shader with live audio reactivity:
Use an audio or video file as the reactive source without opening a microphone:
Display the current 256-bin FFT spectrum over an image:
Display an eight-frame FFT waterfall over an image:
Enable an eight-layer frame-history cache:
Override a custom float declared by library.json:
Shader changes use a 0.5-second crossfade by default. Set the duration in seconds with --cross-fade; use 0 to switch immediately:
Press [ or ] to select the previous or next transition. In 3D mode those keys retain ACMX2's model-scale action as well. Press N to toggle random transition selection for sequential or random autopilot changes. The HUD reports the current one as XFade [index/35]: name.
Playlist autopilot intervals count decoded source frames for video input. Pausing the video or presenting the same frame more than once does not consume the countdown, while source frames discarded to catch up during --use-source-fps playback do count. Camera and still-image modes count rendered frames as before.
The bundled set matches ACMX2: linear, block, wipe, radial, pixelate, dissolve, swirl, glitch, diamond, burn, fade-to-black, fade-to-white, four slides, diagonal wipe, iris open/close, checker, horizontal/vertical blinds, zoom in/out, rotate, ripple, wave, chroma, invert, flash, explode, mosaic, shutter, luma, and noise. Crossfading is the final stage of the active Vulkan chain, so it also works with fragment shaders, compute shaders, mixed multipass playlists, final-output flipping, bypass, and the 3D model texture pipeline.
For an editor or interface, keep a source library.json beside its GLSL files. It uses the normal manifest schema, but its shaders array may contain .frag, .comp, and already compiled .spv entries:
Build it without opening an ACMXVK window:
To repair a large library while omitting shaders that do not compile, replace --builddir with --fix:
For archival cleanup, --prune additionally deletes GLSL entry files that actually fail compilation:
--prune is intentionally destructive and requires both --fix and --force. Without --force, ACMXVK prints a permanent-deletion warning and stops before touching the manifest, source files, or output directory. Confirmed pruning deletes only the failing .frag or .comp source itself; it never deletes .spv source entries or files merely affected by a missing compiler, invalid path, output failure, or interrupted compiler. The source library.json is retained as an audit of what was attempted, while the generated output library.json contains only successful shaders. Regenerate the source manifest afterward if the archive also needs a manifest containing only files that remain.
The result contains effects/color.frag.spv, compute/feedback.comp.spv, the copied prebuilt module, and a runtime-ready shader-library/library.json. Repeating the command skips valid outputs whose timestamps are at least as new as their sources. Both modes print progress at 5-percent intervals. In strict --builddir mode, a failed compile leaves that shader's previous module and the runtime manifest intact and returns a nonzero exit status. In --fix mode, ACMXVK continues, removes the failed shader's stale module, omits it from the new runtime manifest, reports it in the summary, and returns success after all recoverable entries have been examined. This lets an interface choose between fail-fast compilation and producing the largest valid library from a mixed set of sources.
--parallel <jobs> runs between 1 and 256 shader build jobs concurrently and is valid only with --build. The default is one job, preserving the original serial behavior. Parallel mode applies equally to strict --builddir and recovery-oriented --fix builds, including confirmed pruning. Output manifest entries retain source-manifest order even when jobs finish in a different order. Progress reflects completed entries across all workers. If a strict parallel build fails, already-running jobs are allowed to finish before ACMXVK returns the compiler error; the runtime manifest is not replaced.
ACMXVK-generated manifests include "backend": "acmxvk" so a compatible launcher can validate its active engine. Source manifests use "library_type": "source"; --build, --fix, and the conversion script emit "library_type": "runtime" for libraries whose entries are executable SPIR-V. Both fields are backward-compatible metadata and do not change command-line loading behavior.
slot is optional for older manifests. When supplied, every custom uniform must have a unique, contiguous slot starting at zero. Explicit slots are recommended for generated source libraries because JSON object ordering is not part of the JSON standard. The output manifest always records them.
glslc is found through PATH by default. An interface may select a specific Vulkan SDK compiler with --glslc /path/to/glslc. Source and output entries must remain below their respective library roots; absolute paths, parent traversal, duplicate output names, and symbolic-link output escapes are rejected. The output directory must differ from the source directory.
This mode compiles Vulkan-compatible GLSL; it does not rewrite legacy ACMX2 OpenGL syntax or invent custom-uniform definitions. Use the conversion script described below for legacy shaders, then use --build for subsequent incremental compilation.
The Qt interface's List > New Shader Library and New Shader File actions generate and refresh ACMXVK source manifests directly in C++. The generator discovers .frag sources and compute/*.comp sources recursively, reconstructs explicit custom-uniform slots from converter aliases, preserves edited uniform ranges and the canonical 0–26 slot layout, and refuses slot or case-insensitive output conflicts. This replaces the former standalone Perl manifest script.
The interface build also produces a headless frontend using that same C++ implementation:
Use --output FILE to place the source manifest somewhere other than ROOT/library.json.
ACMXVK accepts library.json entries as strings or objects containing a file field:
When library.json is absent, index.txt is read as one relative .spv path per line. Absolute paths, parent-directory traversal, files outside the library, and non-SPIR-V entries are rejected.
Each entry can contain either a fragment or compute entry point. ACMXVK reads the execution model and compute local size directly from SPIR-V, so mixed chains work through --shader-pass, --shader-pass-files, and playlists without stage metadata in the manifest. Vertex and unsupported shader stages are rejected before a pipeline is attached.
The repository includes scripts/convert_shaders_to_acmxvk.pl for bulk conversion of ACMX2/OpenGL shader libraries. By default it reads fragment shaders from shaders_new, compute shaders from compute, and writes a separate shaders_acmxvk directory, leaving every original file untouched:
The output contains Vulkan-compatible .frag and .comp source files, compiled .spv modules, library.json, and conversion-report.txt. Only successfully compiled shaders are added to the manifest. Re-run with --force to replace previously generated outputs, or use --input, --output, --compute-input, and --glslc to select other locations. Converted compute sources and modules are placed beneath shaders_acmxvk/compute. --dry-run lists candidates without writing anything. The converter reports history-cache shaders separately and includes them in the manifest. ACMXVK detects their binding 2 directly from SPIR-V and enables the shared input-frame history cache automatically.
ACMXVK shaders are full-frame post-processing stages. The application renders the input image, camera, or video first, then executes the configured pass list in order. Each pass samples the preceding result. A fragment or compute shader can be the first, middle, or final pass, and a chain can mix both stages. MXVK reads the stage and compute workgroup size from the SPIR-V module; the filename suffix is only a naming convention.
ACMXVK supplies sprite.vert.spv for every fragment pass. A shader library therefore contains fragment and compute modules, not user vertex modules. The first configured pre-pass runs first, the shader selected by --shader-file runs after the pre-passes, and the internal vertical-flip pass runs last when --flip is enabled.
Every user pass operates at the current render/swapchain resolution, not necessarily at the camera or source-file resolution. The output dimensions are available as ext.u0.zw. The sampled input at binding 0 and compute output at binding 5 have those same dimensions.
Fragment shaders receive tc from ACMXVK's vertex shader. It is a normalized texture coordinate, normally in the inclusive range from 0.0 through 1.0. gl_FragCoord.xy is also available when integer pixel coordinates are more convenient. Compute shaders do not receive tc; derive it from gl_GlobalInvocationID.xy and imageSize(output_image) as shown below.
The pass images use normalized RGBA8 color. Sampled values are floating-point RGBA in the 0.0 through 1.0 range, and storage-image writes outside that range are clamped by the RGBA8 target. Binding-0 sampling uses normalized coordinates, linear filtering, clamp-to-edge addressing, and no mipmaps. A full-screen fragment effect should normally write alpha 1.0; fragment pipelines enable alpha blending, and intermediate attachments do not preserve a defined destination color for partial-alpha composition. Do not rely on an alpha below 1.0 unless the particular pass has been designed and tested for that behavior.
All resources use descriptor set 0. A shader should declare only resources it actually needs because optional bindings are absent from the pipeline layout when their feature is unavailable.
| Binding | GLSL declaration | Stages | Contents and availability |
|---|---|---|---|
| 0 | uniform sampler2D input_image; | Fragment and compute | Always present. The image produced by the preceding stage. For PQ/HLG HDR video, ACMXVK decodes this value to linear BT.2020 before the user chain and transfer-encodes it again after the chain. The name may be samp, input_image, or any other valid identifier. |
| 1 | uniform SpriteExtended { ... } ext; | Fragment and compute | Always present. Per-frame state, mouse, timing, audio values, and custom library uniforms. |
| 2 | uniform sampler2DArray history; | Fragment and compute | Optional shared RGBA input-frame history ring. --texture-cache enables it explicitly without adding an effect; ACMXVK also enables it automatically when a directly loaded shader or any entry in library.json declares set 0, binding 2. Every pass in the active fragment/compute chain shares the same ring. Use --history-test only when you also want ACMXVK's built-in echo demonstration applied before the selected pipeline. |
| 3 | uniform sampler1D spectrum; | Fragment and compute | Current 256-bin R32 floating-point FFT. ACMXVK detects this binding from SPIR-V and supplies a zero-initialized descriptor when audio support or an active audio source is unavailable. |
| 4 | uniform sampler1DArray spectrum_history; | Fragment and compute | Circular FFT history. ACMXVK detects this binding from SPIR-V and automatically allocates eight zero-initialized layers when --enable-audio-buffers N was not supplied. |
| 5 | layout(rgba8) or layout(rgba16f) writeonly uniform image2D output_image; | Compute only | Compute destination. Always required by an ACMXVK compute shader and unavailable to fragment shaders. Use rgba8 for SDR input and rgba16f for HDR input. |
Binding numbers, descriptor types, array lengths, and block-member order are part of the ABI and must match exactly. Resource and member names are not part of the ABI and may be renamed.
Binding 1 uses std140 layout. Declaring std140 explicitly is recommended so the offsets are obvious and stable:
Every entry is a vec4 aligned to 16 bytes. A shader may stop the declaration after the last field it uses. For example, a shader which needs only time and resolution may stop after u2. It must not remove or reorder earlier fields: declaring audio_bands immediately after u3 would read custom-uniform memory, not the audio bands.
The complete field map is:
| Field | Offset | Components | Value |
|---|---|---|---|
| mouse | 0 | .x, .y | Mouse position in window pixels. SDL supplies a top-left-origin position. |
| .z | 1.0 while the left mouse button is held, otherwise 0.0. | ||
| .w | Reserved; currently 0.0. | ||
| u0 | 16 | .x | ACMX2-compatible animated alpha, advanced by 0.1 and reflected between 1.0 and 6.0. Video mode derives it from the decoded source-frame index; other modes advance it per rendered frame. |
| .y | ACMX2/Shadertoy-compatible iTime. Video input uses decoded source-video time, making effects independent of processing speed. Camera and still-image modes use elapsed wall time. | ||
| .z, .w | Render width and height in pixels. A convenient alias is vec2 resolution = ext.u0.zw. | ||
| u1 | 32 | .x | Frame delta in seconds. Video input uses the decoded source-frame difference, including skipped source frames. Outside video mode, --normalized uses exactly 1.0 / output_fps; otherwise this is measured wall-clock delta. |
| .y | amp: processed mean audio amplitude after sensitivity, warmup, time-speed, and optional delta scaling. Zero without active audio. | ||
| .z | iamp: zero-crossing frequency estimate in Hz. | ||
| .w | Source-video FPS in video mode; otherwise the instantaneous render rate calculated as 1.0 / u1.x. | ||
| u2 | 48 | .x | Decoded source-frame index in video mode. Camera and still-image modes use the rendered-frame counter, which resets with shader time. |
| .y | Shader time in seconds. It follows --time-speed, normalized time, the T/U/I/Page Up/Page Down controls, and audio-reactive time when enabled. | ||
| .z | iSampleRate: active audio sample rate in Hz; the compatibility default is 44100.0. | ||
| .w | amp_peak: sensitivity- and warmup-scaled peak audio level. | ||
| u3 | 64 | .x, .y | Frame-history write head and layer count. The values are supplied to every fragment and compute pass when the shared input history cache is active. |
| .z | amp_rms: sensitivity- and warmup-scaled RMS audio level. | ||
| .w | amp_smooth: sensitivity- and warmup-scaled smoothed audio amplitude. | ||
| custom_uniforms | 80 | 16 vec4s | Up to 64 user floats from library.json, packed in declaration order. Unused slots are 0.0. The array occupies byte offsets 80 through 335. |
| audio_bands | 336 | .x, .y, .z | amp_low, amp_mid, and amp_high: scaled energy below 300 Hz, from 300 through 3000 Hz, and above 3000 Hz. |
| .w | Reserved; currently 0.0. | ||
| audio_history | 352 | .x | Physical array layer containing the newest FFT spectrum. |
| .y | Number of allocated FFT-history layers. | ||
| .z | Number of bins per layer, currently 256.0. | ||
| .w | Reserved; currently 0.0. |
The entire audio_history vector is zero when FFT history is not enabled. The audio scalar fields remain safe to declare in any build because they are part of binding 1; they simply remain zero when no audio source is active. Bindings 3 and 4 are separate optional descriptors and follow the availability rules in the descriptor table.
mouse.xy uses output-window pixels rather than normalized coordinates. A typical conversion is vec2 mouse_uv = ext.mouse.xy / max(ext.u0.zw, vec2(1.0));. Unlike the Shadertoy iMouse convention, mouse.z is only a pressed/not-pressed value and mouse.w does not contain a click position.
Useful aliases can make a port from an ACMX2 shader easier to read:
A fragment shader needs one normalized input coordinate, one RGBA output, and binding 0. Binding 1 is optional when none of its values are needed:
The fragment-stage names are conventional, but their locations and types are fixed:
| Fragment value | Meaning |
|---|---|
| layout(location = 0) in vec2 tc | Interpolated normalized coordinate from ACMXVK's full-screen vertex shader. The variable may be renamed. |
| layout(location = 0) out vec4 color | RGBA result for this pixel. The variable may be renamed. |
| gl_FragCoord.xy | Pixel-center position in framebuffer coordinates. Useful for grids and effects which must align exactly to output pixels. |
| texture(input_image, uv) | Filtered sample using normalized coordinates. Coordinates are clamped at the image edges. |
| texelFetch(input_image, pixel, 0) | Exact unfiltered integer-pixel sample from mip level zero. Clamp pixel before fetching. |
| textureSize(input_image, 0) | Integer dimensions of the preceding pass image. They normally equal ivec2(ext.u0.zw). |
Fragment shaders may also declare MXVK's 48-byte push-constant block. This block is not available to compute shaders:
| Push-constant field | ACMXVK value for a full-frame user pass |
|---|---|
| screen_width, screen_height | Current render dimensions in pixels. |
| sprite_pos_x, sprite_pos_y | 0.0, 0.0. |
| sprite_size_w, sprite_size_h | Current render dimensions in pixels. |
| effects_on | 1.0 while effects are active. Pressing Space bypasses the user pipeline entirely. |
| rotation_degrees | 0.0 for the full-screen post-process quad. |
| params.xyz | Compatibility constants, currently 1.0, 1.0, 1.0. |
| params.w | Shader time in seconds, equivalent to ext.u2.y. |
shaders/custom_uniform.frag, shaders/audio_reactive.frag, shaders/audio_spectrum.frag, and shaders/audio_history.frag demonstrate progressively larger parts of the fragment ABI.
An ACMXVK compute shader samples binding 0 and writes binding 5. Compute pipelines have no push-constant range, so use SpriteExtended for time, resolution, mouse, audio, and custom values.
The complete example above is the SDR form. Change only the binding-5 storage qualifier from rgba8 to rgba16f when the shader will run on HDR input.
The standard compute built-ins are useful for effects which operate on tiles or share data within a workgroup:
| Compute built-in | Meaning |
|---|---|
| gl_GlobalInvocationID | Absolute unsigned invocation coordinate. .xy is the output pixel in the usual one-invocation-per-pixel design. |
| gl_WorkGroupID | Workgroup coordinate within the dispatch. |
| gl_LocalInvocationID | 3-D coordinate of the invocation inside its workgroup. |
| gl_LocalInvocationIndex | Flattened one-dimensional index inside the workgroup. |
| gl_NumWorkGroups | Number of dispatched workgroups in each dimension. MXVK calculates .xy by rounding the output size up to the declared local size; .z is one. |
| gl_WorkGroupSize | Compile-time uvec3 matching local_size_x, local_size_y, and local_size_z. |
Use texture(input_image, uv) for filtered sampling or texelFetch(input_image, pixel, 0) for an exact input texel. Use imageSize(output_image) for bounds. Because output_image is declared writeonly, read the old color from input_image, not with imageLoad.
MXVK dispatches ceil(width / local_size_x) by ceil(height / local_size_y) by one workgroup. Always bounds-check gl_GlobalInvocationID.xy, declare local_size_z = 1, and write every in-range pixel. Input and output are different ping-pong images, so reading binding 0 and writing binding 5 never aliases the same image. MXVK inserts the required barriers before the next fragment or compute pass. If the final pass is compute, MXVK presents its result with an internal full-screen copy.
Use literal local_size_x, local_size_y, and local_size_z values as in the example. MXVK currently reads SPIR-V LocalSize metadata; specialization-ID forms such as local_size_x_id are not supported for dispatch sizing.
shaders/compute_test.comp is the complete working reference. CMake compiles it as compute_test.comp.spv with an rgba8 output for SDR and as compute_test_hdr.comp.spv with an rgba16f output for HDR. Workgroups of 8x8 or 16x16 are sensible starting points; performance depends on the shader and GPU.
shaders/mandala_fractal.comp is a more complete artistic compute example. It combines polar kaleidoscope folding, iterative inversion, orbit traps, radial lace, animated rings, source-image distortion, and palette synthesis. Its time comes from ext.u2.y, so a particular source timestamp produces the same animation whether video is played in real time or processed offline.
The bundled shaders/library.json assigns the four controls to explicit slots:
| Uniform | Shader location | Effect |
|---|---|---|
| slider1 | ext.custom_uniforms[5].x | Selects approximately 4–20 radial symmetry sectors. |
| slider2 | ext.custom_uniforms[5].y | Selects 3–10 fractal iterations and increases spatial detail. |
| slider3 | ext.custom_uniforms[5].z | Controls animation speed, iterative rotation, and source-image warp. |
| slider4 | ext.custom_uniforms[5].w | Blends from the unchanged source at 0.0 to the complete mandala at 1.0. |
Run it from the bundled shader library so ACMXVK loads the uniform definitions:
The same names work with MIDI Slider 1–4 actions. Direct CC mappings can also be supplied, for example --midi-cc 20=slider1 through --midi-cc 23=slider4.
shaders/mandala_lotus.comp draws nine interwoven petal rings, beads, and fine radial filigree. shaders/mandala_crystal.comp uses nine folded inversion iterations with ring and cross orbit traps for a crystalline mandala. Both generate their own opaque image, so the input picture is not visible. Animation uses media time (ext.u2.y), independently of processing speed.
| Control | Lotus | Crystal |
|---|---|---|
| slider1 | 6–24 petals | 6–24 symmetry sectors |
| slider2 | Zoom and petal depth | Zoom and sharper fractal detail |
| slider3 | Rotation/animation speed | Rotation and orbit animation speed |
| slider4 | Palette hue | Palette hue |
All four controls accept 0–1 and use slots 20–23 (ext.custom_uniforms[5].xyzw), matching the bundled library and MIDI slider actions. A complete palette cycle occurs from slider4 = 0 to 1. The effects remain visible even with all sliders at zero.
CMake builds both automatically into build/acmxvk/shaders and installs their SPIR-V files with the example library. Pcons also picks them up automatically. These examples use the standard SDR rgba8 compute output.
Replace the shader filename with mandala_crystal.comp.spv for the second effect. Add --midi-device 0 --midi-map ACMXVK/midi-examples/nanocontrol-slider.midi_cfg to use the bundled nano controller slider mapping.
Custom uniforms use their explicit manifest slot when one is present; older manifests without slots are packed in declaration order. Uniform slot N is stored in ext.custom_uniforms[N / 4][N % 4]. For example, the first five slots map as follows:
| Declaration index | Shader location |
|---|---|
| 0 | ext.custom_uniforms[0].x |
| 1 | ext.custom_uniforms[0].y |
| 2 | ext.custom_uniforms[0].z |
| 3 | ext.custom_uniforms[0].w |
| 4 | ext.custom_uniforms[1].x |
Given the earlier square_size manifest entry, the shader can use:
The Qt interface's Custom Uniforms dialog displays this location for every entry and provides a Copy button for the complete #define. Adding a uniform appends the next contiguous explicit slot to library.json; deleting one removes its entry and renumbers later slots. Because deletion can therefore change later locations, recopy any affected aliases into their shader sources. The dialog refuses additions beyond 64 scalar slots, matching the custom_uniforms[16] block.
The manifest's minimum, maximum, step, and value fields define the accepted range, adjustment step, and initial value. Repeat --uniform name=value to override initial values. ACMX2 MIDI Slider 1 through Slider 4 target custom uniforms named slider1 through slider4. ACMXVK's generated libraries reserve canonical slots 20 through 23 for these names, so their shader aliases are ext.custom_uniforms[5].x through ext.custom_uniforms[5].w. Keep explicit slots and shader aliases consistent after compiling a shader. See shaders/custom_uniform.frag and shaders/midi_slider.frag.
In an AUDIO=ON build, binding 3 contains 256 non-negative FFT magnitudes in an R32_SFLOAT 1-D texture:
Coordinate 0.0 is the lowest-frequency/DC end and 1.0 is the Nyquist end. The texture uses linear filtering, so coordinates between bins interpolate. Squaring a normalized coordinate before sampling, as in audio_spectrum.frag, devotes more screen space to lower frequencies.
With --enable-audio-buffers N, binding 4 stores previous FFTs in a circular sampler1DArray. ext.audio_history.x identifies the newest physical layer and .y gives the allocated count:
Age zero is newest, age one is the preceding FFT, and so on. The requested history depth may be clamped to the GPU's maximum image-array layer count.
Source the Vulkan SDK used by the project, compile GLSL to SPIR-V, and validate the result before adding it to library.json:
For a filename without a recognized stage suffix, specify it explicitly with glslc -fshader-stage=fragment or glslc -fshader-stage=compute. Test a single module with --fragment or --compute; use --shaders after adding it to a manifest. Validation builds report descriptor-layout, storage-image, and synchronization mistakes at runtime.
Common shader problems are:
Test the standalone compute path after reinstalling MXVK 0.29.0:
The generated test library already contains the compute pass and MIDI-slider fragment pass. Select compute as the pre-pass and fragment as the active shader:
With an AUDIO=ON build, --enable-audio uses live RtAudio input while --audio-file <media> uses the first audio stream decoded by FFmpeg. It also accepts an M3U or M3U8 playlist. Both paths map the same audio metrics into the binding-1 block:
| ACMX2 name | MXVK field | Meaning |
|---|---|---|
| amp | ext.u1.y | Processed mean amplitude after sensitivity, warmup, time-speed, and optional delta scaling |
| iamp | ext.u1.z | Zero-crossing frequency estimate in Hz |
| iSampleRate | ext.u2.z | Active input sample rate |
| amp_peak | ext.u2.w | Sensitivity-scaled peak level |
| amp_rms | ext.u3.z | Sensitivity-scaled RMS level |
| amp_smooth | ext.u3.w | Sensitivity-scaled smoothed amplitude |
| amp_low | ext.audio_bands.x | Sensitivity-scaled energy below 300 Hz |
| amp_mid | ext.audio_bands.y | Sensitivity-scaled energy from 300 through 3000 Hz |
| amp_high | ext.audio_bands.z | Sensitivity-scaled energy above 3000 Hz |
See shaders/audio_reactive.frag for a working shader. Audio values remain zero when capture is disabled or an input device cannot be opened.
File audio is decoded up front to mono 44.1 kHz floating-point samples. It is a silent analysis source by default and advances according to ACMXVK's output frame rate. Add --pass-through to play it through RtAudio's default output; --audio-output <index> selects another output listed by --list-devices. During pass-through, the output device becomes the master audio clock and the shader analysis follows its playback position. This keeps visual reactivity aligned even when rendering runs faster or slower than the requested FPS.
When an encoded --output is open, file audio is automatically encoded as mono AAC at 192 kbps and muxed into the completed video. This is implemented directly with the linked FFmpeg libraries; ACMXVK does not invoke the ffmpeg command-line program. The original encoded video remains untouched until the temporary mux output has been finalized successfully. MP4, MOV, and other AAC-compatible containers are supported; if a selected container rejects AAC, ACMXVK reports the mux failure, removes the temporary file, and preserves the video-only recording. Add --mute-output to keep file audio available for reactivity, playback timing, and pass-through while leaving the recorded video audio-free.
When live --enable-audio input and an encoded --output are active, ACMXVK also records the microphone automatically. Capture begins immediately before the first frame is submitted to MXWrite and stops before the writer closes, so device and shader initialization do not add leading audio. Input channels are downmixed to mono, resampled to 44.1 kHz when needed, encoded as AAC at 192 kbps, and muxed through the same linked-library path as file audio. No WAV file or external process is required. --copy-audio takes precedence and disables live-input recording so the selected video input's original audio is copied instead.
Add --pass-through to monitor live input through RtAudio's default output, or select an output from --list-devices with --audio-output <index>. Live monitoring uses one full-duplex stream, so the samples sent to the output are the same samples used for shader analysis and recording. ACMXVK selects a sample rate supported by both devices and duplicates mono input into a stereo output when needed. Headphones are recommended to avoid acoustic feedback. Use --pass-through-gain <0.0-4.0> when the input device does not expose a system level control. Gain affects only monitored output: shader metrics and recorded or muxed audio retain the original samples. Values above 1.0 amplify the monitor signal, and samples are clamped to the floating-point audio range to prevent overflow.
Use --record-gain <0.0-2.0> to amplify quiet live-input samples before they are encoded into the final video's AAC stream. Recording gain is independent from --pass-through-gain and --sense: it does not change headphone volume or shader response. Unity gain remains the default, and amplified samples are clamped to the floating-point audio range before encoding.
--mute-output suppresses every encoded-video audio path: automatic file-audio muxing, automatic live-input muxing, and --copy-audio. It does not mute an RtAudio pass-through device, disable shader reactivity, alter the audio master clock, or disable a separately requested --record-audio WAV file.
Audio-reactive shader values ramp from zero at startup at a default rate of 0.5 per second, reaching full strength in about two seconds. Set --audio-warm-rate <rate> to change the slope, or use --audio-warm-rate 0 to disable warmup. The envelope scales amplitude, peak/RMS/bands, the current FFT, and FFT history. It does not alter monitoring, recording, frequency estimation, or the reported sample rate.
The ACMX2 audio-time controls are available while a live or file audio source is active. Press Q to make audio amplitude advance shader time instead of the wall clock. Home toggles frame-delta scaling for both reactive time and the amp value, which makes their behavior less dependent on rendering frame rate. End toggles sensitivity scaling for the current FFT and FFT-history textures. The same actions are accepted from ACMX2 MIDI Map codes 81, 268, and 269.
Without repeat, the muxed result is limited to the shorter of the recorded video and decoded audio. With --audio-repeat, the complete file or playlist is repeated to the recorded video duration. --audio-trunc stops recording at the file-audio EOF; this also permits a still-image recording without an explicit --duration. At end-of-stream without truncation, preview continues with zero-valued audio metrics. When repeat and truncation are both supplied, repeat keeps the source active, matching ACMX2 behavior.
For example, loop a song with audible pass-through:
Replace --audio-repeat with --audio-trunc to close the application at the end of the song.
Record five seconds of processed video and mux repeated file audio into it:
Use source-video audio for synchronized reactivity and audible pass-through, but create a video-only recording:
For video-file input with a reactive audio track, the recorded frame PTS now follow the source-video timeline. Encoder queue drops therefore leave timestamp gaps instead of shortening the result. With --pass-through, the audio output device becomes the master clock: ACMXVK waits when video is early, efficiently skips decoded frames when video is late, and submits the displayed frame at the matching timeline PTS. Without pass-through, file-audio analysis advances by exactly one nominal video frame and produces the same offline alignment.
Record processed video with synchronized live microphone audio:
Monitor that microphone through the default output while recording:
Record live microphone input to a standalone mono PCM16 WAV while processing a video. --record-audio enables live audio capture automatically, and --record-gain is applied only to the saved or muxed microphone samples:
The WAV closes when the video ends or ACMXVK exits. It can be written alongside an encoded --output; in that mode the same captured samples can also be muxed into the video. Standalone recording uses live input and therefore cannot be combined with --audio-file.
M3U and M3U8 playlists are read in order. Blank lines, #EXTM3U, #EXTINF, and other comment lines are ignored; relative entries are resolved against the playlist directory. Unusable tracks are reported and skipped as long as at least one entry can be decoded. --audio-repeat restarts the complete playlist, and --audio-trunc exits only after its final usable track.
audio_bands and audio_history are appended after custom_uniforms[16]. This preserves every existing field and custom-float offset; older shaders may stop their uniform block at u3, custom_uniforms, or audio_bands without declaring the newer suffixes.
The FFT uses a 512-sample Hann window and exposes 256 linear-frequency bins. Coordinate 0.0 is DC and 1.0 approaches the Nyquist frequency. Declare it in a Vulkan shader as:
See shaders/audio_spectrum.frag for a complete visualization. MXVK's extended ABI permits the descriptor alongside binding-2 frame history. ACMXVK shares the input sprite's single history array with every user post-processing pass, avoiding a duplicate ring allocation per pass.
--enable-audio-buffers N allocates binding 4 as a circular FFT-history array. The requested depth is clamped to the Vulkan device's maximum image-array-layer count. ext.audio_history.x contains the physical layer holding the newest spectrum, ext.audio_history.y contains the allocated layer count, and ext.audio_history.z contains the number of bins. A shader can sample an age without copying layers:
See shaders/audio_history.frag for a current-versus-history waterfall test.
Configure with -DMIDI=ON to enable RtMidi. List available input ports with:
Open a port and print its incoming byte messages while ACMXVK runs:
When --midi-device is omitted, MIDI monitoring or mapping opens port zero. RtMidi's callback feeds a bounded, thread-safe queue drained by the render loop; if more than 256 unprocessed messages arrive, ACMXVK retains the newest messages and reports the drop count.
Increment 6B reads the same .midi_cfg format written by ACMX2/interface/midi-map. Use that tool to capture controller messages, save the map, and load it with --midi-map. Slider 1–4 actions (600:601 through 606:607) update custom uniforms named slider1 through slider4. Values are normalized from MIDI's 0–127 range into each uniform's minimum and maximum range from library.json. Supported ACMXVK-equivalent actions include shader selection, bypass, playlist/pause, freeze, time stepping and speed, multipass, audio sensitivity, autopilot, and the MXVK screenshot key.
The build shader directory now contains a small library.json and midi_slider.frag.spv test library. The included nanoKONTROL2 example maps the channel-1 CC 20 messages shown above to Slider 1:
Moving CC 20 changes image brightness and prints both the raw message and the mapped slider1 value. To map a CC directly without a .midi_cfg file, repeat --midi-cc [channel:]CC=uniform; an omitted channel matches all channels:
Increment 6C matches ACMX2's paired-knob behavior. The most recent CC value is held until the controller sends neutral value 64; distance from 64 controls the action rate, ranging from about once every 16 frames near center to every frame at either extreme. The included CC21 example controls shader-time speed:
This centered mode is intended for spring-return or relative knobs. Use a Slider 1–4 mapping or --midi-cc for an absolute knob or fader. When a map is loaded, ACMXVK reports how many mappings are active and how many refer to ACMX2 features that have not yet been ported; --midi-monitor also names individual unavailable action-code pairs.
Increment 7A adds the first acidcam-gpu processing path. Configure a separate CUDA build on Linux with an NVIDIA GPU:
Confirm support and inspect the available devices and filters:
Apply one filter to a video before its Vulkan shader pipeline:
--gpu-filter accepts a comma-separated chain such as 1,7,23. Filter indices must be in the range printed by --list-filters, and --gpu-buffer accepts 4–32 temporal frames. Video and camera inputs are filtered for every captured frame. A still image is filtered when its source texture is initialized and reprocessed immediately when Left/Right changes the active filter.
The filter engine uploads source RGBA into acidcam-gpu's temporal CUDA buffer, runs the selected chain, and gives MXVK the resulting CUDA GpuMat; MXVK then copies device-to-device into its Vulkan image. There is no post-filter CPU download. Vulkan fragment and multipass shaders operate on the filtered image. In Increment 7B, Left/Right selects the previous or next filter with wraparound. This matches ACMX2: selecting a filter at runtime replaces a startup filter chain with that single filter. MIDI Map actions 262 (Right) and 263 (Left) drive the same selection when MIDI support and a GPU filter are enabled.
Increment 7C writes the filtered CUDA output into the Vulkan frame-history array, including its initial fill, delayed ring-buffer updates, and still-image filter changes. MXVK imports the layered Vulkan image into CUDA and copies the selected frame device-to-device. If layered external-memory import is not available on a driver, ACMXVK reports it once and falls back to host staging for history only; the primary sprite upload remains device-to-device. On a multi-GPU system, select the CUDA device corresponding to the Vulkan GPU. CUDA filters are unavailable in MoltenVK builds.
Increment 7D removes the CPU RGBA handoff for unrotated video and camera input. MXVK returns a resident CUDA GpuMat, acidcam-gpu copies it device-to-device into its temporal buffer, and the filtered result continues directly into the Vulkan sprite and optional history array. This path is selected automatically when --gpu-filter is active.
Increment 7E keeps --rotate clockwise, --rotate 180, and --rotate counterclockwise on that resident path. Exact CUDA transpose and flip operations run on MXVK's capture stream before acidcam-gpu consumes the frame, avoiding the former CPU rotation and upload. Still graphics continue to begin from their host image because they are decoded only once.
Increment 7F prefers MXVK's FFmpeg capture for video files. On a CUDA-enabled MXVK build and a supported codec, FFmpeg decodes with NVDEC and supplies a resident CUDA RGBA frame directly to the 7E rotation/filter/upload path. The application reports the selected decoder at startup; software FFmpeg decoding and OpenCV remain automatic fallbacks. Cameras continue through VK_Capture, and --repeat reopens the same FFmpeg/NVDEC path at end-of-stream.
Increment 7G removes the acidcam-gpu-filter requirement from the resident NVDEC path. In a -DWITH_CUDA=ON build, a supported video can now flow from NVDEC through optional CUDA rotation directly into the Vulkan sprite and history array even when --gpu-filter is omitted. If direct CUDA/Vulkan interop is unavailable, ACMXVK reports the condition once and uses host staging. The 7F filtered route remains unchanged.
Increment 7H separates MXVK CUDA interop from ACMXVK's WITH_CUDA filter option. When the installed MXVK package was built with CUDA, an ordinary -DWITH_CUDA=OFF ACMXVK build now includes resident NVDEC upload, rotation, and history without locating or linking acidcam-gpu. Configure -DWITH_CUDA=ON only when the acidcam-gpu filter library is wanted. MoltenVK and non-CUDA MXVK installations continue compiling the portable host path.
Increment 7I completes that separation for the CUDA command-line controls. --list-cuda-devices and --cuda-device now work whenever the installed MXVK package has CUDA interop, including -DWITH_CUDA=OFF ACMXVK builds. The selected device is applied before rendering and forwarded to MXVK's FFmpeg capture, so its NVDEC context and optional acidcam-gpu filters use the same CUDA device. --check-cuda reports MXVK interop and acidcam-gpu filter support separately, and capture startup identifies the active NVDEC device.
Increment 7J makes repeated FFmpeg input seek back to the first video timestamp without destroying the decoder. --repeat therefore preserves the active NVDEC device and CUDA hardware context across loops and avoids repeatedly allocating and reopening the decoder. If an input is not seekable, ACMXVK retains the previous close-and-reopen behavior as an automatic fallback.
Increment 7K protects each FFmpeg-owned NVDEC surface until MXVK's asynchronous device-to-device plane copies have completed. A CUDA event marks that ownership boundary while later color conversion, rotation, filtering, and Vulkan upload remain queued asynchronously. This prevents FFmpeg from recycling a decode surface while MXVK still reads it without introducing a full capture-stream synchronization. MXVK reports the active barrier once per capture session.
Increment 7L ports ACMX2's standalone --record-audio <wav> workflow. Live microphone samples are buffered by the existing real-time callback, converted to mono PCM16 only after capture stops, and written with an explicit little-endian RIFF/WAVE header. Recording can run independently or alongside video encoding and continues to honor --record-gain.
Increment 7M ports ACMX2's remaining audio-runtime controls. Q selects amplitude-driven shader time, Home applies frame-delta scaling to reactive time and amplitude, and End applies the current audio sensitivity to FFT and FFT-history samples. ACMX2 MIDI Map actions for all three controls use the same event path as the keyboard.
Increment 7N ports ACMX2's Z PNG snapshot workflow and -e/--prefix destination. A snapshot captures the final processed swapchain image, including the configured shader passes and output flip. ACMX2 MIDI Map action 90 now dispatches the same Z request. MXVK readback is enabled for one frame and then disabled again when recording or periodic PNG generation is not active. A four-job background queue keeps PNG compression and disk I/O off the render thread and rejects excess requests instead of allowing unbounded frame-memory growth.
Increment 7O ports ACMX2's recording overlays through MXVK's Vulkan text renderer. --display-filter identifies the active shader, playlist node, multipass stack, and CUDA filter chain. --use-watermark and --use-watermark-color add configurable text immediately below that status, and E or ACMX2 MIDI Map action 69 toggles the watermark at runtime. The final MXVK compositing pass draws both overlays after the shader effect chain, keeping the text crisp and unmodified while preserving it in one-shot PNG snapshots, PNG sequences, periodic captures, and MXWrite video output.
Increment 7P ports ACMX2's runtime asset-root workflow. -p/--path, ACMXVK_PATH, and the compatible ACMX2_PATH fallback now feed one ordered resource resolver for fonts, internal shaders, default shader libraries, playlists, and MIDI examples. ACMXVK_SHADER_PATH provides a SPIR-V-specific shader default without accidentally consuming ACMX2's OpenGL shader tree. CMake mirrors the installed data/ and shaders/ layout in the build tree.
Increment 7Q ports ACMX2's media-timeline synchronization and extends it to live microphone muxing. Recording starts on the first valid source frame. Video-file frames retain their nominal source PTS even when MXWrite's bounded queue drops work, while audible file audio and muxed live input use their hardware sample counters as the master clock. Late file-video frames are decoded and discarded through MXVK without RGBA conversion, CUDA transfer, or Vulkan upload. Readback is requested only when a new recording frame is due.
Increment 7R hardens every ACMXVK text-input boundary with the shared input_validation module. It covers command-line arguments, relevant environment variables, shader JSON/index entries, shader playlists, M3U audio playlists, MIDI map files, custom-uniform and encoder expressions, device labels, and overlay text. Bounded readers prevent oversized configuration lines from allocating without limit, while count, numeric, decoded-image, live MIDI-message, and SPIR-V checks protect the corresponding non-string inputs.
Increment 7S ports ACMX2's default runtime HUD. It shows the active shader, elapsed time, measured presentation FPS, the current file-audio track, active CUDA filter, and autopilot status. F9 toggles the HUD and --disable-counter starts with it hidden. MXVK composites this preview-only queue after frame readback, so it never appears in MXWrite video, PNG output, one-shot snapshots, or F10 captures. Explicit --display-filter and watermark text continue to be drawn before readback and remain part of saved output.
For video-file input the HUD now separates media position from processing time. Video: HH:MM:SS / HH:MM:SS advances from the decoded source-frame position and reports the container duration, while Elapsed: HH:MM:SS follows the real processing clock. A high-resolution effect can therefore run slower than real time without making the media-position counter run ahead. Camera and still-image modes show only the elapsed processing timer. Repeating a video resets the displayed source position after each in-place seek while recording PTS remain continuous.
--use-source-fps turns that source timeline into a playback clock. ACMXVK waits before decoding an early frame and uses MXVK's decode-only skip path to catch up when rendering is late, avoiding unnecessary RGBA conversion and GPU upload for discarded frames. Pause and rendering-freeze time are subtracted from the playback clock. Without this option, video input retains ACMXVK's offline behavior and processes frames as quickly as the machine permits.
Increment 7T aligns camera setup with ACMX2 by applying a one-frame capture buffer, requested dimensions, MJPG (or --use-yuv YUYV), and the requested FPS in a stable negotiation order. Startup output reports the dimensions, nominal FPS, and FourCC read back from the driver and warns when they differ from the request. The preview HUD labels render throughput separately and adds a camera rate measured from successfully delivered frames. Significant delivery-rate changes are also logged, making low-light automatic-exposure reductions such as 60-to-30 FPS visible without mistaking the driver's nominal rate for actual throughput.
Increment 9R adds native camera capability discovery without opening a Vulkan window. Linux uses V4L2 enumeration, macOS uses AVFoundation, and Windows uses DirectShow, including each device's formats, discrete resolutions, and supported frame-rate values or range endpoints. The output retains ACMX2's interface-compatible format.
--probe-camera 0 is an alias for --enumerate-device 0. A missing device, permission failure, or device with no capture formats produces a diagnostic and a nonzero exit status. When the Qt interface is using the ACMXVK backend, its camera names and resolution/FPS choices come from these native probes. Selecting ACMX2 retains that backend's existing platform-specific discovery.
Increment 7U adds --maximize-fps for camera input. It requires --fps and runs camera acquisition on a bounded latest-frame worker while pacing the Vulkan loop at the requested rate. If a camera supplies 30 FPS and the target is 60 FPS, camera textures update when new frames arrive while shader time, custom uniforms, overlays, and presentation continue at 60 FPS. Only one unconsumed camera frame is retained, preventing latency and memory growth when capture is faster than rendering. Recording follows the render clock in this mode, so animated shader output can be encoded at the requested rate even when adjacent frames use the same camera image. VSync and hardware load can still cap the achieved presentation rate.
Increment 7V removes MXVK's same-frame recording fence wait. Each frame in flight now has its own persistently mapped Vulkan readback buffer; ACMXVK consumes a completed buffer when that slot's normal fence is reached and keeps the original snapshot intent and recording PTS attached to the delayed frame. Pending buffers are flushed before MXWrite closes, so the final submitted frames are not lost.
Increment 7W makes the pipelined path practical on discrete GPUs. MXVK now prefers host-cached coherent memory for readback instead of accepting the first host-visible type, which can be an uncached PCIe mapping on NVIDIA hardware. The repository MXWrite also queues host RGBA frames immediately and performs conversion plus hardware-frame upload on its encoder thread. Devices without a host-cached coherent type retain the portable coherent-memory fallback.
Increment 7X ports ACMX2-style compute image effects into the ordered Vulkan pass chain. MXVK 0.29.0 reflects fragment versus compute entry points from SPIR-V, uses synchronized RGBA8 storage-image ping-pong targets, and preserves the existing binding-1 uniforms plus optional history and audio descriptors. Fragment-only commands and manifests remain compatible.
Increment 7Y makes source-sized rendering the default. After a video or camera is opened, ACMXVK uses the decoder or camera driver's negotiated dimensions to size the window before MXVK creates its swapchain. Clockwise and counterclockwise 90-degree input rotation swap those dimensions. A still image uses its decoded dimensions in the same way. Passing --resolution keeps the requested fixed render/output size; fullscreen presentation continues to use the active display extent without changing that output size.
Increment 7Z separates automatic output geometry from the physical preview window geometry. Sources that fit within 90 percent of the usable display keep their native preview dimensions. Larger 16:9, 9:16, square, and rotated inputs are uniformly scaled down and centered with a locked aspect ratio, preventing the window manager from stretching them when they exceed the desktop. Video and generated output still use the full source width and height. Increment 9N extends the same preview fitting to an explicit --resolution while retaining that requested render/output extent.
Increment 8A removes the remaining preview-resolution processing compromise. MXVK renders the scene and every ordered fragment/compute pass into source-sized offscreen images, reads the completed source-sized frame for encoding, then scales only the presentation copy into the window with letterboxing if necessary. Resizing or maximizing the preview therefore cannot change shader resolution or encoded geometry.
Increment 8B sizes both output overlays and the preview-only runtime HUD from the actual preview-window height. The 1/60 scale keeps long shader, multipass, timer, and FPS lines readable when a high-resolution or portrait source is fitted into a smaller window; the minimum remains 12 points.
Enable the MXVK model renderer with --enable-3d. Without --model, ACMXVK loads its bundled models/cube.obj. Supplying --model also enables 3D mode and accepts Wavefront OBJ, MXMOD, or compressed MXMOD (.mxmod.z) input:
MXVK normalizes the model from its bounds and ACMXVK uploads each decoded RGBA input frame into the model's primary Vulkan texture. The primary texture is explicitly selected for the complete draw, so models containing multiple submeshes receive the same live input across every surface. CUDA-enabled MXVK builds use direct CUDA/Vulkan model-texture interop for NVDEC and filtered frames, with host staging as a reported fallback.
For a compatible single fragment shader, ACMXVK installs that shader directly on MXVK's model pipeline. The shader samples the live source at the mesh UVs, which matches ACMX2's 3D path and avoids applying the effect as a flat screen-space filter after rasterization. Compute shaders, history/spectrum descriptor shaders, and active multipass/playlist chains use MXVK 0.33.0's pre-model offscreen chain. Its completed image is then sampled at the model UVs. Output overlays, snapshots, and encoded video still operate on the completed 3D image. The preview-only HUD remains excluded from saved output.
The initial view is a skybox-style camera at the normalized model center. It uses ACMX2's yaw 270 degrees, pitch 0, distance 0, and 120-degree field of view. Automatic view rotation is disabled at startup.
The main 3D controls are:
ACMX2 MIDI-map action codes 44, 46, 51, 67, 79, 86, 88, 91, and 93 drive the same controls. Model paths are centrally validated, restricted to the supported extensions, and limited to 1 GiB before reaching MXVK's loader.
See Controls.md for the complete keyboard, mouse, 3D, and MIDI control reference.
Run acmxvk --help for the complete command-line reference.
Development builds are tested with Vulkan SDK 1.4 and with validation enabled in both MXVK and ACMXVK. The current increment has been exercised with shader-library loading, multipass rendering, configurable history caches, MXWrite encoding, custom-uniform rendering, optional live audio metrics, FFmpeg-decoded file reactivity, routed-tone FFT visualization, and FFT spectrum history. Increments 7B through 7K were additionally tested with a CUDA+MIDI build, live Left/Right filter changes, filtered Vulkan frame history, resident GpuMat video input, and CUDA-resident clockwise, 180-degree, and counterclockwise rotation on an NVIDIA RTX 2070. Increment 7F was tested with H.264 NVDEC feeding CUDA rotation, acidcam-gpu, Vulkan history, and repeated playback without a host-frame handoff. Increment 7G was also tested with the acidcam-gpu filter omitted while retaining direct NVDEC rotation, Vulkan sprite upload, and layered history. Increment 7H verified the same resident route in a -DWITH_CUDA=OFF build and separately regression-tested the optional filtered build. Increment 7I verified explicit device 0 selection in both configurations, including decode=cuda:0 from MXVK and a clean direct history path. Increment 7J looped an 84-frame H.264 source into a 186-frame output while retaining one NVDEC decoder open, direct rotation/upload/history, and clean validation. The Increment 7K repeated the same NVDEC source through the event-protected surface handoff with and without acidcam-gpu filters. The known duplicate vkBasalt implicit-layer warning is external to ACMXVK. Increment 7L was tested by writing live microphone input to a standalone PCM16 WAV, then writing the WAV alongside an H.264/AAC output from the same capture buffer. Both files' channel, sample-rate, duration, codec, and sample-format metadata were inspected. Increment 7M exercised all three audio controls through SDL keyboard events against repeating file audio and verified ACMX2 MIDI Map actions 81, 268, and 269 as active mappings under Vulkan validation. Increment 7N captured and inspected 1280x720 RGBA PNGs with both one-shot and continuous readback, and verified MIDI Map action 90 as an active snapshot mapping. A 3840x2091 stress capture then processed another keyboard event between the snapshot's queue and completion messages, and an immediate-exit test drained the pending PNG before Vulkan teardown. Increment 7O rendered and inspected watermark-on and watermark-off 1920x1080 processed snapshots, then inspected a 640x360 H.264 MXWrite output frame with both labels present. A second CUDA-filter encode confirmed the SquareBlockResize [3] label from the live acidcam-gpu filter state. All runs completed under Vulkan validation without project validation errors. Increment 7P loaded its font, sprite shader, manifest, and selected effect exclusively from an isolated --path tree, encoded the result, then repeated default discovery through ACMXVK_PATH. Increment 7Q was compiled in CUDA, audio, and MIDI mode and again with ACMXVK audio, MIDI, and acidcam-gpu filters disabled. Both configurations linked against a staged MXVK 0.26.0; their command-line and CUDA capability smoke checks completed successfully. Increment 7R passed its malformed UTF-8, control-character, identifier, structured-value, URL, bounded-line, and UTF-8 truncation regression suite in both configurations. Additional CLI probes rejected control characters, disallowed encoder punctuation, and oversized output dimensions before Vulkan initialization. Increment 7S built against staged CUDA and non-CUDA MXVK 0.27.0 packages. Under Vulkan validation, two deterministic 30-frame H.264 recordings with the HUD shown and hidden produced identical decoded-frame SHA-256 hashes; an otherwise identical watermark recording produced a different hash, proving that preview status is excluded while explicit saved overlays remain embedded. Increment 7V built with CUDA, audio, MIDI, and validation against a staged MXVK 0.28.0 package. A CUDA/NVDEC input and NVENC output run completed without validation errors, and its destructor flush produced all 30 expected frames in the one-second H.264 test clip. MXVK and ACMXVK also compiled cleanly in the non-CUDA configuration used by portable and Apple builds. Increment 7W then repeated a 1920x1080 NVDEC/NVENC recording driven by live-audio PTS. Selecting the RTX 2070's host-cached memory type increased delivery from 6–7 frames to 59 frames over two seconds; the resulting H.264 and AAC streams both measured exactly two seconds, with a 29.5 FPS average video rate. Increment 8I loaded both the bundled OBJ and MXVK's compressed cube.mxmod.z, mapped a 1920x1080 still image over the complete mesh, and encoded the source-sized 3D render through the existing Vulkan post-process and pipelined readback path. A separate bundled-OBJ run completed with the Vulkan SDK validation layer enabled and no project validation errors; the duplicate vkBasalt implicit-layer warning remained external to ACMXVK. Increment 8J then verified the centered skybox transform and direct model-UV fragment pipeline in the portable build; the active fragment was removed from the later screen-space chain so it was evaluated exactly once. Increment 8K validated standalone compute selection and a mixed compute plus selected-fragment pass chain against MXVK 0.33.0. Both produced encoded cube interiors with the processed image following the cube faces, and Vulkan validation reported no project errors. CUDA and non-CUDA builds and the input validation test also passed. Increment 8M compiled all 35 crossfade modules with glslc, built against a staged MXVK 0.33.1, and triggered a fragment-to-compute transition during video playback. The previous source-sized Vulkan result was captured, bound as the transition texture, blended for the configured duration, and released without Vulkan validation errors. The portable input-validation test also passed. Increment 8N compiled the wave-enabled model vertex shader and complete portable and CUDA/audio/MIDI configurations against MXVK 0.33.1. The input-validation test passed; the 3D model path was exercised with the wave toggled through C under Vulkan validation. Increment 8O built in both portable and CUDA/audio/MIDI configurations. Its input-validation test passed, and the 3D model path was exercised with O oscillation and C wave deformation active together under Vulkan validation. Increment 8P built in portable and CUDA/audio/MIDI configurations. Runtime keyboard injection verified that brackets change only XFade, shifted plus/minus change only model scale, plain plus/minus retain camera zoom, and MIDI actions 91/93 use the isolated model-scale path. Increment 8R built in both portable and audio-enabled configurations, including the optional WebP feature. The input-validation test passed in each build, and the command-line smoke test accepted and reported --mute-output. Increment 8S built with TIFF/WebP/audio enabled and in a portable build with both optional image formats disabled. The input-validation test passed in both configurations; the enabled executable linked libtiff, while the disabled executable had no direct libtiff dependency. Increment 8T built in the full CUDA/audio/MIDI/validation configuration and in a portable audio-disabled configuration. Both input-validation tests passed. Runtime camera recordings now announce when real-time PTS preservation becomes active so reduced 4K delivery rates can be verified from the log and resulting media duration. Increment 8U built in full-featured and portable configurations. The raw RGBA8 writer uses the shared snapshot worker without an optional image library, and the input-validation test passes in both configurations. The ACMX interface integration now accepts live multipass enable/disable and ordered pass-name updates over its synchronized shared-memory channel. Each requested fragment or compute pass is resolved against the active compiled library before the Vulkan pipeline is rebuilt. The same channel now accepts live playback Repeat and Normalized Time state. Changing either action in the interface updates a running ACMXVK process without restarting it; Repeat is consulted at video end-of-stream, while Normalized Time switches non-video shader timing between wall-clock and one fixed step per output frame. Display Filter and Watermark settings now use the live channel as well. Watermark enable state, validated UTF-8 text, and RGB color update together; enabling a watermark at runtime hides the HUD by default just as it does at startup. Interface-controlled runs initialize overlay font resources even when all overlays begin disabled, so enabling one later is safe. CUDA-enabled builds now accept live GPU filter-chain and temporal-buffer updates from the interface. A replacement engine is fully validated and constructed before the active engine is changed; invalid requests leave the current chain intact. Still-image mode immediately reprocesses its resident source after a successful update. Running file-audio sessions can now switch tracks or M3U playlists from the interface. ACMXVK decodes and configures the requested replacement before stopping the current source, so invalid paths or undecodable media leave the active audio intact. Repeat, stop-at-audio-EOF, output device, and pass-through state are applied with the replacement. ACMXVK validates the interface shared-memory object size with fstat before mapping it. Errors now identify the failing shm_open, fstat, or mmap operation and include the operating-system message. Saving an active-library .frag or .comp source in the ACMX editor now compiles only that file with glslc. Compilation targets a unique temporary file; the interface validates its SPIR-V magic and atomically replaces the matching .acmxvk-build module only after success. ACMXVK validates that the reload path belongs to its active runtime manifest, refreshes active fragment, compute, multipass, playlist, or direct 3D pipelines, and leaves inactive modules ready for their next selection. Failed compilation keeps the running pipeline and previous cached module unchanged. The interface log retains and prints the compiler's complete failure diagnostic. Its ACMXVK Properties dialog can resolve glslc automatically or select a custom glslc-compatible executable; that selection is also passed to full Build and Fix Build runs.
I have been using the Codex CLI from OpenAI as an engineering aid while porting ACMX2 to MXVK. Codex has assisted with incremental code translation, CMake integration, shader conversion, debugging, Vulkan validation testing, and documentation. Project direction, testing decisions, and maintenance remain under the project owner's control.
1.18.0