ACMX2 explained in plain English

The app reads command line options to learn where the assets live, which shader or shader library to use, whether to use a camera or a file, what resolution and frame rate to target, whether to go fullscreen, whether to loop a file, whether to enable a four frame texture cache, and whether to run a simple 2D quad or a 3D model. If a library is requested the program reads an index file that lists many fragment shaders and compiles them all so you can switch at runtime. If you point to one fragment file the program compiles just that one.

If a camera is requested the program opens it and negotiates a resolution and frame rate. If you pass a video filename the program opens the file and learns its width, height, frame count, and fps. The window is sized to the content so the shader uniform called iResolution always matches the actual drawing surface.

FrameCache remembers a fixed number of recent frames. When the texture cache feature is turned on the program stores up to four frames here. Shaders whose names contain the word cache get four extra texture uniforms named samp1 to samp4 so a shader can read the recent past and blend time.

ShaderLibrary owns the compiled OpenGL programs and keeps track of uniform locations. It sets up iResolution and then updates alpha, iTime, time_f, and iMouse every frame. It can compile a single shader or load a whole list from a folder. It remembers which shader is active and you can move forward or backward through the list with the arrow keys. Time can advance automatically from the system clock or be driven by audio amplitude when audio is enabled. If time is paused you can nudge it forward or backward in small steps.

MXArguments holds the choices you pass on the command line. ACView does the work every frame. It owns capture, drawing helpers, an optional 3D model, and a writer for export. It also owns an offscreen framebuffer so it can render the shader output into a texture and then present it. MainWindow sets the icon, constructs ACView with your options, calls load, and loops drawing frames until you quit.

ACView compiles your shader or your library through ShaderLibrary and selects the initial one. If 3D mode is enabled it opens a model file and tells the model to use the same shader. It also compiles a small postpass shader that draws whatever the offscreen pass produced.

The program opens a camera or a video file. For a camera it requests either a specific camera size or falls back to the window size. For a video file it reads width, height, and fps from the file. If you ask for a stretched output resolution the app resizes the OpenGL window to match. If you provide an output filename the writer opens and prepares to write frames at the chosen bitrate and fps.

If the texture cache is enabled the app allocates four OpenGL textures and fills them with blank images. The frame cache is also prefilled so a shader that expects historical textures will not read uninitialized data. The program then creates a texture that will always hold the latest camera or file frame, prepares a full screen sprite, and creates an offscreen framebuffer plus a texture that will hold the rendered result. Titles and fullscreen are set as requested. Background writer and capture threads start when needed.

Each frame the program tries to get a fresh image. For a camera this comes from a queue fed by a background thread. For a file it is read on the spot. The image is flipped vertically so texture coordinates match the shader. If a new frame exists it updates the OpenGL texture named camera_texture. If the texture cache is active and the current shader name contains the word cache the program occasionally pushes the new frame into the frame cache and updates the four cache textures as a set so your fragment code can read multiple time slices.

The program binds the offscreen framebuffer and clears it. It selects the current shader and updates uniforms for time, mouse position, mouse click start, and resolution. Time can come from the system clock or from audio amplitude when audio is enabled. Mouse is reported as a four component vector where the first two components are the current cursor position and the last two are the point where a drag began.

In 3D mode the program sets up a view and projection matrix, computes a camera direction based on keys, and draws the model with the live video bound to a sampler named samp. In 2D mode the program draws the sprite that covers the window with the live video. Either way the fragment shader runs on what is drawn.

If recording is active or if you pressed the snapshot key the program reads the offscreen texture back into system memory, flips rows so the image is right side up, queues the pixel block for the writer thread, then draws the offscreen result to the window. The window title is updated with progress and the code sleeps a little in file mode to approximate the original fps.

The capture thread grabs frames from the camera and pushes them into a queue so decoding does not block rendering. Each frame is flipped to match the OpenGL coordinate system. The draw loop pops frames from that queue so the camera cadence and render cadence can be independent.

The writer thread waits for frames. For normal recording it encodes them into the video file. For snapshots it builds a filename with a timestamp and an incrementing counter and writes a PNG file. When you stop the program the thread finishes outstanding work, closes the writer, reports how many seconds were written, and can copy the original audio track from the input file into the new output file when requested.

iResolution matches the current window size. iTime is seconds since start. time_f is a second time source that can be paused, stepped, or driven by audio. iMouse carries the current cursor position and the click start position so you can make effects that depend on dragging. The main texture sampler is named samp. Some shaders can also receive samp1 to samp4 when the name contains cache and the texture cache feature is on.

In 2D mode the app draws a single rectangle the size of the window and runs your shader on it. The rectangle is textured with the latest camera or file frame. The shader can use that image, the time values, and the mouse vector to create any look you want. In 3D mode the app loads a model and runs the same shader while the model spins or the view rotates. Your shader can use mv_matrix and proj_matrix to do model space work or just treat the incoming texture the same way as in 2D.

If any OpenGL call reports an error the code throws an exception with a readable message. If a camera or video file cannot be opened the code stops with a clear message. If the framebuffer fails to assemble the program refuses to continue. If the shader fails to compile you see it in the log with the file name that failed. Risky sections are wrapped so a friendly message is printed and the app exits cleanly.

You can print help and version information. You can point to the asset path where data files live. You can choose a window resolution and a camera resolution. You can pick a camera index or pass a video file path. You can choose a single fragment file or a shader library folder with an index file. You can pick the initial shader index when using a library. You can choose where snapshots will be saved. You can set the output filename to record to a video. You can set bitrate and frames per second. You can loop a video file when it reaches the end. You can request fullscreen. You can enable the texture cache and choose how many frames to wait before each cache update. You can request that the audio track from an input file be copied to the output after encoding. You can turn on a 3D mode and choose a different model file. When audio support is compiled in you can enable audio reactivity, set the channel count, and set sensitivity.

Think of ACMX2 as a live visual synthesizer. The camera or a movie gives it raw pixels. Your fragment shader transforms those pixels into art. The app keeps the GPU fed with textures and uniforms, keeps the window updated, and writes the final result to disk when you ask for it. The arrow keys and mouse let you explore. Optional features let the image respond to audio or be mapped onto a 3D object. Across every part the goal is the same. Keep the frame rate smooth by moving slow work into threads, keep the shader input consistent, and give artists straightforward controls that do not require programming knowledge.

A shader is a short program that runs on the graphics card for every pixel. The rest of ACMX2 exists to keep the shader fed with fresh information and to carry away the finished picture. The terms framebuffer and texture refer to GPU objects that hold images. Uniforms are named variables the app updates every frame so the shader can know things like time and mouse position. Threads are background workers that keep input and output from slowing down the picture.