Home > Blog > Making glitch patterns using only ffmpeg

Making glitch patterns using only ffmpeg



This post is inspired by Rob Mac's submission for the Blue \x80 glitch art show that took place in Paris @ Villette Makerz in Octobre 2018. His artwork featured an ffmpeg script that generates a continuous flow of glitch art using only ffmpeg arguments.

Rob Mac's art installation running at Blue \x80
# Rob Mac's code
ffmpeg -f lavfi -i cellauto=s=640x360:rule=15:r=50 \
-vf scale=1280x720 -bsf noise=1000 -r 30 -f mpegts \
-c:v h264 -crf 31 -preset ultrafast -tune zerolatency - | \
ffplay -i - -loglevel quiet -fs -vf tblend=all_mode=darken

Aside from the aesthetic and optimization fixes, his code features two important video filters. A video signal generator and a bitstream noise function

  • cellauto=s=640x360:rule=15:r=50
  • -bsf noise=1000

Video Generator

As experimental digital artists, finding a source is often challenging as their narratives can come in the way of the voice we wish to give to media and its quirks. But there exists a whole history of generative artist creating their own neutral content from code using what we call generators. Ffmpeg itself allows the use of complex generators such as Conway's Game of Life, the Mandelbrot fractal, the cellauto filter, and many others.

I won't be explaining here how cellular automaton's work here, so for more extensive knowledge on the cellauto filter you should check out the ffmpeg manual cellauto entry and the rule 110 article on wikipedia (and other rules too). What's interesting to us about the cellauto filter is that it outputs different results depending on which rule we give to it, giving us a varying output whenever we wish.

Let's make a really simple code that generates a small video for each of the cellautorules :

import os
rule = 0
while rule < 150:
	rule = rule + 1
	os.system(" \
ffmpeg  \
-f lavfi -i cellauto=s=1280x720:rule=" + str(rule) +":r=60 \
-f mpegts -t 30 \
-c:v libx264 \
" + str(rule) +".mp4 \
")

Yeayy we suddenly now have 150 video of 20 seconds that each have generated diagonal/horizontal lines, and even sometimes some weird shit like video 110.mp4

6.mp4
110.mp4

Noise

We have our source now, instead of glitching it all by hand we're going to delete everything and re-generate it using ffmpeg again but this time using the -bsf noise filter.

The noise filter in ffmpeg is documented here, but for short, bs stands for bitstream. Its purpose as that of testing stream resilience by purposefully forgetting tiny bits of data from the bitstream. The lower the value you give to it, the more often packets from the bitstream will be dropped. So let's add this to our code !

import os
rule = 0
while rule < 150:
	rule = rule + 1
	os.system(" \
ffmpeg  \
-f lavfi -i cellauto=s=1280x720:rule=" + str(rule) +":r=60 \
-bsf noise=3000 \
-f mpegts -t 30 \
-c:v libx264 \
vidz/" + str(rule) +".mp4 \
")

That's it, you should now have 150 videos of moving glitch textures

112.mp4
110.mp4

a e s t h e t i c s + o p t i m i z a t i o n

You may have noticed, these are all pretty grey and if you use VLC to view your results, often you'll even get the software's signature green screen which some people don't find really aesthetic. So let's cook something up to make this more fun. What we're going to do next is :

  1. optimize the encoder settings to make it faster
  2. input and overlay a second video to grab some its colors/randomness in the mix
  3. make it more fluid by changing GOP size and bi predictive frame settings
  4. add a tblend filter at the end to stop the gray issue by subtracting consecutive frames

Using these options :

  1. -strict experimental -crf 31 -preset ultrafast -tune zerolatency
  2. -i random.avi -filter_complex…
  3. -g 9999 -bf 0
  4. -vf tblend=all_mode=darken
import os
rule = 0
while rule < 150:
	rule = rule + 1
	os.system(" \
ffmpeg  \
-f lavfi -i cellauto=s=1280x720:rule=" + str(rule) +":r=60 \
-i random.avi \
-filter_complex '\
[0:v]scale=1280x720,setsar=sar=1/1,format=rgba[cells];\
[1:v]scale=1280x720,setsar=sar=1/1,format=rgba[random];\
[cells][random]blend=all_mode='addition':all_opacity=0.3[out];\
[out]format=rgba' \
-g 9999 -bf 0 -bsf noise=5000 \
-f mpegts -c:v libx264 -strict experimental -crf 31 -preset ultrafast -tune zerolatency -shortest - | \
ffmpeg -i - -t 30 -vf tblend=all_mode=darken -c:v libx264 vidz/temp-" + str(rule) +".mp4\
")

And because I've been showing only images throughout this whole blogpost, here is a video compiling two seconds of every rule from 1 to 250 using a variation of the script above.

If you have any questions, notice any mistakes or want to hire me for research/art
feel free to send me an email me at kaspar.ravel[at]gmail[dot]com
for more visual art and less text you can follow my work on facebook or instagram
And whoever you are stranger, thank you for existing and being passionate about things.