In the past few years I’ve repeatedly revisited 2D Gabor Wavelets as an approach to generative image creation, culminating in several NFT projects like Wavelet Pools, Wavelet Pools s02, and Wavelet Flats, along with several works minted on HEN. I’m not totally sure why this concept/technique has captivated me so much, especially given how mathematical it is. I’ve been meaning to write this for some time now, in hopes that someone else might get the urge to explore alongside me.

How I Got Here: Wavelet Origin Story

I was introduced to Gabor Wavelets/Filtering in the Fall of 2020, while beginning a collaborative project with Weirdcore to create visual materials for the release of Arca’s AI-driven “Riquiqui Bronze Instances.” The album used Bronze’s AI-driven remixing software Bronze Composer (most famously used by Jai Paul) to generate 100 remixes of the song “Riquiqui.”

For the visuals on this project we were inspired by a Shadertoy demo by Matt Zucker of Gabor Wavelets trained by machine learning to approximate an image. I dug into the shader and tensorflow code on his github repo, and ended up spending a lot of time researching Gabor Filtering and the concept of wavelets in both image and audio processing. This was also my first time using machine learning properly, and ended up having to rewrite a lot of the python code in order to get it running on Google Collab with recent versions of tensorflow. I reached out to Mr. Zucker via Twitter while working on this project, but he was rather uninterested in my curiosity and had moved on from this exploration.

I really enjoyed working with machine learning in this way, to train the parameters of an arbitrary image synthesis algorithm to approximate an image. It was as if this collection of little fuzzy bits were slowly revealing the representation of an image. It was never quite perfect, but the artifacts around the edges were luscious colored gradients that bled off of the image subject. I was hooked.

https://x.com/XLRECORDINGS/status/1340001384674906112?s=20

The Pools

I’m very captivated by overlaps between sound and visual processes, and enjoy the idea of synthesizing a complex image with abstract and idealized elements like sine waves. I also enjoy processes that somehow attempt to represent reality but fall short in some beautiful way that can be further exploited. Wavelets are often used in image compression, which is a fascinating but pretty opaque process, and I’ve always wished I could go inside of the JPEG algorithm and paint with just the artifacts of the process. This is sort of like that, in a way. After the Arca project, which was still driven primarily by representation, I was curious to explore this process in a purely abstract setting, to just revel in the look of the overlapping gradients.

This led to the Wavelet Pools series, which takes the synthesized image and applies it to a deformed, animated mesh. The original Wavelet Pools were pre-rendered GIFs listed on the Opensea shared contract, and I’d long meant to revisit the project as a generative series instead. This required quite a bit of porting of code to work with WebGL and took over a year to slowly come together as Wavelet Pools s02.

https://highlight.xyz/mint/656e531b15f119b945152e11

I was vaguely surprised to see that the original Wavelet Pools series resonated for so many people, as a kind of minimal color study with a strange and lively pool of liquid in the middle of it. It’s very different from what I think of as my practice, but the investigation has taken on a life of its own.

While preparing to write this text, I wanted to come up with a flat representation of the Gabor Wavelets that are generated behind the scenes of Wavelet Pools. As I put that together, I started to really appreciate these flat images in their own right as unique artworks, so I followed that path to Wavelet Flats, an open edition of generative wave patterns. There is something so hypnotic, slow-moving, and tender to the way these look that I’ve come to see them as expressive in a very different way.

https://highlight.xyz/mint/65cc8bc107cf244c41bb0c81

An Intro to Wavelets

The idea behind Gabor Filtering is similar in many ways to Fourier transformation - you are breaking down a more complex data set into component sinusoidal waveforms at different frequencies and phases. Fourier transformation is still the most popular way to represent and manipulate audio signals in the frequency domain, but wavelet transformations are also a subject of much interest in the more academic signal processing field. With wavelets, instead of representing the whole signal with a bank of constant sine waves, the individual waves are localized by a gaussian window or in other words, feathered edges. In 2D wavelets, you have a number of parameters to work with - frequency, phase, amplitude, size (x/y), and rotation. To synthesize an image with them, you combine some number (usually quite a few) of individual wavelets with different settings additively across the canvas.

Gabor filtering is often used in Computer Vision and image-based Machine Learning because it is quite good for recognizing features and representing an abstraction of an image, using very little data. A good explanation of this usage can be found here.

Part of the reason this interests me is that very often certain techniques and concepts become so embedded within a particular practice or workflow that they become invisible as image processes. I like finding things like this, that have a defined role to play in a larger technological context, and find other ways to utilize them. Sometimes very interesting ideas are short-changed by their common, accepted usage, and changing the context for a technique opens up a whole new pathway for exploration.

https://highlight.xyz/mint/656e531b15f119b945152e11

Making Pictures with These Little Waves

This is where we get a bit more technical.

In the original Shadertoy demo, you’ll see a single fragment shader that has hardcoded (because it’s Shadertoy) a whole set of wavelets with specific values that are all added together into a single output image that resembles a photo of Zsa Zsa Gabor. If you’re making a more general shader, for WebGL or other environments, you could create a similar number of uniform variables to pass in with whatever settings you like rather than hard code all of them. My personal preference however is to render each wavelet as a quad with a custom vertex/fragment shader using a floating point framebuffer to accumulate all of them. The resulting shader is much more manageable as a result, and it becomes a lot easier in something like Javascript/WebGL to loop through these individual elements and render them. As you can see in the fragment program below, this makes for a pretty tidy little shader that just focusses on rendering one wavelet. You might notice that most of the parameters are actually passed as varying variables from the vertex shader, which in turn is importing them as attributes.

precision highp float;
varying vec2 tc;
varying float sc;
varying float bd;
varying vec3 ap;
varying vec3 ph;
uniform float phaseoff;
uniform float adjust_shape;
const float two_pi = 6.283185307179586;
void main()
{
    float f = two_pi/bd;
    vec3 wave = cos(f*sc*tc.x+ph+vec3(phaseoff));
    vec2 p = tc;
    p = p*adjust_shape;
    float w = exp(dot(vec2(-0.5), p*p));
    gl_FragColor = vec4(wave*w*ap,w*dot(ap,vec3(1.)));
}

Using attributes to pass these values in allows me to generate GL buffers to represent them when creating the geometry during the initialization phase and bypass having to pass in a lot of uniforms at every frame. The only regularly updating uniform here is “phaseoff” which is how I do animation of the wavelets and create the color/wave cycling motion used in my projects.

https://opensea.io/assets/ethereum/0xD3028A6304c4099627078d60A25dF88f3FF6951E/8

Mathematical Images

I’ll be the first to admit that this kind of highly mathematical approach to image making is not really my style in a lot of ways. I don’t typically have a lot of time for beautiful algorithms, voronoi patterns, or sacred geometry. I felt fairly inundated by M.C. Escher drawings by math teachers who wanted to connect with my creative curiosity, to the point where I developed a strong distaste early on for a certain kind of image that seems to say “math is cool too!” To my dismay, I have a deep innate love and curiosity for digital image processes,video art, and synthesis and was sort of tricked into learning to program and learning a lot of complex math in the process of scratching those itches. I’m usually much more interested in where things get messier and more expressive, using gestural inputs, feedback processes, texture deformations, geometry glitches and simulation to create unpredictable results. For this reason, it’s strange and slightly unsettling to me that the wavelets continue to pull me in. It might be partially just an effect of the hypnotic movement of wave patterns colliding with one another, or me settling down to a more contemplative life as I get older. Maybe I’ve discovered that math is cool too in my own stubborn way. I feel like there is something newer and stranger that I will discover down this path. Maybe it just needs more noise for me to feel okay with it.