# Devolution

By [Untitled](https://paragraph.com/@0x9c65ac91c38598d95099068c0634a5d3276de51c) · 2022-01-17

---

![Devolved Punks](https://storage.googleapis.com/papyrus_images/f3b749cc589f254367e309ef7f4297fa1eaf41a0dfb7b9ecbdb6e2bdd6e70e87.jpg)

Devolved Punks

When working with images in deep learning for generative art, you will almost always use the convolution operator to build networks. Convolution is a function that takes an input image and creates a scaled down, convoluted, representation of the image by passing a kernel over the image in steps called “strides”, multiplying each cell and adding the results to create a value in the output matrix.

![Convolution Example](https://storage.googleapis.com/papyrus_images/193d050e551603bfc475d7a5ad63f170b53d22afea4ac4195a561c92e72a312a.png)

Convolution Example

The opposite of convolution is called transposed convolution or as I like to say, devolution. In devolution, a model is trained that can create a new kernel that can be used to restore the input matrix from the convolved output matrix.

I’ve struggled to find simple examples of convolution and devolution that can be applied to arbitrary datasets; most tutorials use MNIST and gloss over the details of adjusting the network to use new input sizes. This post will walk through the simplest possible example by working the CPUNX-10k dataset.

Getting Started
---------------

The CPUNX-10k API can be installed into a virtual environment using pip.

    $ pip install cpunx-10k
    

After installation, you can work with the dataset in a notebook much like you may work with the MNIST dataset made available in Tensorflow.

    import cpunks10k
    cp = cpunks10k.cpunks10k()
    (X_train, Y_train), (X_test, Y_test), (labels) = cp.load_data()
    

Now, import the minimum set of Keras modules needed for this tutorial.

    import numpy as np
    import keras
    from keras.models import Sequential
    from keras.layers import Conv2D, Conv2DTranspose
    

The CPUNX-10k dataset is a set of 10,000 (24,24,4) images encoded in four RGBT channels. For this simple example, we want to work with simpler images, that is, greyscale encoded (24,24,1) images. The following code can be used to transform the images to greyscale.

    cmap = plt.get_cmap('gray')
    def rgb2gray(rgb):
        '''covert an rgb image to greyscale using weights found
           when fumbling around on stackoverflow
        '''
        weights = [0.2989, 0.5870, 0.1140]
        return np.dot(rgb[...,:3], weights)
    
    # convert X_train and X_test to greyscale
    X_train_g = np.array([rgb2gray(img) for img in X_train])
    X_test_g = np.array([rgb2gray(img) for img in X_test])
    X_train_g = X_train_g.reshape(9000,24,24,1)
    X_test_g = X_test_g.reshape(1000,24,24,1)
    
    # check work
    plt.imshow(X_train_g[0], cmap=cmap)
    plt.show()
    

![Grey Punk Number One](https://storage.googleapis.com/papyrus_images/66e96bc5e7b82cf52d9768e3ab875aea763709e916669a3ce01bfe4e94b4545e.png)

Grey Punk Number One

Now, we can build a simple Keras network that applies three convolution operators to create a convolved representation of the images and then uses the transposed convolution operator to reconstruct the image from the intermediate representation output by the convolution. In Keras, the Conv2d operator is used to apply `filters=N` different kernels to the input to create the convolved output representation.

![Conv2d Operator Kernels](https://storage.googleapis.com/papyrus_images/0d505f0f4d55eaf4cf21ddc9eb04c987f6521a3de4ad8b9acccbb991cbcbe184.png)

Conv2d Operator Kernels

The code below creates the model by adding 3 layers each of convolution and transposed convolution.

    input_shape = (24,24,1)
    model = Sequential()
    model.add(Conv2D(32, kernel_size=(3, 3), activation='relu', kernel_initializer='he_normal', input_shape=input_shape))
    model.add(Conv2D(16, kernel_size=(3, 3), activation='relu', kernel_initializer='he_normal'))
    model.add(Conv2D(8, kernel_size=(3, 3), activation='relu', kernel_initializer='he_normal'))
    model.add(Conv2DTranspose(8, kernel_size=(3,3), activation='relu', kernel_initializer='he_normal'))
    model.add(Conv2DTranspose(16, kernel_size=(3,3), activation='relu', kernel_initializer='he_normal'))
    model.add(Conv2DTranspose(32, kernel_size=(3,3), activation='relu', kernel_initializer='he_normal'))
    model.add(Conv2D(1, kernel_size=(3, 3), activation='sigmoid', padding='same'))
    

You can view the network architecture using the `summary` method.

    model.summary()
    

    Model: "sequential_1"
    _______________________________________________________________
    Layer (type)                 Output Shape              Param #   
    ===============================================================
    conv2d_34 (Conv2D)           (None, 22, 22, 32)        320       
    _______________________________________________________________
    conv2d_35 (Conv2D)           (None, 20, 20, 16)        4624      
    _______________________________________________________________
    conv2d_36 (Conv2D)           (None, 18, 18, 8)         1160      
    _______________________________________________________________
    conv2d_transpose_25 (Conv2DT (None, 20, 20, 8)         584       
    _______________________________________________________________
    conv2d_transpose_26 (Conv2DT (None, 22, 22, 16)        1168      
    _______________________________________________________________
    conv2d_transpose_27 (Conv2DT (None, 24, 24, 32)        4640      
    _______________________________________________________________
    conv2d_37 (Conv2D)           (None, 24, 24, 1)         289       
    ===============================================================
    Total params: 12,785
    Trainable params: 12,785
    Non-trainable params: 0
    

Now, compile and train the model

    model.compile(optimizer='adam', loss='binary_crossentropy')
    model.fit(X_train_g, X_train_g,
                    epochs=num_epochs,
                    batch_size=batch_size,
                    validation_split=validation_split)
    

You can use the model to “predict” a devolved punk using a punk in the test dataset as input. In the line below, we create the first 10 devolved punks from the test set.

    devolved = model.predict(X_test_g[0:10])
    

We can compare the devolved punks against the greyscale punks using a bit of matplotlib trickery.

    fig = plt.figure(figsize=(7, 3))
    n_rows = 2
    n_cols = 5
    for i in range(0,n_cols):
        ax = fig.add_subplot(n_rows, n_cols, i+1)
        ax.imshow(X_test_g[i], cmap=cmap)
    
        ax = fig.add_subplot(n_rows, n_cols, i+1+n_cols)
        ax.imshow(devolved[i], cmap=cmap)
    
    plt.subplots_adjust(bottom=-0.2, right=1.4, top=0.9)
    

![Grey Punks vs. Devolved Punks](https://storage.googleapis.com/papyrus_images/ef4e0b84ab5058a5b0915631dbf4580eaf7aae532db647e0b328820faa4ee24b.png)

Grey Punks vs. Devolved Punks

An understanding of the transformed convolution function is critical to building an intuition for the tools that we will use to make generative art e.g. autoencoders, variational autoencoders and generative adversarial networks. As an exercise, experiment with the network shape, the sizes of the convolution operators and the number of epochs used in training to create new sets of punks that use the imperfection of the transposed convolution operator to create aesthetically interesting variants.

---

*Originally published on [Untitled](https://paragraph.com/@0x9c65ac91c38598d95099068c0634a5d3276de51c/devolution)*