Over 200 subscribers
The following text is an edited transcript of my presentation at the 30th Anniversary International Light Symposium, held by Kepes Society on 14-15 September, 2023. The talk examines the possibilities of algorithmic thinking and generative methods in artistic systems through epistemological perspectives. In addition to visual elements, dynamic simulations and self-organizing algorithms, emphasis is placed on temporality, through introducing various auditory, psychoacoustic, and sonification directives.
The title “Audible Code” refers to different modalities of artistic expressions, with a focus on the concepts of space, resonance and memory as mediated components within post-digital culture. I incorporate these fields into my artistic practice, where I conduct various types of research with different institutions and forums. As a starting point, let’s elaborate a bit on the concept called Deep Listening, that was originally invented by Pauline Oliveros. In a wider context, the word “listening” itself has multiple meanings, thus it is also interesting to interpret this concapt as deep attention or deep awareness. The approach of this activity does less with the concepts of music listening or understanding and reconstructing musical content, instead it turns much more towards the nature of sound as a physiological phenomenon, which results from the combination of physiological, communicative, and — in some cases — linguistic expressions. The term “listening” here refers to the attempt to focus one’s attention toward audible resources, through a special presence of agency with an increased awareness to the environment. Speaking of the second term, Code: as it is widely known, nearly every aspect of our activities today can be represented through the language of code, since much of our mediated presence is now digital. The structure of digital information requires algorithmic processes, which we usually refer to as
In relation to sound and code, one interesting practice I have experimented with is called “live coding”, which is a way of interacting with a system in a direct way, by manipulating symbols and executing commands on the fly, while the programs are running. This approach emerged as computers became faster over the past few decades. As opposed to the beginning of computation, recently real-time computation became crucial, meaning a programmer does not need to calculate a composition or any structure in advance, but rather, the results can be calculated in real time, during execution. Our machines are fast enough that they can alter and modify the parameters of a program, and render the result immediately, during the process — this immediate nature is the essence of the live coding paradigm. When we write the code, it is compiled in real time and the artwork itself is created as we write it. This is a very interesting and dynamic form of coding, especially in the field of performative arts and musical compositions. In art, live coding is primarily used for performing music, visuals and similar aesthetic activities.
In relation to my work, I write code many times that is generating audible outputs like noise, oscillations and other patterns of soundwaves, using basic building blocks of digital audio. What usually characterizes my work, is that these sounds are directly triggered by functions, events and conditions that are part of the code itself. We can identify these procedures as forms of sonification, where the role of sounds can take different semantic functions such as symbolic signals, alarms, notifications, information feedback or other properties. As I mentioned, sonification is not a musical concept, but is more like making inaudible things audible, built on the physiology of sound, where data is transformed into vibrating frequencies. On the illustration above, you can see diagrams of an auditory display: usually such environments refer to systems where communication is conveyed through sound rather than through visual displays like screens. This approach has many applications, from parking sensors (built in radars in modern cars for navigation) to keyboards, radioactivity measuring signal systems and various user interfaces in many other fields. An interesting example of sonification is the Morse code system, which encodes the alphabet into sound sequences, or more precisely, into rhythmic patterns. In the system of the Morse code, the pitch of the sound does not hold any information; communication is carried through rhythmic patterns, transmitted via electrical signals over time.
Speaking of repetition and patterns, that’s exactly what rhythm is made of: cycles and loops. Computation also works with loops and sequences, where internal cycles are repeating the numerical processes. Rhythm is very important. Rhythm is resonance, that can be found in the vibrations of sounds, lights and other waves of the entire spectral world. We navigate this world through experiencing varying lengths of waves as resonating structures. In the case of sound, these waves are relatively slow, while in the case of light, these vibrations are much faster. There is a very interesting, extreme example of high speed activities which actually inspired some of my earlier works. This phenomena is called high-frequency trading, which is a set of very strange computational activities. As it is widely known, on the stock exchange, it’s becoming less common for humans to initiate trading decisions directly, instead tiny robot software are used for these occasions, who make their trading decisions in nanoseconds. High-frequency trading is an example of how the world has sped up. No human could make decisions with such short temporal conditions, and there are many interesting characteristics that can be found within these systems, such as flash crashes, the emergence of "co-locations" or "electronic front-runnings" to name just a few. Originally, the cycles of computation started very slow, but today these tools can make decisions in real time, combined with predictions of the future, based on historical data. High-frequency trading represents a very unusual and bizarre aspect of this approach. In the following, I will introduce three of my works that relate to the concepts of time, space and memory in the context of computation, in one way or the other. The first work is an installation called Echo, which was exhibited at the Kunsthalle Budapest in 2016. It is based on the idea of sonificating the activities made on Wikipedia articles. In Echo, a custom software robot is monitoring the edits which are made on the encyclopedia. It is making tiny notifications when these edits are made from governmental IP addresses. When a new edit happens, the system sends a message to a social platform, which both publishes this activity on its feed and also triggers the installation to emit a sound. The installation is showing how active these edits are. An important aspect of the work is that it is focusing on governmental IP addresses. At the time of its making, the MMA (Hungarian Academy of Arts) took over the control and management of the Kunsthalle, which was received as a highly divisive step of the government from the position of the broader contemporary art community. As an artist, I felt it was important to highlight the worsening underlying changes within the Hungarian cultural field, so this installation acted as a kind of mirror turned at the Kunsthalle and at governmental systems in general.
The installation itself consists of a physical device with a dial, much like an old radio, where the audience can rotate this object and tune the system. Just like sweeping frequencies on an old radio, the listener can scroll through different points in time to listen and read about which article was edited from which IP addresses. As a poetic gesture, the device is based on methods of surveillance. This type of work can be interpreted not only as a media art installation, but also as a form of indirect activism, transmitting messages in real time between locations and drawing attention to anomalies and concealed activities within bureaucratic systems.
Another important aspect of my works I’d like to introduce briefly here is the concept of generative procedures. These are methods that form structures through the use of autonomous systems, algorithms, and computational processes. This means, designing a set of rules, functions and parameters will define how an artwork is generated, where the process allows for variations and often unpredictable outcomes. Within these unique visual, auditory, and often experiential pieces, I usually have partial control or no control at all over the aspects of an artwork. In the past few years, I’ve begun connecting sonification systems with these visual, generative processes. The next work is closely related to this method of creation. My piece 1111000 points to a general phenomenon: cellular automata, which can be found in various natural disciplines. A cellular automaton is a discrete computational model consisting of a grid of cells, each of which can be in one of a finite number of states. The grid can be of any finite dimension, but is typically one-dimensional or two-dimensional. The state of each cell changes over time according to a set of rules based on the states of neighboring cells. They are used to model complex systems and phenomena in fields such as physics, biology, and computer science by simulating the interactions of simple, local rules, they are also applied in urban planning, traffic simulation to create dynamic patterns and predict system behaviors. An example of how these forms appear in the environment can be observed in the growth of shells, they can be described through mathematical models, which have been studied extensively by Stephen Wolfram especially in his book A New Kind of Science. Some of my works are published in the gallery section of a recent, extended release of the book, which is a celebration of the twentieth anniversary of the original publication.
In such automatons, individual cells can change themselves based on the surrounding cells in the grid. Some of them survive, while others die. This is the general idea, behind 1111000, which was exhibited at a large factory hall, called Artus Contemporary Art Space, in 2019. A tiny thermal printer was continuously generating these cellular automata patterns, creating never repeating, new iterations over time. The sound of the printer was also amplified, and it was set up to create rhythmic patterns between the new lines, so that there would be timed delays between each new row. The result turned out a bit like the Morse code system, yet it also had some similarity with insect based communication or similar small scale, high-frequency organisms. This installation was displayed in a large space where papers hung down from the ceiling as part of other installations, forming a unified structure with this printer on a speaker stand, that was producing the endlessly printed paper rolls every day, accumulating many meters of paper. I like to refer to this work as an endless shell-pattern generating machine.
What’s interesting here is that I’ve been talking about using real-time computation to generate images, sounds, behaviors, and conditional decisions. Apart of the real-time possibilities of reading, writing, and manipulating data, what also interests me is how artificial memory allows for the long-term organization of these consecutive events and how they create time-based imprints. While using all these nuanced tools, there are unintended and automated consequences coming from the usage of digital storage itself: self-documenting micro-histories that we create while running software or interacting with databases and performing any task in the digital domain. By their nature, these systems continuously measure what we do, adding and collecting data to existing databases across distant servers and computer networks. Recently I became interested in methods, where instead of overwriting and erasing these records, there are ways to keep storing these changes in an immutable manner across the whole infrastructure. Based on cryptographic consensus models, our digital imprints can be sequenced and layered upon each other, just like layers of the soil or stones in a physical environment. I already covered some problematic aspects of the consequences of constant data manipulation activities in the previously mentioned Echo project, where I observed how others’ activities manifested into patterns of sound. If we think of the ways how credibility, trust and resilience can thrive within digital protocols, we need a system that is not dependent on fuzzy intermediates and vulnerable third party elements, and a blockchain is exactly doing this, by eliminating centralized actors from its verification process, using mathematical tools only. Recently I work more intensely with this framework as a medium, investigating some very basic cryptographic characteristics of these systems, such as the hash numbers of the blocks, deterministic sequences, made with pseudo random number generators and the immutable nature of these collaborative digital consensus making systems. A recent work I created on a blockchain is called tur1ng, from 2022, which is a generative audiovisual project. It is based on code, which can have outputs of infinite number of variations based on a few initial parameters. The final number of outputs from this work has been limited to 300 editions. Some of the outcomes have been printed out and showcased as still images for an exhibition setup in the spaces of M21 Gallery (Pécs, Hungary), while the dynamic realtime animation of the generative process was projected on a small canvas. This setup is the intersection of the two different approaches I described before. While the real-time computational capacity can be seen in the movements of the projection, the imprint of these structures can be seen as a curated set of static images, which is a manifestation of long-form generative art.
An important addition is that these versions were not created all at once, but they emerged during the process when people interacted with the system on the blockchain. When they “minted an iteration”, meaning to initiating a record in the corresponding blocks in the database with a cryptographic signature, a new version was generated, where the seed of the random parameters in the code was defined by the hash of the minting transaction. At the end, the complete series is formed by the myriad interactions of the different people, during their collecting activities, each of them participate in the macro-structure of the work, where the algorithmic parameter space becomes explorable. Hence the term "long-form". By incorporating deterministic random in the process, I usually don’t know exactly what each iteration in the final series will look like when I prepare the code for the work. I simply define which colors are allowed, which shapes, forms, and algorithms can be used to generate a composition—but the final outcome emerges without the creator’s direct control, bypassing subjective decision-making. The system reveals itself as it unfolds, offering countless opportunities for discovery and exploration—for both the creator and the audience. These aspects serve as the basis of a very interesting new medium, turning the space of blockchains and digital infrastructures into community-driven, collaborative playgrounds with surprising and often unpredictable manifestations of artworks. With the case of tur1ng, the initial concept of the work was inspired by the model and purpose of the Turing machine itself — it is considered one of the first computers which had readable and writable memory units. Alan Turing created this machine in the form of a tape, on which memory could be written and read. The machine’s head moves based on this. So, Turing laid the foundations for writable-readable memory, amongst his numerous invaluable contributions to cryptography, artificial intelligence and computation in general. As with any decision making process, memory is essential to make it work, and in the case of blockchains, there is a fundamental difference compared to other digital tools: it is not possible to erase and overwrite previously recorded blocks in the database, we can only reference an already existing element. There is only addition of new records and information, such like with geological layers and rock strata of the deep time. There is no undo, there is no delete, only consecutive layers that point to newer transactions in time. Blockchain is an interesting technology because it introduces a paradigm in the digital world where it’s impossible to “forget,” fake, or overwrite past events with false attributes. Instead, it relies on an immutable set of transactions, organized into historical blocks that build upon one another. Many experts agree that this method of preserving digital information offers a promising solution to the complex moral and legal challenges posed by large language models and generative AI, especially in relation to the spread of fake content and issues around intellectual property.
In conclusion, my motivations can be summarized as follows: the real-time capability to actively simulate dynamic systems (applicable across various fields, from user interfaces to predictive systems) offers exciting possibilities when integrated into artistic practice. Secondly, the concept of artificial memory is a complex and, in my view, highly significant artistic research area, particularly in relation to media infrastructures, collaborative content creation and algorithmic prediction within networked communities. Addressable space, hyper-temporality, decentralized memory storage in computational practice has many research potentials that are still underrepresented in contemporary art and algorithmic culture.
Special thanks to the International Kepes Society, and in particular to László Zsolt Bordos and Márton Orosz, for their kind invitation and for their efforts in organizing the symposium.
* header image: Drawing of a 80-column and a 96-column punched card
Agoston Nagy
