From Creative Coding to Neurocinema

Alessia: Today we want to talk about creative coding with Gennaro Bosone, a Fullstack developer at Develer. Gennaro, what is it?

Gennaro: Creative coding is the act of writing code for the pure pleasure of it, where the goal is not functional but creative and expressive; where code becomes a medium, just like a canvas for a painter.

It doesn’t have a specific purpose, or rather, the only purpose is to have fun, to create, there is no commission upfront. It’s a way of spending time, alone or with others, to communicate something in a more abstract language.

Alessia: So, is it also a way to make friends?

Gennaro: Yes, just as there are collaborations between photographers, videomakers, musicians, and dancers, there can be collaborations between artists and programmers.

Alessia: Is it only virtual collaboration or also in person?

Gennaro: It can be both, of course. There can be virtual collaboration when meeting on forums and working together, and I’ve made various friendships this way. Or there can be “real” collaboration. One aspect of creative coding that personally excites me the most is live coding, which involves generating images real time. For example, collaborating with musicians: while they perform musically, a live coder can perform visually.

Alessia: So, in a real concert?

Gennaro: Yes, in a concert, an electronic music event, a live music event, but also in a dance, a music video, anywhere there’s a need for a visual element.

Alessia: And how did you start?

Gennaro: I’ve always been fascinated by transforming code into something visible. I started using terminals with C language; then, in High School, I created my first websites in HTML. That was the first time I saw what I wrote being displayed on a screen. The first time I used mathematical formulas for drawing was during a probability and statistics course in High School. I saw that mathematical functions generated graphs, and so I started doing my first visual experiments through those lines.

Later, I wanted to delve deeper into one of my favorite languages, JavaScript, for drawing purposes, because it provides more or less low-level APIs for drawing. Then there was a wave of generative art on social media, which gave me the push to write my first library for drawing with code.

Alessia: Is there any tool you would recommend for someone who wants to try this?

Gennaro: It depends on the preferred language, but almost all of them provide APIs for drawing. These are called Canvas and with them you can draw lines, circles, points; with the combination of these, you can generate shapes.

Alessia: Do you use JavaScript?

Gennaro: Yes, and one of the most important frameworks for creative coding is Processing P5. There are tons of tutorials, and it’s used by many people in both amateur and professional artistic contexts.

For drawing, there’s also Hydra, and I also recommend my library, Urpflanze. Additionally, there are native Canvas APIs that don’t require a framework.

There are many frameworks available, but I believe that creativity lies in how you use a tool, not in the tool itself.

Alessia: For those who want to visualize what you’re talking about, do you have any websites or artists you would recommend?

Gennaro: As for recommending artists, it depends on what visually aligns with your idea of beauty; some may prefer more abstract or geometric things. There are also many artists to be inspired by.

For example, Andreas Gysin is one of the artists I follow and appreciate the most. He often uses ASCII Art and pixel manipulation, and writes frameworks to create his visual effects.

I’d also point out Matt DesLauriers, who is one of the most skilled. He has been a strong advocate for generative art and has gotten many people interested in it.
Technically speaking, he uses P5, but also writes custom code that he shares on GitHub. He has a very rich artistic output, ranging from more abstract works, like the Meridian collection, to others that somewhat resemble cubism, with simple geometric shapes.

Alessia: And what about music?

Gennaro: There are obviously tools for live coding with music. One option is Strudel for JavaScript, or Tidal Cycles, based on SuperCollider. I don’t know if you’ve ever used a DAW to create beats or record electronic music. Tools like Strudel or Tidal Cycles allow you to generate your own sounds and your score to be played through code.

Among the artists, I’d mention Char Stiles and dj_Dave, who are interesting because they’ve organized electronic nights for developers, where code is the aesthetic component of what’s being heard.

Alessia: If an average person wanted to approach creative coding, do you have any advice on how to get started?

Gennaro: To approach creative coding, you don’t necessarily need to focus on visual aspects. As I mentioned, creative coding means having fun and doing something outside of work. If a programmer writes code for themselves, because they enjoy it and not because they need it, then they are engaging in creative coding. It doesn’t need to have a specific goal; it should be a way to spend time, just as a musician might play guitar even outside of a concert.

The advice I can give is to do what makes you feel good. I mean, if one enjoys terminal drawing, that is also creative coding. If someone wants to experiment with visual aspects, because they find it enjoyable and interesting to show something written as output, then they can approach P5, which has the most extensive tutorials. For example, there’s the YouTube channel The Coding Train by Daniel Shiffman, one of the creators of P5. It’s great because he manages to convey his passion for what he does and encourages experimentation. This channel is very famous, because it teaches the approach to thinking in pixels.

Alessia: Thinking in pixels? What do you mean?

Gennaro: The approach to drawing is different from what we know, where you take a pencil and draw a line. To draw a line in a programming language, you have a point defined by two coordinates, x and y, and another point with different coordinates, and then you draw a line between them. Thinking in pixels also means considering how that line is rendered on the canvas.

If you think about it, this happens in “normal” drawing too. If we take a line and zoom in, we see it isn’t perfectly straight, but there are fill areas that look straight from afar. Thinking in pixels means placing coordinates on our canvas, and it’s up to us how to fill them. It also means not only drawing a line through x and y functions, but also using generative algorithms. For a line it is a little difficult, but for slightly more abstract things you can think of drawing as something that is transformed by code.

The works in the Meridian collection, for example, which we mentioned earlier, are certainly the result of a generative algorithm: particles fall and accumulate like grains of sand. So, thinking in pixels also means allowing yourself to be surprised by algorithms that are more or less predictable.

Alessia: How does a generative algorithm work?

Gennaro: A generative algorithm means that the algorithm influences the drawing. Let’s use an example: you draw a line, but you don’t want it to be perfectly straight; it should have random waves. There are infinite ways to create these waves. One way is to use randomness. If you tell the algorithm to place a point higher by a random value between 100 and 200, you don’t know what will happen because the algorithm will choose a random number. This is what is meant by generative algorithms: when the behavior is somewhat predictable, but can change with each iteration of the drawing.

Alessia: Like shuffle play on Spotify: you never know what you’ll get, but within a range you provide.

Gennaro: Exactly. That’s the simplest example. Because you can extend this randomness in a much, much more complex way. You can also say: randomly draw a block of pixels, and then apply another algorithm to this block that makes random color changes. Then, once that’s done, apply a displacement to these pixels, also randomly. It’s an endless way to iteratively randomize a drawing.

Alessia: So each time you generate something different?

Gennaro: Actually, you can always generate the same output through what is called a ‘seed’ in programming, i.e. you set an initial value and the random function will generate the same random values sequentially. But you can also not set this function and each time generate something new, totally unique in that case.

Alessia: It will be unique, unrepeatable.

Gennaro: Unique and unrepeatable yes. You have lost it forever if you don’t know those values. And it is irrecoverable.

Alessia: But that’s also the beauty of it, right?

Gennaro: Yes, of course. One of the things I like most about live coding is that sometimes, in moments of inspiration, amazing things are created. Then repeating them is almost impossible 🙂 🙂

Alessia: We are talking about a developer who spends 8 hours programming for work and then gets into creative coding. In the collective imagination, the programmer is someone who is locked in a dark room staring at a screen. Doesn’t this activity risk reinforcing that stereotype even more?

Gennaro: It depends. If programming allows you to get out of the house, even if you spend most of your time in front of the computer, you might still have the opportunity to meet other people. During these encounters, you are probably not constantly glued to the screen; instead, you might take a break and have a chat.

For example, last night, after work, I went to rehearse for a live show I’m going to do with two friends of mine: I was still in front of the computer for a while, but I spent the rest of the evening with them. 

It can be physically and mentally exhausting, that’s true, but the emotions you experience are wonderful: participating in events where you do live coding, where you also challenge yourself, where there are people you network with, which then leads you to other events and situations. That’s the beauty of it. For example, with the neurocinema project, I have visited Verona, the province of Benevento, various areas of Campania, and many other places I had never been before.

Alessia: What is neurocinema now?

Gennaro: It is an experimental cinema project that I am pursuing with my friend Dino Esposito: it is an interactive short film, in contrast to what cinema is now, more and more for the masses. Our goal is to create an individual and subjective cinema, where the choice is unconscious: it is what your body wants to see.

At the moment, the version we have developed uses an electroencephalogram, through which we decipher alpha, beta, theta, gamma and delta waves.

And through these brain waves, we manipulate the texture of the short film, or apply post-processing effects such as colouring or distortion on the video.

Alessia: How do you change the short according to my waves?

Gennaro: There is an editing game, there are clips, and the structure works like a tree. You start by seeing the initial scene, which is the same for everyone. During this scene, the sensor sends your brainwaves to the software, the software processes them, and decides which clip to show you in the next scene.

Alessia: Like a book game: instead of rolling the dice, I choose what I want to happen.

Gennaro: Exactly. There was also an experiment by Netflix, Bandersnatch from the Black Mirror series, where you could choose what the protagonist would do with the remote control. In neurocinema, the choice is unconscious, there’s not even a break: it feels like watching a single, continuous video.

Alessia: So if I see the two protagonists of the video breaking up and, unconsciously, I don’t want it to happen, it won’t happen on the screen?

Gennaro: What we decipher are the alpha, beta, gamma, theta, and delta waves, which are associated with concentration, meditation, and relaxation. So, we cannot detect emotions directly, but we can assess your physical state – whether you are tense, relaxed, or attentive. Beta waves, for example, indicate whether you are paying attention or not. 

Alessia: Going back to the example from before: so if the protagonists are breaking up and I stiffen up, it means I don’t want this. If I relax, it means I’m OK with it.

Gennaro: Exactly

Alessia: And this is an application of creative coding?

Gennaro: Yes, because this isn’t something a videomaker can do alone. This is one of the things I’m most passionate about in programming: interactivity, making something dynamic that isn’t normally dynamic. You take an object and make it smart, enabling it to do things it normally couldn’t do. Take a painting, for example: instead of being on canvas, it’s displayed on a screen and when you wave your hand in front of it, you can change it. This is the beauty of generative and digital art, making the viewer feel like part of the artwork.

Alessia: How do the viewers react?

Gennaro: They react well! Everyone asks us if they are crazy or if we stole their social passwords. 🙂 Just kidding, but we have always had a good response. And also some very, very strong reactions.

Alessia: In what way?

Gennaro: During a screening in Verona, a woman started crying when a scene with a horse appeared, followed by another scene with dunes and a meaningful sentence. This is because neurocinema combines rather abstract images with textual content. The whole context leading up to that sentence brought back memories of her childhood, arousing strong emotions in her. It was a very evocative and involving moment.

Alessia: Was she more excited or more uneasy?

Gennaro: She was definitely excited, but she was also a bit uneasy. The first thing she said to us, as soon as she took off the sensor, was: “Please, tell me these things weren’t generated just for me.” We explained that this wasn’t the case. Certainly, wearing a sensor can be a bit suggestive, but the goal of neurocinema isn’t about how accurate it was, but whether you liked what you saw.

Alessia: But the fact that she got excited is a success. That’s what you wanted: for that sequence of images, sounds, and words to provoke emotion.

Gennaro: That’s why I said it impressed me: because she truly immersed herself in what we wanted to show her.

Alessia: And that’s a big success: being able to convey emotion through code. Code is seen as the coldest thing in the world, yet you manage to bring emotions to people.

Gennaro: In this case, there was also the narrative work; the credit isn’t just due to the programming. As we said, this was a collaboration of equals between cinema and programming: the director did a great job, the writer did a great job, and my algorithm did a great job selecting that plot. So yes, code can evoke emotions.

Digital works can also evoke emotions, just like a painting by Dalí.

Alessia: But while it’s normal to think that a painting by Dalí can evoke emotions, you would never think that code could generate those feelings. So I think we take this away today: code can evoke emotions.

Gennaro: Some people think that just because it’s digital art, it’s “easy,” because the software, not a person, created the work.

Alessia: More than easy, I think sometimes it seems cold; the fact that it comes from algorithms and numbers thrown onto a computer and that the human interaction isn’t understood, makes it seem cold.

Gennaro: Take the Meridian works; the artist printed them. If you see those prints, not knowing he’s a generative artist, you think they are paintings. So the point is: it’s cold because we consider digital cold, but in things where one puts their soul and passion, those don’t feel cold.

Alessia: Indeed, it’s the emotion you put into it that people perceive.

Gennaro: Exactly, I mention those works because, in my opinion, you can feel the study behind them, but also the meticulousness of wanting to reproduce something that, as Walter Benjamin said, is “the unique appearance of a distance, however close it may be.“

You can feel it when a work is art, whether digital or not.

Alessia: Gennaro, let’s close with this beautiful phrase you said, and I thank you for this journey that took us from creative coding to neurocinema.