Auto-poetic Abstraction

Thoughts and meanders on art, technology and everything under the sun. Digital Arts Computing student at Goldsmiths, Yasmin's the name.

[polemicnature]

Degree Show Participation

For our Third ___ online exhibition, I worked tirelessly as part of the Events team to plan and execute the public facing side of the exhibition and ensure that we have some kind of celebration after all of our hard work.

After some kind words from the head of Computing and Rachel and Jesse, I kicked off the 'Opening Night Showcase', which was a chance for students to show off their work and get a chance to speak publicly about their process and inspiration.

Rachel had this awesome idea of a 'relational circuit', where each person plays three different roles: mediator, artist (presenter) and respondent. This 'threeing' is not only a more engaging way of showcasing work and putting the pressure off everyone, but it also fits in very well curatorially with our thinking of always being in flux and change.

With this relational circuit, we rehearsed once and then prepared more of what we wanted to say and got more comfortable with the format. We smashed it, adding photos and interesting visual material to a presentation. Below I'm mediating for Nate's Oort Cloud, an awesome CGI documentary film about this relatively unknown cloud of asteroids that is super difficult to conceptualise because of the sheer distance of it. I had a lot of fun and felt like it was a good way of getting closure for this exhibition.

On Saturday the 10th of May, I chaired a panel discussion with a group of 3 wonderful women, working in the intersection between art and technology in various ways. We spent a wonderful afternoon 'traversing the binaries' that was a big part of our curatorial statement and meandered around a few different topics, such as:

Binaries and constructs (ie., knowable⟷unimaginable, structure⟷agency, everyday life⟷unending history)

Mediated architecture (the space and conflict between digital and 'real' landscapes)

Hyperlocality and proximity

How to organise our thinking of displaying work and being represented online or in new spaces

Below is a screenshot of me talking to Ibiye Camp about how she investigates the biases and hidden conflicts of technology. I also had the chance to ask quite deep questions related to my personal interests of Artificial Intimacy and how Paula, Julia and Ibiye think (or not think) about what it means to be close and have machine mediated relationships.

We had many ideas about what kind of events we wanted to do and I would have loved for individual students to host their own discussions related to their work/practice, but it was difficult to get people involved. In fact, apart from a couple meetings with 2nd year Arshia and support from Ashley and Georgie (who had other roles), I was essentially working alone on planning these projects. I am indebted to Rachel for being so supportive and helpful. Both of these events were received well and I couldn't have done it without the wonderful support and encouragement of my peers.

[polemicnature]

Not Just 1 Shader, 2!!

Above is a screenshot of two shaders ported to p5.js. I used the tutorial in the open-source project p5.js shaders called 'Applying Shaders as Textures'. Instead of just setting the active shader, the shader is actually passed onto a createGraphics() layer and then create some geometry for the shader to lay on, such as a .rect. Finally, this createGraphics() layer is passed as a texture, which can then be mapped onto any WEBGL shape p5.js has available. In this instance, I used two cubes and placed them slightly away from each other so they're not quite touching and you get that nice perspective of depth. In a way, this is like a virtual version of the shaders inhabiting separate monitors, so they do have some sort of substance in the digital world.

[polemicnature]

poseNet working!

I had several issues that were stumping me on this project.

training poseNet model to work with a shader, to modify/augment the shape or rotation of the abstract entities

getting two shaders to run at once on the same sketch

Above is a short demo of one of my entities rotating according to my different poses. I initially passed mouseX and mouseY values mapped to something more useful for the shader (which I still need to tweak to find the optimal mapping) and then I used regression to train the poseNet model as my arms went from left to right (mouseX) and up and down (mouseY). It appears that the movement is most clear or definable when going in the y direction, but left and right is more janky. The next steps is to smooth out the movement, and I know I can do that from my previous experience with poseNet, which is interpolating between the previous position to the current position. What is still a large question is the relevance of this interaction. When a new pose is detected, does that prompt the entities to generate their text? This is something I’m still working through.

Next I will talk about how I got the shaders to run together (aka texturing an object like a plane with the shader) and how I got the text to generate continuously, which was surprisingly easy!

[polemicnature]

poseNet working!

There were several issues really stumping me on my project:

1: training poseNet model to modify/augment the shape or rotation of the abstract entities

2: getting two shaders to run on one p5 sketch.

Above you can see a quick demo of a poseNet model modifying the rotation angle of the shader. It is a bit jumpy, but I know how to make the movements more slow by interpolating between previous and current points. I achieved this by originally sending mouseX and mouseY as a uniform, mapped to different values for the shader (that I still need to check to optimise). I then created some sliders (for visual feedback) that would hold the mouseX and mouseY values. The reason why it was not working before after I trained the model is because I was still passing the original mouseX and mouseY values to the uniform, rather than passing the values that are being changed by poseNet. Now I just have to work on making the poseNet movements meaningful and more precise and think about the fact that most people will be viewing it sitting down. Next post will be about running two shaders at once and hopefully getting some sort of 'communication' between them.

[polemicnature]

Exhibitioning

I wanted to update about the exhibition process. I'm leading the Events team, but it has ended up being mainly myself with Ashley supporting from the writing team. We've worked well together on writing the curatorial statement and I implemented some creative writing methods I learned in my residency with Arebyte last summer.

I have greatly appreciated Rachel's support in the events/curatorial side of things and she has been invaluable at bouncing ideas off of.

We are planning on having a mini showcase of work on the opening night with students pairing off and discussing/comparing their work for a couple minutes each. I think it'll be a fun and engaging format!

We finally sent emails to the shortlist of people we would like to speak at a panel discussion on the Saturday of the exhibition. So far, Paula has replied but the two others have so far been silent. This means that we will have to go back and find some adjacent people and work fast before it gets too close to the exhibition.

[polemicnature]

Progressing + Crit with Anna Ridler

I finally have some progress to share! Slowly but surely, I have figured out how to get the two models to generate text through two separate buttons. I'm working now to make it 'automatic' by using setInterval() or some other method.

At first, I was generating the text into a p element but I found Ellen Nickle's great little project Personalized Privacy Policy that uses ml5.js charRNN trained on a corpus of different terms and conditions. She implemented a really smart way of continuously generating text by putting it into one textarea element and checking whether there is a full stop, which will stop the generating. That means that the text can be generated letter by letter (stateful) and will stop at full sentences.

I credit her completely for getting me over the block of making the text generation easier. Rather than having many differently styled div elements to design a conversation, instead I'm going to have two text areas that will look like they're 'talking' to each other.

Above is a full screenshot of a shader working on a p5.js canvas. Currently something is making the site run very slowly but I'm going to install htop to figure out what exactly is slowing down my laptop.

Last Monday, Anna Ridler gave us a masterclass talking about her work and also had some crit sessions. I took the opportunity to show how Amorphous Butterfly was working so far. It was enlightening to get an outsider's view on my work, especially from a great artist.

My biggest takeaway is an important question I need to ask myself: what is the significance and importance of touch? Make it clear of the intimacy required.

This is something I need to figure out the importance of because I originally wanted to depict what a machine relationship could be and exist without human interference, the audience 'interaction' was meant to be more of an 'intra-action' of understanding that there is a relationship here devoid of a human. BUT I do want to integrate poseNet, I want something there so I need to have a good think and it's own blog post probably. Anna suggested that the conversation could start with an interaction, a required element. I need to ask myself why.

Is the audience searching for meaning with the entities?

Is there a log of previous conversations?

Anna also found interesting that there is 1 model (koan) trained by me and 1 model ‘trained by the world’, is that a choice, a dichotomy that is intended for something? Something else to think about, because I like it. Much to ponder!

[polemicnature]

Physical or Online?

Earlier this month, I borrowed two screens from the Computing office to set about experimenting with Amorphous Butterfly as a physical installation. I photographed them on top of the beanbag because the beanbag represents intimacy but also something very casual, trying to represent what this relationship between the two 'machines' have.

However, now I feel that my work may just be an online piece because there are a lot of elements working at once, a ML language model with two shaders communicating and I feel that it's not easily feasible to run a shader on separate screens while also communicating to each other.

[polemicnature]

New Year, Same Project

Project summary

My final piece, tentatively named Amorphous Butterfly, will be two AI entities conversing with each other, independent of human contact or interference. A speculation on machine relationships. The two AI’s will be abstract entities morphing and animating as they talk, probably through generated text that feeds back into each other.

What’s Been Up Lately?

Focusing on my dissertation these past few months has meant that physical iteration has been put to the side. However, I’ve developed some thoughts conceptually. I’ve been inspired by interaction research Phil Van Allen’s work on animism, leveraging human’s tendency to imbue liveliness in to inanimate/non-human objects. This opposes the dominant ‘slave/master’ relationship model in HCI (Human Computer Interaction). This has led me to emphasise the ‘personality’ of the AI entities, manifested somehow in their attitudes, maybe physically as they could look jittery when reacting to something uncomfortable, excited, etc. I want them to be unique, in a way, not just a pretty representation.

Exhibition Talk

Because the exhibition is now going to be online, I am thinking about experimenting with AR (Augmented Reality) so that my piece can be translated to an audience viewer’s room so that the physical manifestation can be linked to the conversation happening on the piece’s webpage/screen. I want them to inhabit some sort of space. I have to decide soon whether this can be viable.

Next Steps

The most important next steps is to have a working MVP, a first prototype that shows: two AI entities ‘talking’ to each other. Depending on how much I can achieve, I’ve been thinking of going the route of combining a deep learning language model (such as charRNN) that is trained on a large corpus of text and then link that to a chatbot scripting language (such as Rivescript) so that the conversation can happen in real time and there can be a log of it for people to look through. It will be interesting to see how I can trigger certain animations or behaviours to occur when talking about certain things.  

[polemicnature]

Artificial Intimacy: Connection in the Digital Age

Last Tuesday, I had the pleasure of giving a talk through Hacksmiths, the Goldsmiths tech society, on the topic revolving around my dissertation. You can take a look at my accompanying presentation here.

It was a fun evening, where I had the ability to verbalise my thoughts and theories in front of a little group of 10 people and I loved taking this opportunity because the input and feedback from other people apart from myself, who has been thinking about this topic non-stop for the past couple months, can give me a fresh perspective.

Slightly unconventional for an art dissertation, I’m conducting a small study to get some qualitative first-hand research into people’s experience being ‘intimate’ with an AI, by looking at an AI chatbot called Replika. I’ll keep the findings from the study mainly for my dissertation, but I’m finding it fascinating how everyone so far can see how helpful an application of this technology, the human element of AI, can be for people. There are so many questions to answer. Last year, I was more focused on late capitalism and the datafication of individuals and how Machine Learning affects knowledge production and hence realigns the way our society functions, as it becomes more ubiquitous in the world, but now I’m more concerned on (I guess) the affective dimension that Artificial Intelligence can afford.

It’s no longer a thought experiment. AI is changing the emotional predicates of what a relationship consists of. It’s not just a simulation of emotion. I state in the presentation that I cannot actually tell for sure that you understand what I am saying. (I’m sure Kant has something reasonable to say on this, how we represent each other the way we want to). Yet even though I cannot know this 100%, it does not stop us from communicating and possibly having a very deep understanding and affinity for each other. Looking at it this way, there is nothing stopping us from developing deep relationships with AI. Now the nature of AI, it’s convenience, reliability, allegiance to us as creators of it, is something we need to think very hard about whether the balance of that relationship is too slanted in our favour.

[polemicnature]

On Touching

I first became acquainted with Karen Barad’s work in the Automated Gaze seminar in 2nd year. Their work is a challenge to read through and understand. The conceptual framework around the transposition of quantum physics onto feminist critical theory proves why.

Recently, I was introduced to an essay Barad wrote on ‘touch’ and as I’m doing my dissertation on Artificial Intimacy and my final project on the relationship of two AI entities, I think this could be a good piece of writing to draw from. What follows are just a few surface reflections. Many quotes and ramblings will entail.

Each “individual” always already includes all possible intra-actions with “itself” through all the virtual others, including those that are noncontemporaneous with “itself.”

Barad describes how on the smallest scale of life, atoms interact with each other in such a way that what forms a ‘thing’ such as oneself is a multitude of interacting phenomena, so that each present conglomerate of a person/being is a collection of everything that could have or could have not existed. Intra-action is a term Barad uses to explain that agency comes from not a singular origin, but as arising from multiple causes/areas/experiences. 

I especially enjoyed how they talked about virtuality and these ‘virtual others’, as Barad sees virtual particles (therefore virtual anything) as real, but not present. Virtual particles are seen as “quanta of vacuum fluctuations”, as in they are the quanta of the void and on what is not being measured or a version of oneself that ‘could have’ happened.

Put very beautifully, “virtuality is a kind of thought experiment the world performs”, and I like that because it makes me think of all the kinds of multitudes of people I have been, as well as could have been if my particles interacted with others in just slightly different ways, and most excitingly all the kinds of things I will be from now on.

Indeterminacy is an un/doing of identity that unsettles the very foundations o non/being.

Identity is a funny thing. We go through crises over it, over our frustration with reconciling different parts of ourselves. I think Barad’s thoughts on this is very freeing, as we are in this constant state of flux. However, this doesn’t mean that they take a pessimistic view of the world, that because nothing is stable = nothing has meaning. Instead, they advocate for more response-ability, that we have the capability and empowerment for change in the world. It makes me think of a shared blanket, where the very fact of living is powerful yet contingent on so many other people and things. We are threads, weaving through being/nonbeing and we have a shared responsibility to act in the world. Yes, I’m also a fan of Hannah Arendt. 

[fbrie002]

more renders . .

[polemicnature]

mini planetarium

[polemicnature]

Amorphous Butterfly - Final Project Prototype

On mimicry, indeterminacy, (mis)understanding / (mis)classification, intercorporeal connection.

In Blender, I made a few speculative renders about my final project.

There are two bright, abstract entities side by side, giving visual form to an AI. These animations are experiments with a ray marching technique on a WEBGL shader, abstract imaginings of a future deity, circa 2050. The twisting tori also has a really nice palette, which I borrowed from Inigo Quilezles’ nifty color palette function.

The music that accompanies the above deity was made in FoxDot and recorded in SuperCollider, really fun to play around with. Because sound can be quite polluting in a group exhibition, if I would have music or noises, I think I would provide headphones.

These models, as they twist and turn, talk to each other. The AI companions are having a conversation, about what I’m not so sure yet, about humans? I also want them to discuss and try to identify photos. The photos are indeterminate, weird and grotesque things, being (mis)classified and labelled by another. Crappy facebook memes, cats, GAN generated deformities, etc. The pink screen below the two wall-mounted screens is where text is going to be displayed.

I’m really excited to use the new GPT-3 language model, but it’s being slow to release for many reasons so I will probably use GPT-2 for the two models. I would like to explore further natural language processing and generative text.

[polemicnature]

Trevor Paglen’s “Bloom” at Pace Gallery

Reputation exceeds presentation.

That’s how I felt about this show, as I visited the Pace Gallery last Saturday, a contemporary art gallery situated within the Royal Academy of Arts. I came with a couple friends, both that are quite academic but not heavily into fine art. They found it boring, they had absolutely no context to bring together the series of strangely coloured flowers, the bust of a head and some generated text and images.

If I was to rate this show, it would be very low. I’m disappointed that Pace had absolutely zero text to read and contextualise what visitors are seeing. Nevertheless, it should be possible to glean some sort of meaning from simply experiencing these artworks, but this show lacked any emotional response.

Perhaps that is the point, machine vision is devoid of beauty. Computer vision is only able to see through the concepts we assign to it. Therefore, as I’m reading Kant’s Critique of Judgement and auditing the ‘Philosophies of Nature’ module, computer vision cannot really identify/witness/create beauty because the machine requires a conceptual framework to ‘understand’ what it sees. 

As Kant states that a beautiful object is something that shows “purposiveness without purpose”, computer vision can only understand beauty in relation to what the object represents.

The series of photographed flowers, coloured in strange areas, feel cold and slightly nauseating.

Trevor Paglen, born in Maryland but now living in Berlin, I consider first and foremost an investigator. He first came to prominence when taking abstract photographs of classified military sites around the US. He is researching and exploring the way we now represent the world in terms of the “politics of images”, and this is being heavily influenced by Artificial Intelligence and Machine Learning’s ontology of classification and prediction. I find his work fascinating and first came to know him with his huge work at the Barbican called From ‘Apple’ to ‘Anomaly’. He is one of the artists that got me interested in these topics.

Through meditating on Paglen’s work after visiting the exhibition, I have more of an appreciation of the blooming flowers, but was this show meant to be disappointing?

A lot of this work was made in the heat of lockdown in New York, so perhaps not disappointing, but in fact hollow and sad.

[polemicnature]

Cuties - Girl’s in The Digital Age

I finished Cuties (Mignonnes) today and the ending made me cry. It was beautiful. At the very end, after everything that happened, the protagonist Amy walks down her apartment block and joins in jumping rope with some other kids. As she plays, there is a harrowingly beautiful song being sung in Wollof, the language they speak in Senegal, where Amy’s family is from. As she jumps, gleeful, the camera rises up to the sky, and she jumps higher and higher and higher, smiling at us. Simple childhood joy.

I enjoyed the elements of surrealism in this film, paired with an amazing soundtrack. It’s subtle, yet adds a magical almost mystical tone to the film.

The ending made me cry, cry for women, cry for the little girls growing up today, and my own memories that resurfaced.

I grew up with the Internet, in-between mySpace and Instagram, so I was playing games on Facebook, talking to creepy strangers online and messing around with ChatRoulette and Omegle.

The outcry of this film, the over sexualising of young girls and accusations of pedophilia, is exactly the subject matter the director was trying to address. I read in a film critic’s review this phrase: “depiction is not endorsement”. I agree with this completely.

The film is hard to watch, especially in the moments when the group of girls are dancing sexually, twerking and being very suggestive with their bodies. It’s meant to be hard to watch, it’s meant to make us think “Oh shit, 11 year old girls actually do these things”. It made me think of all the things I did as a young person and teenager, thinking I was grown and talking to older kids. Being placed in difficult and dangerous positions. I only think of it now in hindsight, but at that very moment I was trying to prove the world that I was not a child.

I can’t imagine becoming conscious in the age of Instagram and TikTok, this obsession over becoming viral and getting more likes. This self-consciousness over young people’s bodies and appearance at such an early age can only be devastating, and I’m not surprised it may lead to loneliness. That’s who Amy is, she’s a lonely girl who becomes part of this dance friendship group and does mean things, in order to tend to her very basic needs: love and belongingness.

There is another element to this story, where Amy’s father becomes engaged to a second wife and plans to live in their small apartment. I cannot imagine what it must feel to see your mother stripped and humiliated, yet unable to do anything. This is when Amy realises her place as a woman, as a second-class citizen of sorts.

This clash of cultures, between a devout Muslim country and secular France (the wider Western world), in the film shows that no matter where, young women are still fighting to be taken seriously, for ownership over their own bodies and lives. This film shows that we are not finished yet. Women are not empowered.

As I grew up a Muslim, I felt very strict gender roles placed upon my thinking, even though I was never told I couldn’t do anything, the general feeling was there. Yet I don’t think the West has figured it out either. In some interesting conversations I had with good friends a couple days ago, it dawned on me that we have a long way to go to an equal society. We may be able to achieve similar things as men in a public realm, but in our most private and intimate moments such as our relationships with our body, sex and love, we feel we cannot talk about things. That if we feel wrong or uncomfortable, it’s our problem, we’re the fucked up one, when in fact it is our society that still doesn’t take women’s needs seriously.

[polemicnature]

Deptford Day Out: Gossamer Fog and Seal Gallery

Last month, I spent a wonderful afternoon exploring Deptford, in South East London. On the high street is an inconspicuous door that leads to a small basement gallery called Gossamer Fog. This duo show called Zeitgeber is “external phenomena that inhabits and activates the synchronized cycles of the Earth, enabling our circadian rhythms to align with the cycles of the solar system”. I never heard of this term and originally thought it was fabricated, but upon looking it up it is in fact a German word that literally means “time-giver” or “synchroniser”!

In the main room were several sculptures by Andrew Sunderland. It was my first time seeing work created with thermoplastics, which melt in water and can be moulded and re-moulded, solidifying once cooled. From the gallerist that was present, she told me that he used recycled plastics and they can be melted again and again. Fascinating material!

Stretched along what seems like a black silhouette (corpse) of a body, Sleepers 11 looks like the remnants of a nuclear future, sickly pale yellows, oranges and pinks oozing out from the body. The sculptures were so tactile and enticing, looking both repulsive and delicious. 

Bits of moss grow from the shoulder, chest and head, signifying that this toxic man-made excess goo has been here for lifetimes, inviting life-forms to inhabit and colonise it. 

Scattered on the left are plastic nasal drops, making me think more of these bodily fluids oozing out of the body. As this exhibition seems to be about critiquing late capitalism’s domination over all aspects of our lives, disrupting our few hours of sleep and rest, I first thought they were eye drops.

Part of of the title of this piece was “(its body formed an upright building, growing towards the sun)” and I loved that as this being seems to inhabit multiple timescales, abandoned and reclaimed by nature, as well as still in flux, the textures of the thermoplastics never quite at rest.

There was a two-channel video piece, OK SWALLOW GREAT, by Solveig Settemsdal. I took no photos but it was a wonderful meditative piece, overlaying images of water, still canals with big cement walls going through eutrophication, the process of when there are too many nutrients in the water (pollution like nitrogen and phosphorous) that depletes the oxygen in the water, thus killing off wildlife.

 When I was in high school studying environmental systems, eutrophication was a favourite topic for me and I ended up doing a major project for my final year, testing the oxygen levels in different water areas around my apartment neighbourhood in China.

Settemsdal also had a fascinating installation in the main room, which looked uninteresting at first but upon further investigation, the elaborate contraption moved strange fluids with magnet pumps and other things to create a downward stream of liquid down a full tank. I will take a look more at what materials she used, definitely thinking about the inversion of time and the strange physics and chemistry of these fluids. Perhaps they are in us?

In the same room as OK SWALLOW GREAT, Sunderland had another wall-mounted sculpture.

Titled BAD COPIES 3, this sculpture was beautiful and eery. I first thought, because of the slimy hood and bumps down the middle, it was like an alien vulva, but also maybe a mutated, dying stingray. This was not thermoplastic, but it felt so dense and weighty it looked like ceramic. I like seeing these strange corporeal forms and trying to wrap my head around what it is, whether it’s a piece of a larger entity or an auto-poetic organism, one of the few species left in the depths of the ocean, once we destroyed the world and ourselves.

I visited another small gallery next to Deptford Bridge Station, that was having an opening for the founder of Gossamer Fog, Samuel Capps. Exudater was a solo show. I did not spend long here and took no photos unfortunately, but there were some awesome tree-like forms. He seems to work with a lot of muted, ashy greys and deep matte and sleek blacks, mixed with glowing purples. 

There was VR environment to explore that I decided not to try as I find using the strange eye masks too uncomfortable to enjoy the experience.

I want to read more on these artists, especially to look deeper at Settemsdal’s video works.