Yuxi Liu and I were invited to produce an artwork that would respond to data produced by BBC Radio 4 on the solstice day (21st June).The resulting work, “No Machine is an Island”, was featured in the Digital Human programme Data Stream Day broadcast on the 24th September.
The installation presents a conversation between two machines based on machine learning algorithms. The conversation starts with one machine translating the poem “Light Breaks Where no Sun Shines” into images gathered from the web. The poem was written by Dylan Thomas and was part of the Four Seasons radio session broadcast on the solstice day. While one machine interprets the poem into images, the other uses Google’s Tensorflow Img2txt algorithms to interpret the images back into text, and finally construct other poems based on LSTM (Long Short Term Memory) algorithms pre-trained to interpret seed texts into one that mimics the style of writers such as Shakespeare and Hemingway. The first machine would then interpret the machine-generated poem into images again in a continuous loop.
The conversion of images into text uses an algorithm trained to recognise images based on the Common Objects in Context (COCO) dataset, which is highly practical and biased towards technical artefacts. This restricted ability to recognise things in the world contrasts with the rich imagery initially drawn from the association of images of nature with Dylan Thomas’ poem, which demonstrates a contrasts between human and (possible) machine sensibilities.
We used Processing, RunwayML and ml5js in the final installation.
Wuxi has published further documentation here.