Lipreading in the age of COVID-19

I was taught lipreading before I began using BSL (British Sign Language) and use the two communication methods contiguously. The term “lipreading” is somewhat misleading, as it’s not just lip patterns that contribute to understanding – you pick up information from the rest of the face, from body language, and from the contextual environment (context is something I’ll be picking up on again). My biggest obstacle is dark sunglasses that block the information you see around people’s eyes; if lip patterns provide information on words, the eye area often gives the equivalent of tone of voice, and lipreading people wearing sunglasses translates as a monotone to me.

During 2020, the ratio has been flipped, with masks preventing lipreading, but often leaving the eyes and surrounding areas clear; I’ve started to notice that I still pick up information, so can (sometimes) recognise tone, even if there’s no way to pick up words. I’ve also noticed hearing people struggling with understanding (with masks muffling voices), and am wondering if this will impact on people’s thinking about hearing loss in the longer term. Will their own difficulties lead to more understanding? Will they turn to D/deaf people as communication experts they can learn from?

I’ve been playing with images from the 1986 edition of “Lipreading – A Guide for Beginners”, masking the lip patterns illustrated. I’m planning a duotone risograph print, perhaps using several of these images – due to local lockdown I can only prep my files at present, so anything shown at this stage is an approximation.

Black and white photo showing a woman's face. Superimposed is an orange facemask. Only her eyes are visible. On the bottom left are the letters "OO"t ar
“OO”- Lipreading in the age of COVID (study for risograph) 2020

The virtual site of Cattle Depot Artist Village in Minecraft

Photograph showing the entrance to Cattle Depot in Hong Kong. The entrance is an archway inside a red brick building on two storeys, with two smaller outbuildings either side. The roofs of the buildings are covered in a dark rough tile, which is in parts covered in a red moss. Behind the buildings are tenement buildings, each approximately 10 storeys high; the one on the left is pale blue and grey, the central one is cream with orange stripes running beneath the windows, and the one on the right is pale salmon-orange/pink. The sky is overcast, pale grey-blue.
The physical space of Cattle Depot Artist Village in Hong Kong

Cattle Depot Artist Village was originally used as a slaughterhouse from 1908 to 1999. It was renovated and developed into an artist village in 2001. It is now home to around 20 art groups including the media art organisation Videotage.

For its first digital residency program, Videotage recreated in one-to-one scale the virtual site of Cattle Depot in the sandbox game Minecraft.

Screenshot of the virtual site of Cattle Depot in Minecraft. Buildings in digital red brick showing the Cattle Depot, with grey roof tiles and trees in the courtyard at the centre. 
Behind the Cattle Depot is a small park with trees. And to the left of Cattle Depot is another red brick building with a dark green roof. Buildings a re a mixture of one and two storeys in height.
A grid structure was built in the sky above the site by the previous resident artist. This forms a matrix against the blue sky.
Screenshot of the virtual site of Cattle Depot in Minecraft

I downloaded Minecraft today and got into the realm of Cattle Depot. What you see here is the aerial view of the site. The glass grid structure was build by the previous resident artist. I basically wandered around, tried to get familiar with all the keyboard commands, broke a gate by accident but managed to find a very hidden passageway that led me to the top of the grid.

Cosmic Call

For this residency, I want to create a series of symbols from the Dutil-Dumas message as an intervention in a virtual art space. The work will reference a previous work Cosmic Call. (An audio description of the visuals in the trailer is available)

One minute trailer of Cosmic Call
Audio description of the images in the trailer
Installation view of Cosmic Call with a flatscreena and two cube monitors. The seats are designed based on a code from the Dutil-Dumas Message
Installation view of Cosmic Call at Tai Kwun, Hong Kong

Cosmic Call is a work commissioned by Wellcome Collection in 2018

The main video weaves together facts and fiction to create an alternative understanding of epidemics. Cosmic Call resists the dominant narrative of a disease outbreak – a formulaic plot of a detective story about disease emergence and eventually the triumph of western medical science – and suggests multiple belief systems in which science is only one of many ways to understand communicable diseases. It points to the danger of reducing all knowledge to scientific terms while addressing issues that are pertinent to Hong Kong, which is characterised by its high population density and mobility – conditions that make Hong Kong extremely vulnerable to disease outbreaks.

In the video, an interstellar message (the Dutil-Dumas message) was sent by extra-terrestrial life forms that warned human beings of epidemics caused by passing comets. Information about the Dutil-Dumas message can be found on the RESEARCH section of the website.

What On Earth am I doing?

Well for starters it’s going to be made within a computer program (CAD)!
Continuing of my past projects shared Universe of ‘Tuner’ and ‘ProDancer’ we may find our selves in an off world sorting office re-distribute saved animals from past realms and sending them off adequately!

Hope to conclude as a downloadable executable both VR and Monitor based! As a wide narrative I’ve often related works on signalling and ever find it captivating in a virtual sense, here are some past worlds I may extend their essence into this currently unnamed one.


disabled becon from Buddha Geomatry child

“He maybe did” or “He may be dead”?

2020 has been a year of information distortion for many people in many ways. Being deaf means my focus has been around digital and physical access – or lack of access – to information, something I’ll be exploring during the residency.  Originally, I had been viewing my planned areas of research – typography and dummy text, AI transcription, and sound effect captioning – as distinct, albeit with some loose links. Maybe these areas have more in common than I envisaged?

I’ve begun looking at Dr Rachael Tatman’s research around conversational AI and linguistics and have been struck by the similarities between the areas of error in AI, conventional listening, and lipreading. These are often mirrored in typographic and compositing errors.

The images in this post are taken from

  • Lipreading – A Guide for Beginners
  • Finer Points in the Spacing and Arrangement of Type
  • An AI voice transcription programme

The header text on this post is taken from a research paper written by Dr Tatman in 2017.

Black text on white background discussing hyphenated works. The example given is "camellia" This word is split across two lines so that the word initially reads "camel-".
Dark blue text on white background from AI transcription programme. The text does not make sense showing the translation is distorted

Plastic Tat

Colour photo of a pink resin figure in clear plastic bag with a black, red and pink title card insert and illustrated cardboard label depicted the square headed figure.
Resin figure in clear plastic bag with title card insert and illustrated cardboard label.
Colour image depicting the reserve of the resin figure in clear plastic bag with title card insert and illustrated cardboard label. The following words are written in pink against a black ground on the title card: "Pure Electricity, Pure Data! A Message to Another World! A Traveller Through the Bleak Emptiness of Space! A Traveller Through Time! A Lover of Small Animals! Content with Their Own Company! Witnesser to the Awful Wonder of the UNIVERSE!"
Resin figure in clear plastic bag with title card insert and illustrated cardboard label.

I’ve made a short run of 15 resin Inaction figures based on my character Aracibo. These figurines are objects of power through which you can look back across vast swathes of time and space. Each figure is handmade and individually numbered, get ’em while they are hot.

Colour photograph of 9 pink resin figures arranged in a triangle, 4 at the back, 3 in the middle and 2 at the front.