AI Supported Image Descriptions

My fellow resident, James Kong (be sure to check out his studio here) has written a post about how AI can aid accessibility, especially when it comes to image descriptions. I was really interested in how I could incorporate this into my own work.

As a disabled artist, I try my best to make my work as accessible as possible. This may take the form of subtitles, audio descriptions and image descriptions. It’s been interesting to see AI experts like James use AI to support the image description process.

I make disabled explainer content on social media and one of the more tedious tasks is image descriptions. Of course, it’s worth the effort, however, I often struggle with writing these descriptions because of the cognitive dysfunction that comes with my disability.

In my previous post where I spoke about ‘process’ and using Midjourney for the first time. I also used AI to support me with the image descriptions. Accuracy for image descriptions is important, so I didn’t rely on them completely however, I enjoyed how much easier it was for me. This is another example of AI being included in the creative process to increase accessibility for all audience members.

Below is an screenshot example of me using Chat GPT to write an image description for me. Interestingly enough, I’ll probably use Chapt GPT to write an image descriptions of the screenshot example.

Image Description: A screenshot image that shows shows a user interface for a text-based image description request. It includes an uploaded image and an AI-generated response. The image itself, seen in the screenshot, features a Black woman with long dreadlocks tied in a bun, pouring coffee from a French press into a white cup. She is in a cozy, well-equipped kitchen. The woman wears a colorful patterned top and bracelets, focusing intently on pouring the coffee. Steam rises from the cup and French press, indicating the coffee is freshly brewed. The scene is bathed in soft, natural light, creating a serene ambiance. The AI-generated description closely matches this depiction, detailing the woman’s appearance, the kitchen setting, and the overall atmosphere.

I think Chat GPT did an excellent job at writing the image description. If I were to use it in real life, I would make a few tweaks. Here’s what my edited version would look like.

“An AI generated image that depicts a Black woman with long, neatly styled locs tied up in a bun, pouring coffee from a French press into a white cup. She is in a cozy kitchen setting, adorned with modern appliances and a warm, homely atmosphere. The woman is wearing a colorful, patterned top and several bracelets. Her expression is focused as she carefully pours the coffee, with steam rising from both the cup and the French press, suggesting the coffee is freshly brewed. The lighting is soft and natural, casting a serene ambiance over the scene.”

On Process & Algorithms

[Image Description: A photorealistic AI generated image of a French press filled with steaming coffee sitting on the counter of an English traditional style kitchen.]

Before embarking on this residency, I have been thinking a lot about the idea of ‘process.’

During lockdown, repeat processes and set routines kept me grounded. I particularly think back to buying a coffee machine. You know, the ones where you buy the fancy-looking pods, pop them in the machine, press a button and watch your espresso be poured for you at the perfect temperature. There was something so unappealing about this. I was craving a deeper connection between the process and output of my morning coffee.

Now, in 2024, the coffee machine gathers dust while I use a simple French press instead. I enjoy grounding the beans to my liking, scooping the grounds into the press, filling it up with hot water, waiting a while, pushing the plunger down and finally pouring my coffee into a mug.

Perhaps this all sounds silly, but I learned a lot from reintroducing ‘process’ into my coffee routine. It’s for this reason that I’ve previously been a bit apprehensive about using artificial intelligence in my creative practice. The idea of it felt like I was once again removing the process of making art.

Now on this residency, I feel quite differently. I’m excited to explore the ways I can use AI within my process. I predict that I’ll use AI during the research and the early visualisation stages of my projects and then take that inspiration into the 3D production work that I’ve been developing over the last few months. However, I’m also happy for any of these predictions to change.

Anyway, the initial image on this post is one of my first attempts at using Midjourney, a generative AI tool. It did a really good job, but there are a few mistakes if you look more closely. The actual plunger used to press the coffee grounds to the bottom is nowhere to be seen. Also, you can see quite a lot of liquid coffee at the top. These mistakes are really exciting because it means it is my job as an artist to figure out solutions to these errors..or even exploit the errors for creative purposes.

[Image Description: A photorealistic AI generated image of a Black woman with dreadlocks pouring coffee from the French press into her cup in front of an old kitchen table.]

The next image I have included here is me trying to recreate my mornings with my French press using Midjourney. This is the prompt I used to generate the image:

a photorealistic image of a Black woman with dreadlocks pouring coffee from a French press into a mug in a traditional British kitchen

What I notice in these images (and the many others I tried to create) is that the algorithms do not understand how we as humans pour coffee from French presses into mugs. In the above photo, the person is pouring from a French press into a mug from a strange angle. They also hold a random unidentified object in their other hand. I tried to fix the issue with further prompts, but I think it will take more time for me to get it right.

The last thing I’ll add is that the image descriptions used in this post were supported by AI, but I’ll go into more detail on that in another post.

This is all part of the fun and I’m excited to see what will come of these AI experiments.

Some recent roses…

Over the past two weeks the rose garden has begun erupting into bloom. Now in its second year of growth (third year in design build), things are starting to knit together. The herbaceous perennials that i’ve incorporated into the design as supporting acts for the roses are emerging into their first year of growth; alliums, geums, alchemilla mollis, red campion, cuckoo flower, geranium dusky crug. There are probably too many roses. Even in just its second year of growth, I can already see that i’ve over planted and some of them are taking up way more room than i had anticipated. Some of the central allium clumps are completely swamped by the roses, particularly with the roses that i’ve struggled to support and are now growing much more horizontally than intended. I think i perhaps could have cut the numbers by half, but what a challenge that would have been… i don’t think i could have hampered my desires for these particular plants in what has been my first large scale design project. Perhaps it’s a bit like how people describe an author’s debut novel, the writer gets a bit carried away and wants to put everything in it. The eternal struggle of the edit.

It’s so interesting how needed the herbaceous perennials are in design. Without them, the roses are all orgasms, just these big shouts of shape and colour that leave little room for a crescendo or a breath. They are the crashing symbol, the exploding rocket, and they need other plants to buffer how loud that can be in design. The alliums in particular have been a really important way of capturing rhythm in the scheme, using them repetitively across the design that allows them to ripple in consistency amongst the changing shapes and colours of the roses. I’m already finding myself taking some of it out. The pulmonaria i planted last year are the most intense shade of powder-paint blue, so striking in late winter and early spring. But they simply don’t fit in with the colours im working with; a bruising blush of burgundy, deep purples, reds, pinks, faun-browns, pale yellows and dusky, sunset oranges. I always imagine that those uninterested in a colour scheme or particular garden design must find the idea of something “not working” and needing to be removed so unbearably snobby. And it is a bit outrageous. That pulmonaria is objectively a beautiful plant, quite worthy of any place in the garden. But i need it to support – to sculpt – the space in ways that the blues, and then later after flowering, the plain and pointed green hairy leaves, simply don’t achieve. I want to add more of the burgundy chocolate of the geraniums dusky crug and pink spice, both of which make for fantastic foil against the brightness and focus of the roses. I’ve been trying to incorporate bronze fennel into the scheme too, although they often succumb to my limited capacity to maintain plants from seed. Seed sowing to this day remains some of my most difficult aspects of gardening whilst sick, never feeling like i’m able to meet the demands of this needy process. I’ve grown so many fennels from seed that have shriveled up in overwatered trays or rotten over winter sat in cramped pots when they should have been planted out. I try and let it go as part of crip-gardening, but it is a grief to not feel able to keep up with the intensity that is nurturing plants from seed… perhaps something i’ll write more about. All of this to say, the perennials are very much a work in progress, and hopefully this summer i can begin to adjust the balance in the design.

/// /// /// /// ///

Here is what the garden is looking like currently. I’m mainly sharing bigger shots of the plot here and just a couple of close ups of the roses, just to try and give a feel for the overall scheme and layout. I will share more specific roses in my next post where i want to explore the different textures and forms that each rose offers, not only in genus but in the duration of flowering.

A digital photograph (landscape, colour), of what looks like a lush garden in summertime on a sunny afternoon. There is blue sky and large green trees in the background to the left of the image, and a large hedgerow in the background to the right. In the foreground is what looks like a large bed of shrubs with some pink and purple flowers, though the details are not clear. The image is facing the direction of the sun, giving the garden a golden glow in the light.

The garden erupting into bloom at the studio-allotment….

A digital photograph (landscape, colour), of what looks like a lush garden in summertime on a sunny afternoon. There is blue sky and a row of red brick terraces in the distance. In the foreground is what looks like a large garden bed of shrubs with pinky purple flowers and green leaves, as well as some perennials but the details aren't clear. There is a greenhouse behind the shrubs, and a grassy lawn to the right.

Here you can see planted amongst the roses the alliums on long thin stems with pom-pom like flowers on top coming into flower. Other herbaceous perennials like the green alchemilla mollis are also bursting into acid green bloom.

A digital photograph (portrait, colour), of what looks like a lush garden in summertime on a sunny afternoon. The image shows a green grassy path to the left of the image which runs along a busy garden boarder bursting with plants with pinky purple flowers. Some are large shrubs with green leaves and others are perennials with long stems. To the top left at the end of the garden sits an empty electric wheelchair under a small tree and in front of a green hedge. To the back right is a large green tree with a table and chairs and closed parasol in front of it.

I love this image so much, i really think you can start to get a sense of the rhythm in the design that the alliums bring to the scheme. Here, the drumstick alliums – which are only just coming into bud and are yet to bloom – are placed in clumps at intervals all along the front face of the design. Its really lovely to see things intermingle with one another more, especially after the first year of the design last year which felt very stiff and static as i gave the roses space to breathe and grow – it really did feel like a 1960’s front garden with tea roses! I love capturing my wheelchair amongst the garden in photos like this as well; small portraits of myself beyond my body.

A digital photograph (portrait, colour), of what looks like a lush garden in summertime on a sunny afternoon.  In the foreground is the front of what looks like a busy garden border brimming with plants. There are rosebuds on the end of stems, alliums just coming into flower in shades of pale pink/white and deep burgendy red/purple, and plenty of other shrubs and plants that we cant make out the details of. In the background a grey-brown shef sticks up over a green hedge. It is a bright sunny day with blue sky.

A closeup of one of the beds from the front (the garden design is essentially two large beds split down the middle).

A digital photograph (portrait, colour) of what looks like a close up of a rose in bloom. There is one large rose that is open with big blousy petals in shades of pale pink fading to almost white. The open flower sits in a cluster of buds yet to open, some still green and others turning deep pink. Surrounding the buds are lots of stems and green leaves. It is a sunny day.

A close up of the Kazanlik rose. Kazanlik is a beautiful, once-flowering rose that is found growing in parts of eastern Europe where it is used to make rose oil. It has an incredible turkish delight scent, and I love that it only flowers once, one of the few roses in the design that does so with most of the others being repeats.

A digital photograph (portrait, colour) of what looks like a close image of a number of roses in bloom. The flowers are bright pink in the centre fading to pale pink at the edges. The petals are big and blousy and crumpled at the centre giving a very loose and relaxed structure. Surrounding the flowers are lots of buds, stems and green leaves. It is a sunny day.

A close up of the Kazanlik rose.

A digital photograph (portrait, colour) of what looks like a close up of a rose in bloom. There is one large rose that is open with big blousy petals in shades of bright, pale pink fading to light pink. The open flower sits in a cluster of buds yet to open, some still green and others turning deep pink. The petals are big and blousy and crumpled at the centre giving a very loose and relaxed structure. Surrounding the buds are lots of stems and green leaves. It is a sunny day. There are fingers holding up the flower to the cameral. The person's fingers have pale skin.

A close up of the Kazanlik rose.

A digital photograph (portrait, colour), of what looks like a lush garden in summertime on a sunny afternoon. The foreground of the image is busy garden boarder bursting with plants with pinky purple flowers. Some are large shrubs with green leaves and others are perennials with long stems. There are pink roses dotted about that are clearly visible in the foreground. In the background at the end of the garden sits an empty electric wheelchair under a small tree and in front of a green hedge. It is only just visible behind the mass of plants and flowers creating a sea of green and colour. It is a bright sunny day with blue sky.

A close up of the garden showing it as a sea of colour, shape and texture. Seeing it emerge into this current phase of the design is truly thrilling to me, after it living inside my head as an imagined place for years now.

greenhouse reflections 1

A digital photograph (portrait, colour), of what looks like the interior of a greenhouse. There is dry, brown soil at the bottom of the image, with thin struts of grey aluminum supports running vertically up the left of the image, with a bracing piece running at an angle across it. Behond the large panels of glass we can see a mass of green foliage with what looks like yellow snail shells hanging seemingly suspended between the greenery and the glass. The foliage is dense and thick, and we cannot see anything behind it. It looks like to could be nettles but it isnt clear. There are a number of reflections in the glass caprturing the rest of the greenhouse structure and other shapes. There looks like there may be a figure captured in the reflection of the glass, with them wearing a patterned jumper and a cap, but it is only faintly visible. At the base of the glass lies a small heap of wilted leaves from a plant as if wilted in heat. The image is strange, ordinary and calm.

I find myself continually drawn to creating images found in reflective surfaces, inspired by the way they facilitate image-making and how portraits can be captured in these transient spaces. In 2022, I created a solo show of self portraits and images found in reflective surfaces found in my home which was displayed at Level Centre, Derbyshire. The series was titled Sick Gaze, and explored the views, observations and contemplations both of and from the perspective of the sick body amongst domesticity. The images were printed onto brushed dibond. I loved creating this series, and i always want to pursue these ideas of image-making further.

I took this photo at the allotment inside the greenhouse, and suddenly i found it a really interesting place to consider these reflective concepts of sickness caught in a momentary image. What i was trying to explore in Sick Gaze was some of the vastness found in the small, cramped spaces of the sick existence in domesticity. Scale is endlessly fascinating to me as a sick artist, how sickness is often a practice of taking those small, cramped experiences of sameness and sitting with them, zooming in until they become vast landscapes. I hadn’t really considered how these ideas could be applied outside of domestic interiors, and it’s really interesting to me to think about how the allotment functions for me in this way.

I’m interested in the threshold of where the disabled body leaves the private space and meets a public one, and how the liminality of sick and disabled experiences results in this threshold often becoming a permanent state of being. The allotment holds lots of this sentiment in that way; not open to the public yet not wholly private, external architecture creating pockets of interior shelter found in sheds, greenhouses, chicken pens and polytunnels. We don’t live there, but the domestic finds itself out in the open all the time; old carpets used as weed suppressant, milk bottles on canes to scare the pigeons, bathtubs become waterbutts and salvaged windowpanes make up magnificent glass houses. I love the architecture of the allotment, this strange jumble of wreckage and bounty, it has a language all of its own. I’ve been thinking a lot about the way this language disrupts, how there is never a clear line of sight at the allotment, your gaze constantly interrupted by the combination of the knackered rake and rubble weighing down the tarp on a shed roof, the slump of a muck heap half covered or the debris of community life that the allotment often hosts like the storing of youth football club nets. It makes me think of illness as disruption, how there is never any straight line found in that landscape either. I love to think about gardening in this way, utilising design by not removing the disruptions but finding breaks and gaps amongst it, finding a kinship in the constant collaboration between my body and its own disruptive, uncomfortable limits.

I’m tired now and so I’m going to leave it there for now… I haven’t shared my writing/thoughts like this publically for some years now, having once been very present online via my Instagram @bella.milroy. It all feels very alien to me now to share my thoughts in a live/contemporary way like this, to share text that isn’t very polished or fully thought out. But this residency has been the first time i’ve made work in this way for so long now and i’m trying to embrace the format. It feels a bit weird, scary and nice.

Couch grass

Digital scans of Couch Grass roots taken from the allotment.

A digital photograph (portrait, colour), of what looks like some kind of organic plant-like material. The bacground is grey/black and the plant material is white with notches of brown/black. It is a single piece of material that ends in a point. It could be the thick root of a plant but it isnt clear.

A digital photograph (portrait, colour), of what looks like some kind of organic plant-like material. The background is grey/black and the plant material appears as a small clump of grassy strands in yellow/brown/green/red, with a small clump of soil at the base and fine roots. It has one sharp root sticking out to the right.

A digital photograph (portrait, colour), of what looks like some kind of organic plant-like material. The background is grey/black and the plant material appears as a single stalk of long grass with large flowy green leaves. There is a small clump of soil at the base and fine roots. It has two sharp roots sticking out at the base, they are bright white and look like runners coming off the main plant.

A digital photograph (portrait, colour), of what looks like some kind of organic plant-like material. The background is grey/black and the plant material appears as a large clump of grassy strands in yellow/brown/green/red, with a small clump of soil at the base and fine roots.
A digital photograph (portrait, colour), of what looks like some kind of organic plant-like material. The background is grey/black and the plant material appears as a small clump of grassy strands in yellow/brown/green/red, with a small clump of soil at the base and fine roots. It has one sharp root sticking out to the right.

A digital photograph (portrait, colour), of what looks like some kind of organic plant-like material. The background is grey/black and the plant material appears as a small clump of grassy strands in yellow/brown/green/red, with a small clump of soil at the base and fine roots. It has one sharp root sticking out to the right.

Couch Grass is a native grass to the UK that is an important food source for butterfly and moth caterpillars, as well as being incredibly fast growing. Left unchecked, it tunnels its way throughout the ground using its piercing runners to create rhizomes, and swamps everything it encounters. An elegant and graceful species, it is also very difficult to maintain in the garden, and can easily overwhelm other plants if left unmanaged. 

It’s a real bastard to weed, its roots forming interconnected pathways under shallow soil. Upon pulling at a clump, you can easily break one of the extensive runners, simply leaving behind another plant ready to break through the ground and grow on. But every now and again you can find yourself picking at loose, dry topsoil where all of a sudden you are lifting a root-runner from the ground like a hidden rope found in sand in some desert-island-fiction. Perfect lengths of tough, sinewy roots are untangled from the soil, often pulling whole clumps of grass with it. My favourite part of the plant however, is the brand new runners that form these blanched white spears just below the soil surface. They are so neat and clean, pushing their way through the earth in energetic spikes. There’s something about the contrast of the tangled mass of older roots that form the clumps of grass, and the newness of these glossy threads that makes for a very satisfying and tactile interaction when weeding this plant in the garden.

GenAI: A First Step

As part of my residency with Vital Capacities, I’m exploring the relationship between GenAI and artists, focusing on how AI can enhance our creative processes. Today, I’m excited to share my first use case: using ChatGPT to extract and describe image content for accessibility purposes. Here’s a photograph I captured during my urban exploration.


To ensure that my work is accessible to all, I used ChatGPT to generate a detailed description of the image. The AI accurately identified the silhouette of a person standing on a high vantage point, overlooking a brightly lit cityscape at night, with an overcast sky adding to the dramatic atmosphere.

A screen capture of James requests ChatGPT to describe his portfolio picture. ChatGPT response the image description

This initial use of GenAI demonstrates how artists can utilize AI tools to create accessible content, ensuring that our work can be experienced by a broader audience, including those with visual impairments. By integrating AI into our practice, we can enhance the inclusivity and reach of our art.

Stay tuned as I continue to explore and share more ways in which AI can support and transform our artistic endeavors. This is just the beginning of an exciting journey with Vital Capacities.

I have been making some watercolours over the last couple of weeks. In the hope of slowing down my brain and enjoying the process of making art again. It felt like a really regenerative process and really allowed me to take in/reflect on the work i had made in the past few years while also looking forward to what im going to be making next

And recently in the last few days ive been turning some of the thoughts i had during making the watercolours into sets/ideas/thoughts for the film im making with jen.

The memorial hall

The structure was ready made by the architect team. We are now building the detail of the interior.

The eventual form will be a digital grave , we will be able to maintain the content for the rest of our life, as an artist last will. Also, the spaceship looking hall is an extension of the video work Space Warriors.

Guts

A animated gif of a crossroad of white guts on a black background. The guts are pulsing, like they are either breathing or in the act of digestion.
Gif from Surface of Scum, a character generator computer game made by Sammy Paloma & Uma Officer

I’ve been thinking about guts. Or rather, I’ve been thinking about thinking about guts. It’s more of a hunch that there is work or attention needed in that area, both of my life and my art. Guts have appeared in my work for at least the past 5-6 years, if not longer, and yet I feel like at some point they became a kind of stand-in for an unfinished thought. When it started I think the thought was around unseen processes happening in the background (like how wallpaper can function like an engine to the actions inside a room even though u may not consciously notice it). But it’s evolved to take on digestion, and waste, and mess, and desire, and grief. And in a funny way, guts were also always there almost as a reminder to myself to actually listen to my own guts. But there’s some other stuff there too – appropriately – beneath the surface of the image of guts. The guts of the guts. And this post is a kind of note-to-self to feel into that. To get on up inside those guts again. Sorry not sorry.

La Mort

A greeby-brown, browny-green ink painting. A wet looking expanse of brown-green, in the centre of which is a dark-green bubbling bog. Out of the bog can be seen three hands, one foot, and the head of a person who's gender is unclear, wearing a crown on their head (with their tongue hanging from their mouth).

A little painting I did vibing with the 17th Century Tarot de Marseille Death card, taking it’s iconography of harvesting the dead to the site of the bog. I enjoyed making the inks as boggy as possible.

The Bog is a Thin Place; A Sanctuary, A Portal & A Home

A watercolour painting featuring a pair of Simpson-yellow arms with black chains around the wrists. The hands have long and gnarly bright green nails wriggling their way towards the sky. The chains around their wrists also extend upwards towards the sky.
The arms are splashing out of a boggy, bubbling hole in the ground. This hole of green bubbling sludge is surrounded by brown earth blooming out at it's edges.

I love that phrase u sometimes hear about Halloween/Samhain: “the thinning of the veil”. From the idea of thin times it’s only a small leap to that of thin places. In Shetland witch (or heksi) folklore it is said that the stretch of sand that runs between high and low tide is the undisputed territory of the devil. Folk magic loves a threshold, a space between spaces (oh so liminal): a beach, a fog, the meeting point of two rivers, a keyhole, midnight, a crossroad. A bog. To get lost in a thick fog whilst traversing a bog might just about be the perfect setting to summon the dead. My sense is that these thin places and times are potent precisely because they can function as temporary sanctuaries, a home for those pushed to the margins. Not unlike Doom Patrol’s Danny The Street (DC Universe/HBO) a roving sentient genderqueer street that goes where it’s needed and provides a home to for us misfits.

I took a break last week and dissapeared to the country side, it was magical mostly because i had very limited internet access. there was a lot of cows, rhubarb and ants (so, so , so many ants)

It really made me slow down and in that process I returned to drawing and watercolours. I finished a major project in November last year and havnt stopped since, so it was nice to decelerate and return to a more normal pace of life 🙂 :), Ill post some of the watercolours soon.