Showing posts with label machine readability. Show all posts
Showing posts with label machine readability. Show all posts

Thursday, August 12, 2021

Odeuropa's Olfactory Iconographies

€2.8M grant for research project on European olfactory heritage and sensory mining:

Odeuropa bundles expertise in sensory mining and olfactory heritage. We develop novel methods to collect information about smell from digital text and image collections. They will identify and trace olfactory information in text and image datasets using AI, and promote Europe’s tangible and intangible cultural heritage.

Here's one of their ongoing projects, seen in the picture above, an odor wheel based on art historical references to smells: The Odeuropa Art Historical Scent Wheel from the Mediamatic Aroma Lab.

The Odeuropa “Nose first art historical odour wheel” starting from scent families in the centre, connected to odorants in the second ring, and then to artworks and artefacts around that, ending with an outer ring with Iconclass codes. Iconclass is a multilingual online database that museums use to tag historical images and artworks. -link

This wheel is based on imagery like paintings that were then coded with words, allowing a machine-readable dataset of odors, which will become increasingly more popular as we apply this approach to larger and more recent datasets, such as the Wikimedia Commons, or the running corpus of Instagram's zero-liked images.
 
Speaking of datasets, there are many to choose, from dog breeds to aerial photographs to 3 million "Clickbait, spam, crowd-sourced headlines from 2010 to 2015," to "4,000 physical dimensions of abolone." Wikimedia commons has 7.5 million images.

Looking back at the odor wheel, there are some interesting associations here. But they do reflect the dataset. I'm having a hard time finding details on Iconoclass but it was developed by one person in the 1950's, and based I assume on Western Art. It's currently maintained by RKD Netherlands Institute for Art History.

Here's some examples -- "street scenes and horses;" not a common conjunction in today's world. "Unicorns and cinnamon," anyone? "Prostitutes and civet," less unexpected. The two "body odors" are armpit and vagina, fyi. 

The most interesting part of all this? The lead researcher Sofia Ehrich has "become familiar with detecting depictions of smell." She can smell words and pictures. 

And speaking of words and pictures: 
Ocularcentric - like a visual bias, like we as humans tend to have an ocularcentric view of the world, with our trichromatic vision and fancy visual cortices, etc. 

Notes:
Mediamatic (in Amsterdam) is an art centre dedicated to new developments in the arts since 1983. We organize lectures, workshops and art projects, focusing on nature, biotechnology and art+science in a strong international network.

IconoClass dataset -  specialized library classification designed for art and iconography.

Lifting One's Hat

Layers of IconClass system

This is an example of the layers of the IconoClass system, pretty deep stuff. You can see how the image has words attached to it, making it a machine-readable cultural object. This is how we will teach robots of the future how to better understand us, and maybe we can even teach them how to smell.

Mario Klongmann x BigGAN - 2019

Mostly Unrelated Post Script:
An AI Artist’s Twitter Feed Is an Art Gallery
The images and videos Mario Klingemann posted under the hashtag #BigGAN can only be appreciated by treating his Twitter feed as a digital exhibition. (Images taken from the ImageNet dataset)
Feb 2019, Hyperallergenic


This AI Creates Art From Instagram Posts With Zero Likes
“Zero Likes” is trained to create glitchy visuals from forgotten social media images.
May 2017, Vice

Melbourne artist and coder Sam Hains created Zero Likes, an AI trained to respond only to those lost and lonely images that miss out on attention.

Wednesday, January 3, 2018

Beyond Literate Machines

 
Good thing I ran across this article today -
When A Machine Learning Algorithm Studied Fine Art Paintings, It Saw Things Art Historians Had Never Noticed
The Physics arXiv Blog via Medium, 2014.

Here's an article talking about a robot that can see similarities between artworks, and it's praised as finding something that no art historian has yet to discover. I wrote something about this on Network Address, because I like to write about more art-based things there. But there was a quote from the article that I thought was perfect for this blog in particular. They describe the process of training this algorithm to do its art-historian job. They feed it countless images, and with each one they have tagged with descriptions of its style, design, content, and (perhaps?) historical context. That way the algorithm 'knows' what it's looking at. And they conclude:

Comparing images is then a process of comparing the words that describe them, for which there are a number of well-established techniques.
Source document: Toward Automated Discovery of Artistic Influence, arXiv.org via Cornell, 2014


One of the main points of Hidden Scents is that the internet is and must be (for now) machine-readable - it must be made of words. Even the pictures must be reduced to words in order for this algorithm to 'see' them. Consequently, smell is word-averse. There is no language of smell, meaning there is no universal language to label the things we smell. Therefore, there cannot (for now) be such a thing as an internet for smells.

Image source: Sunmin Choi

Post Script
Causal diagrams by Edward Tuft