Fruit Flies Forever

Human sense of smell resembles that of insects

Oct 2021, phys.org

https://medicalxpress.com/news/2021-10-human-resembles-insects.html

Good, because we would really like to use insect antennae to better understand human olfaction. They're easier to do experiments on, because their system is more simple than ours in many ways. Also good because the fruit fly is where so much smell science comes from.

They modeled the brain of a cotton bollworm so they could inspect its operations, and found they're a pretty good match for humans (minus the phermomones, of course). This is good for helping us understand the inner-workings of a robust neural network, or should we call it the prototypical, the primordial neural network: 

"We find striking similarities in the structure and function of the olfactory system across different organisms," says Xi Chu, a researcher in NTNU's Department of Psychology and senior author of the new publication. The similarities are probably related to the fact that the olfactory system is evolutionarily the oldest of all sensory systems. ... "It's worth noting that the primary olfactory center in the mammalian brain is located only one synapse away from the outside world," says Dr. Chu. "This means that the incoming information goes directly into the primary olfactory cortex, unlike all other sensory signals, which travel through a different brain structure before dispersing to their respective cortical areas. -Steinar Brandslet, medicalxpress

via Norwegian University of Science and Technology's Chemosensory Lab: Jonas Hansen Kymre et al, Distinct protocerebral neuropils associated with attractive and aversive female-produced odorants in the male moth brain, eLife (2021). DOI: 10.7554/eLife.65683

http://dx.doi.org/10.7554/eLife.65683

Image credit: Antenna of a male moth by Dr. Igor Siwanowicz at the Howard Hughes Medical Institute in Virginia for the 2015 Nikon Small World Photomicrography Competition [link]

Post Script:

Mapping the olfactory system in fruit flies

Feb 2022, phys.org

https://phys.org/news/2022-02-olfactory-fruit-flies.html

They describe the fly's olfactory system as having "the ability to make quick assessments of odors in an unusual way that circumvents synaptic communication, which is metabolically expensive."

They have created a map of receptors based on variations in the functionality of the molecules, but with one extra step -- the activation-inhibition dynamic at the neuron level.

This is a feature of the olfactory system that has been researched a lot lately (see this post for example). It also sounds like the model for neuromorphic processor systems, where the advanced processing via a dedicated cortex is eschewed for a complexity-based, emergent phenomenon at the neuron level.

via University of California - San Diego: Shiuan-Tze Wu et al, Valence opponency in peripheral olfactory processing, Proceedings of the National Academy of Sciences (2022). DOI: 10.1073/pnas.2120134119

http://dx.doi.org/10.1073/pnas.2120134119

Flipping the Switch

The 'surprisingly simple' arithmetic of smell

Jan 2022, phys.org

https://medicalxpress.com/news/2022-01-surprisingly-simple-arithmetic.html

The age of artificial olfaction is upon us.

This is now the second report in the last few months that presents a computational model for the olfactory bulb, which is the biological supercomputer on your face that crushes gigtons of databytes per attosecond (slight exaggeration).

The last paper came from a physicist working on information theory (Tavoni et al at Penn State). Another paper the month prior, which came from none other than the lab that discovered olfactory receptors, found, again, a computational model, discovered via machine learning, that compresses the n-dimensionality of odorant sensory data. 

But again, this new paper comes primarily from a department of electrical and systems engineering, in collaboration with the biomedical engineering department. I don't know everything that's going on, I only read the papers on the weekends, but that's a lot of papers about computing in olfaction, and from people who do not study olfaction exclusively.

And this is at the level of the bulb. We're not talking about the DREAM project, where big-data's worth of words and molecules are processed by GPT-3 to predict the names of smells. This is about looking at the hardware. How in the world does that bulb, which compresses thousands of receptors, themselves receiving information from un-countable stimuli, into dozens of signals that go on to control the entire enormity of a mammalian body via its limbic system. The bulb is the choke point for this system, and it's using magic that we are only now beginning to understand to the point of copying it. 

I doubt this is the earliest example, but as far back as 1991, scientists were talking about olfaction as a model system for computational neuroscience. These were neuroscientists and psychologists writing about this. But they could see the significance -- it's literally wired like the deep learning neural networks you hear about in the news (you know, powering the AI in your toaster, your tissue box and your alarm clock). 

It really looks like we're getting the hang of this. They started with a simple question -- how come things smell the same to us, even in different contexts or environments? Like how a plaid shirt looks okay in your sunlit bedroom, but later looks like a shit sandwich in the fluorescent lights of your office (or remember the black-and-gold dress? maybe you're trying to forget). 

If smells come from evaporating molecules, which are literally volatile, changing all the time based on environmental conditions, how come they always smell the same to us? Maybe olfaction would be a good model to investigate. 

So they did, by pairing locusts with a training smell, under all kinds of different conditions, hungry, full, hot, cold, humid, dry. Every time, the locust recognized the training smell (with the locust equivalent of a salivating dog). Yet, "The neural responses were highly variable," one of the researchers said. Same molecule, same response, but completely different receptor patterns, every time. It just doesn't make sense.

Deep learning to the rescue (obviously). The algorithm found that it's the interaction of activating and inhibiting neurons; I'll copy the copy directly:

Finding the features you want is similar to the information conveyed by the ON neurons. Absence of deal breakers is similar to silencing of the OFF neurons. As long as enough ON neurons that are typically activated by an odorant have fired—and most OFF neurons have not—it would be a safe bet to predict that the locust will open its palps in anticipation of a grassy treat.

via the Department of Electrical and Systems Engineering and the Department of Biomedical Engineering, Washington University in St. Louis: Srinath Nizampatnam et al, Invariant odor recognition with ON–OFF neural ensembles, Proceedings of the National Academy of Sciences (2022). DOI: 10.1073/pnas.2023340118

http://dx.doi.org/10.1073/pnas.2023340118

And further reading:

via University of Pennsylvania: Gaia Tavoni et al, Cortical feedback and gating in odor discrimination and generalization, PLOS Computational Biology (2021). DOI: 10.1371/journal.pcbi.1009479

http://dx.doi.org/10.1371/journal.pcbi.1009479

via Massachusetts Institute of Technology's McGovern Institute for Brain Research: Peter Y. Wang et al, Evolving the olfactory system with machine learning, Neuron (2021). DOI: 10.1016/j.neuron.2021.09.010.

http://dx.doi.org/10.1016/j.neuron.2021.09.010

via MIT: Davis J L & Eichenbaum H, eds. (1991). Olfaction: A Model System for Computational Neuroscience. Boston: Bradford Books/MIT Press.

https://mitpress.mit.edu/books/olfaction

Deep Nose, 2022

http://www.limbicsignal.com/2022/02/deep-nose.html

Signal to Noise for the Win, 2021

http://www.limbicsignal.com/2021/11/signal-to-noise-for-win.html

Olfatory Overload, 2021

http://www.limbicsignal.com/2021/05/olfactory-overload.html

Image credit: Inhibitory Synapse - TAO Changlu, LIU Yuntao, and BI Guoqiang; Image design: WANG Guoyan, MA Yanbing - 2021

Deep Nose

Artificial networks learn to smell like the brain

Oct 2021, phys.org

https://medicalxpress.com/news/2021-10-artificial-networks-brain.html

We now have an artificial neural network that works like the nose. It's not an electronic nose; that's not the big deal part. 

What's important here is that, first of all, this is from the lab that brought us olfactory receptors. Next, these scientists didn't even model the network on evolution. They made an algorithm to solve an odor categorization task, and let the network run with it. Some might call that artificial evolution, but scientists will call it machine learning. After several iterations, the network found an optimized structure for solving this task -- the network ended up looking just like our olfactory system. Go figure.

The "brains" of a neural network lies in its ability to reduce the dimensionality of the information thereby optimizing computation. This is done using compression layers that learn to accept information from some neurons and not from others. After enough iterations, a pattern emerges between the layers of neurons.

Talking about this pattern, and the number of neurons connected to by each neuron on the compression layer:

"It could have been one, it could have been 50. It could have been anywhere in between," Yang says. "Biology finds six, and our network finds about six as well."

-Guangyu Robert Yang, associate investigator at MIT's McGovern Institute for Brain Research, and postdoc at Columbia University

The first part of our nose where the hundreds of olfactory receptors collapse into far less neural nodes, one layer up the network, is very similar to the type of artificial neural network used in this study. It is, apparently now supported by these findings, a very effective way to condense a multi-dimensional information-space of chemical signals. Would it work for other information-spaces? What other Big Data can this deep nose model compute?

The side story: Evolution found this organization through random mutation and natural selection over eons; the artificial network found it through standard machine learning algorithms, in under one minute. 

via  Massachusetts Institute of Technology: Peter Y. Wang et al, Evolving the olfactory system with machine learning, Neuron (2021). DOI: 10.1016/j.neuron.2021.09.010

http://dx.doi.org/10.1016/j.neuron.2021.09.010

Image credit: Google's quantum computer, totally unrelated, just for looks

Post Script:

Neural network reveals new insights into how the brain functions

Dec 2021, phys.org

https://medicalxpress.com/news/2021-12-neural-network-reveals-insights-brain.html

"The neural network model approach we have developed in this work presents an 'instruction manual' for other researchers to use to study other areas of the brain or other organs"

-co-author Dr. James Martin, co-author and professor of molecular physiology and biophysics at Baylor College

Their model is called Spatial Transcriptomics cell-types Assignment using Neural Networks (STANN).

via Baylor College of Medicine: Francisco Jose Grisanti Canozo et al, Cell-type modeling in spatial transcriptomics data elucidates spatially variable colocalization and communication between cell-types in mouse brain, Cell Systems (2021). DOI: 10.1016/j.cels.2021.09.004

http://dx.doi.org/10.1016/j.cels.2021.09.004

I'm not certain about this, but I think the reason they chose the olfactory system is because they were looking at the interaction of transcription genes and brain cell types, and the olfactory receptor brain cells are the only one's that each get their own gene. So that would make the olfactory bulb an ideal nexus for investigation of this kind. See studies below for further reference:

Marei H.E.S. et. al. Gene expression profile of adult human olfactory bulb and embryonic neural stem cell suggests distinct signaling pathways and epigenetic control. PLoS One. 2012; 7: e33542. https://doi.org/10.1371/journal.pone.0033542

Nagayama S. et. al.  Neuronal organization of olfactory bulb circuits. Front. Neural Circuits. 2014; 8: 98. https://pubmed.ncbi.nlm.nih.gov/25232305/

Neuromorphic Odor Translator Helps Robots Express Their Feelings

Neural network trained to properly name organic molecules

Aug 2021, phys.org

https://techxplore.com/news/2021-07-neural-network-properly-molecules.html

Do you ever have a problem naming that smell? It's not just you. Science has this problem too, but maybe not for long. 

Smells are volatile organic compounds that have evaporated and entered your nose. Although they almost always occur in combination with others and not in isolation, us humans want to reduce smells to their individual components, and then name them. After all, in order to think about something, you have to know it's name. (Is that true?) 

The problem is that organic molecules are big, with lots of chemicals joined together in lots of ways, so coming up with a naming convention for all these permutations is hard. IUPAC, the International Union of Pure and Applied Chemistry, sets the convention for naming molecules. And boy is it complicated.

Take sugar, a common molecule known to us by its simple name "sucrose;" in IUPAC, it's called (2R,3R,4S,5S,6R)-2-[(2S,3S,4S, 5R)-3,4-dihydroxy-2,5-bis(hydroxymethyl)oxolan-2-yl]oxy-6-(hydroxymethyl)oxane-3,4,5-triol.

Since we do live in the computer age, folks want to automate this naming process for when they discover new molecules. But as you can imagine by looking at the IUPAC name for sucrose, the algorithm at the core of that naming convention is really hard to write. So they decided to use a neural network instead.*

*I'm casually calling a neural network "neuromorphic," but in the past few years, real neuromorphic computers have forced a distinction here that I'm ignoring here for the sake of a more clickable title. 

This new artificially intelligent chemical translator is not a magical structure-to-name translator that can just look at a chemical and give it a name; that's still out of reach. It does, however, translate between IUPAC and another naming convention called SMILES.

Trained on PubChem's 100 million molecules, this translator ultimately shows how the utility of the new approach of using neural networks to help us write algorithms from the bottom up instead of the top down, which really is a revolution in computing. 

And if you think it would be cool to have robots that can smell, or to ensure that future humans maintain their sense of smell as they evolve into hyperdimensional algorithms, then making odors machine-readable is how you do that.

via Skolkovo Institute of Science and Technology, Lomonosov Moscow State University and start-up Syntelly: Lev Krasnov et al, Transformer-based artificial neural networks for the conversion between chemical notations, Scientific Reports (2021). DOI: 10.1038/s41598-021-94082-y

Promiscuous Pattern Recognition

Study reveals how smell receptors work

Aug 2021, phys.org

https://phys.org/news/2021-08-reveals-receptors.html

Big smell news - for the first time ever, using cryo-electron microscopy, we can see an olfactory receptor in action. And as expected, it doesn't work like any other receptor.

Odorant receptors are known for their 'promiscuous chemical sensitivity;' that's a scientific term, by the way. It means that any one receptor might be sensitive to hundreds of molecules, so it's been really hard  to figure out what makes any particular molecule match with a receptor.

They looked at the jumping bristletail (surprise - not the fruit fly) because it has only five types of receptors, and because one of those receptors (OR5) is really broad, responding to 60% of the smell molecules they presented to it (promiscuous).

So they look at this receptor in its default state, and then again as they expose it to smell molecules (either eugenol or DEET).

And? Its ion channel pore dilates. That's it. Both of the competing theories about how smells work were wrong. It turns out they work via nonspecific chemical interactions -- they are not recognizing a specific chemical characteristic, but something more general about the molecule itself.

And there you have it! Olfaction is still one of the strangest senses we have.

Don't forget to thank cryo-electron microscopy, and the hundreds of scientists who have been trying to figure this out over the past hundred years.

via Rockefeller University: del Mármol, J., Yedlin, M.A. & Ruta, V. The structural basis of odorant recognition in insect olfactory receptors. Nature (2021). https://doi.org/10.1038/s41586-021-03794-8

Artificial Olfactory Perception and the Olfactome

Chemical informatics, machine learning and the indispensable fruit-fly, Drosophila melanogaster have been used by researchers at University of California Riverside to predict odor perception. 

Olfactory prediction is kind of a holy grail of sensory perception. Sounds sus. Let's get into the data.

image credit: Diatom, by Dr. Jan Michels for Nikon Small World 2020

Using artificial intelligence to smell the roses

Aug 2020, phys.org

https://phys.org/news/2020-07-artificial-intelligence-roses.html

First sentence they're referencing Asifa Majid. That's a great start. Her work shows us that culture, language and experience influence individual odor perception. Nonetheless, the search for the human odor code continues.

After reducing a larger dataset of 84 olfactory receptors and 54 allelic variants (138 total), they took 34 receptors, each of which is controlled by a single gene, and trained machines to predict their descriptors. The descriptors, or "the words we would use to describe the smell," came from the Vosshall Keller Rockefeller University 2016 lexicon. They've got about 170 odorants, working on 34 receptors. 

Remember that each odor receptor gene can be activated by a number of chemicals, sometimes by only one, but usually by more than one. This is what makes things complicated. Olfaction is a combinatorial affair that breaks down at the granular level.

And they made a model for each receptor, 34 different models, and fed those models the odorants. They found that you could predict chemical properties of the molecules that match each receptor tested. So now, we can use the 450,000 library of chemicals, run them through each of the 36 artificial receptors, and predict what those receptors would perceive.

Figure 5A: Few Key ORs or Chemical Features Sensibly Cluster the Perceptual Descriptors

(A) Dendrogram representation of the Euclidean distances among perceptual descriptors based on overlap of perceptual response data (% Usage) from chemicals in the ATLAS study.

(B) Dendrogram from the top five ORs picked per perceptual descriptor.

(C) Dendrogram created from five randomly chosen ORs per perceptual descriptor.

(D) Dendrogram from the five best overall predictors including OR and chemical features per perceptual descriptor. Clustering is hierarchical and based on Euclidean distance (A) or the Jaccard distance (B–D). Cluster number (colored branches) inferred from gap statistic across bootstrap samples. [find the pdf for fine-resolution]

-source

I think, and I could be wrong, but it seems the big deal here is that they made a model for each receptor, instead of just making one model for all receptors. Whereas others have created an n-dimensional predictive space to collapse the behemoth of the chemosphere into a single equation, this team just reverse-engineered the receptors themselves.

They haven't found the odor code, but they did write 34 of them. We have hundreds of olfactory receptors. That's not everything, but we are definitely getting there.

What it CAN do? It can help us discover new chemicals, and also to discover substitutes for other chemicals that are expensive, rare, or ethically-troublesome (fear-pheromones from tortured cats for example).

What it CAN'T do? It can't predict how an odor will smell to you, as an individual. It can approximate, however, and pretty good. They mention only getting 20% of the human olfactome, or human olfactory receptor repertoire.

via UC Riverside: Joel Kowalewski et al. Predicting Human Olfactory Perception from Activities of Odorant Receptors, iScience (2020). DOI: 10.1016/j.isci.2020.101361

http://dx.doi.org/10.1016/j.isci.2020.101361

Post Script:

They mention something called the ATLAS dataset, but I don't know what that is, other than a proprietary data analysis software. Maybe it's their own dataset through ATLAS?

And for fun, I'll report that they do mention "substantive portion of odor identity arises early in the processing stream" which is a good way of describing the the two-layer perception process of olfaction.

The second layer, and this is the one that Asifa Majid tells us is influenced by culture, experience, and language: "It is likely that the remaining portion depends on experience-dependent modulation, supporting a downstream model with reliance on distributed neuronal networks for human perceptual coding."

Further: "Unlike the retinotopic and tonotopic patterning observed in the visual and auditory cortices, representing spatiotemporal properties of visual and auditory stimuli as they are processed at sensory neurons, piriform activity appears randomly distributed, without a clear mapping of physicochemical features (Stettler and Axel, 2009)."

Interesting: "In our analyses, the OR specialized for musk was not a top candidate for

musk predictions but contributed strongly to predictions of 'sweaty.'"

Perhaps because the model isn't "smelling" it among other calculated fragrant mixtures such as perfumes, but rather "in the wild?" 

Post Post Script:

Can't talk about the odor code without mentioning code smell, a term for when something is wrong with your code, but we're not sure what it is. 

https://en.wikipedia.org/wiki/Code_smell

Also, going deep on the topic here:

The Dream of Olfaction Prediction

http://www.limbicsignal.com/2020/01/the-dream-of-olfaction-prediction.html

Neuromorphic Buzzwords

Recent advances give theoretical insight into why deep learning networks are successful

Aug 2020, phys.org

https://techxplore.com/news/2020-07-advances-theoretical-insight-deep-networks.html

It's just like olfaction.

If you didn't know what a deep learning neural network was in 2015 when Hidden Scents came out, you do now. Face recognition? Deep learning. Speech recognition? Deep learning. Deep fakes?? You guessed it. 

But why would someone spend an entire chapter of a book on smell talking about brain-like computing systems? Because the little part of our brain that smells is about as close as you get to a deep learning neural network.

And the story goes like this -- Big data brings Dirty data, which then brings the curse of dimensionality. It's not like mammals->dogs->poodles. It's like "that dog that bit me one time" and "the kind of dog that likes kids" and "dogs that were selected to hunt rodents" and "coyotes" and "pet cemetary" and "totem poles." Imagine a spreadsheet that has just as many columns as it has rows. For every rule there's an exception. 

What you probably know as a "computer algorithm" is just a bunch of rules. But when every rule has an exception, algorithms don't work so good anymore. This is the curse of dimensionality. 

This is also the chemosphere being described. Chemicals are myriad and ever-changing. Any means of chemosensation will have to employ something closer to a deep learning network than to an old-fashioned computer algorithm of IF/THEN functions. And that's why our olfactory system could really be called the deep nose, and why olfaction will become the representative sense of the Age of Approximation born of the datapocalypse. 

This thought-provoking paper does a much better job describing these networks, and makes implications for their use in society:

Tomaso Poggio et al. Theoretical issues in deep networks, Proceedings of the National Academy of Sciences (2020). DOI: 10.1073/pnas.1907369117

The Origin of Artificial Olfaction

Researchers unveil a pruning algorithm to make artificial intelligence applications run

Apr 2020, phys.org

MIT researchers have a new and better way to compress models.

It's so simple that they unveiled it in a tweet last month: Train the model, prune its weakest connections, retrain the model at its fast, early training rate, and repeat, until the model is as tiny as you want.

In other words, in order to make a more efficient artificial brain, you grow it from scratch, like a person. 

This is a welcome development for artificial olfaction enthusiasts, because we won't see fully-functioning synthetic olfactory systems until we can first get a "lifetime" worth of autobiographical data for that system. 

That feeling you get when "the smell of grandma's attic" hits you, it will not work if you didn't have a grandma. The data used by an olfactory system, artificial or otherwise, will come not only from infinite odorous molecules and their physiochemical properties, but also from the limbic system. And not just a limbic system, it has to be one that is preloaded with physiological datapoints as they relate to different combinations of odorous molecules. That requires a lifetime of matching bodily experiences, social experiences, and ultimately autobiographical moments to odors. 

To decode olfaction is not so much a phylogenetic (species) problem as an ontogenetic (individual) problem. There is so much variety in the way we perceive smells, that to use a phylogenetic approach would exclude the majority of what makes smell such a powerful experience. It needs meaning; it is by nature subjective, not objective. In other words, it needs a subject, and in ways that other senses can do without (see object recognition, for example).

Image source: Hiroto Ikeuchi cyberpunk

Notes

Comparing Rewinding and Fine-tuning in Neural Network Pruning, arXiv:2003.02389 [cs.LG] arxiv.org/abs/2003.02389

Post Script

Artificial brains may need sleep too

June 2020, phys.org

"We study spiking neural networks, which are systems that learn much as living brains do," said Los Alamos National Laboratory computer scientist Yijing Watkins. "We were fascinated by the prospect of training a neuromorphic processor in a manner analogous to how humans and other biological systems learn from their environment during childhood development."

Watkins and her research team found that the network simulations became unstable after continuous periods of unsupervised learning. When they exposed the networks to states that are analogous to the waves that living brains experience during sleep, stability was restored. "It was as though we were giving the neural networks the equivalent of a good night's rest," said Watkins.

Stand Corrected on Smelling Robots

It’s already happening, in Edinburgh: Robot noses are taking our jobs – doctor’s jobs, that is. We already know that dogs can tell when you’re sick just by the way you smell. And maybe less of us know that dogs can smell the place on your body where the sickness comes from, like if you have some kind of cancer hiding inside you. Alexendra Horowitz went into great detail about that kind of magic in her book on dogs’ smell.

It’s different now, however, because these aren’t dogs but computers. To get a bit more specific, it’s a gas-sniffing machine (called a GC-MS spectrophotometer, gulp, the de facto artificial smelling machine) combined with a special kind of ‘computer’ called a neural network.

If you’ve ever read my book or my blog or you’ve not been under a rock for the past 5 years, you’ve heard of neural nets. They are these magical new* ways of computing that created Google’s DeepDream and AlphaGo and every other headline where a computer did something we never thought a computer could do (like to dream and make art, yes). And now they smell.

But not really; we’ll get to that. First, it’s important to point out that this news comes from Nvidia, who makes GPU chips, which are not CPU chips. The computers we use, and have used forever, run on CPU chips – that’s the way it’s always been. Then the part of the computer that does the graphics, a GPU, started to do more and more of the computing (CPU).  We heard about GPUs first in regards to video games, but then because of Bitcoin because they use tons of interconnected GPUs to do their mining (and yes all those gamers got pissed because the price of GPUs exploded in tandem with the cryptocurrency bubble).

GPUs do more than provide smooth, clear graphics for your video games or authenticated cryptocurrency for your third world country blackmarket terrorist druglord network. They make a computer more like a brain, and hence the term artificial neural network.

Brains are all interconnected – neurons and axons, hub and spoke. Neural nets, with their GPU-neurons, approximate a brain better NOT because of a better algorithm software, but a better hardware. And with all this, we’re seeing artificial intelligence explode – I hate to say it – but it’s happening just like Ray Kurzweil said it would.

So after beating a human at Go, after successfully debating a human on the benefits to humanity of space travel, after creating its own language that humans can’t even understand, after detecting health abnormalities in patients’ xrays better than doctors, and after being able to play paper rock scissors so well that it can predict what we will throw before we throw it and hence beat us every single time – the damn thing now smells. (The paper rock scissors example is simply processing speed – the system sees our hands about to make a shape, and counters so fast that to us it seems like it happened ‘at the same time.’)

But let’s not get ahead of ourselves here. First thing to note is that this thing is not smelling. It’s been trained to recognize a very small subset of molecules related to cancer.  Whereas humans can detect any volatile organic molecule (rough definition), this thing can only detect what we’ve trained it to detect.**  And this is not the first time a system has been trained to smell – it happens a lot with bomb sniffing, for example, and artificially augmented bomb sniffing remote control cicadas are also real. Anyway, next is where I have to geek the F out: the part of us that smells IS a neural net.

Granted, our whole brain is like a neural net (yes, hence the use of the words ‘neural net’). But the part of our brain that specifically processes, or organizes the electrical signals from molecular contact and turns them into electrical signals for perception, is a pyramid-structure network (it’s called the piriform cortex for that reason, but it’s also known as the olfactory cortex) where hundreds of receptors are whittled down to a few signal fibers. And if you’ve ever seen a picture of a neural net, well, it’s the same thing.

This is one of the underlying themes in my book, and one of the reasons I was compelled to write it. Our sense of smell, the most under-studied of all the senses, is actually more like the most advanced technology there is right now, that being artificial intelligent brain-like systems. I like to call them intelligentities (which is gender neutral btw, and also neutral on some other thing we aren’t even upset about yet, where we make a biased distinction between humans and computers).

Although it seems like we’re making serious progress in this area, I still assert that studying olfaction is an ideal way to optimize these kinds of systems. Until then, you can rest assured that although these things can already do basically everything better than you, they still can’t smell.  (And many of us will have to wonder – is that a bad thing? I.e., will humans in the distant future, once we have the option, will they still want to smell?)

*Marvin Minsky et al were talking about neural nets in the early 80’s but the hardware wasn’t there yet to make them sing.

**Artificial Intelligence can only do what we train it to do. And this is a major part of the inherent biases that show up in these programs, and the reason we need to do a better job of choosing their training programs and then testing these programs to see if they discriminate and against who. Search up this phrase to find out more – ‘man is to computer programmer as woman is to homemaker.’

Notes:

Image source: Olfactory Bulb (aka non-artificial neural network)

Image source: Artificial Neural Network

Article source:

AI Can Smell Illnesses in Human Breath

June 2018, nvidia.com

Wiki:

Artificial Neural Network

Olfaction Meets AI

Headline reads like this:

TEDGlobal: The computer that can smell explosives

Aug 2017, BBC

And inside:

Nigerian Oshi Agabi’s modem-sized device - dubbed Koniku Kore - could provide the brain for future robots. It is an amalgam of living neurons and silicon, with olfactory capabilities — basically sensors that can detect and recognise smells.

And an explanation:

While computers are better than humans at complex mathematical equations, there are many cognitive functions where the brain is much better: training a computer to recognise smells would require colossal amounts of computational power and energy, for example.

The prototype device shown off at TED - the pictures of which cannot yet be publicly revealed - has partially solved one of the biggest challenges of harnessing biological systems - keeping the neurons alive. "This device can live on a desk and we can keep them alive for a couple of months," Agabi told the BBC.

And what do we think about this?

As much as this story is pretty nuts (if the sentence “They can live on a desk” doesn’t make your head spin…), it’s all too common a story in the tech world. Not that it’s fake news or anything, but let’s just say it is misleading to talk about “smelling robots” in this way.

The less interesting truth is that they can only be trained to smell specific molecules, not even signatures, or combinations, of molecules. A system able to smell “anything that might come up,” and able to use that information for something important, such a system could not be trained. Well, hmmm,  we get trained to do this from birth, in fact we are already learning about our olfactory environment in utero.

So if we want AI to meet olfaction, what we need to do is keep them alive for a lifetime, and give them a body, and friends and a job. You know, just like a real person. They would need to learn from the ground up, just like a real person.

However ---

There is a point being made here by Mr. Agabi that is totally in-line with the thesis of Hidden Scents. The way we use computers today will eventually be supplanted by something else. Traditional computation will still be useful, but something else will take us beyond the capacities of today’s technology (whole lotta talk in the sci-fi sphere of quantum computing, for example).

As of now, neural networks are taking us in a new direction. Granted they were used back in the 80’s, but only recently have they become a marked change in computing technique. (I like to note here the contemporaneous link between the architecture of neural networks and how it is the same thing used to mine bitcoins – the processor is no longer the key component, it’s how many graphics cards you have all wired together.)

The olfactory bulb, the crux of the olfactory system, from an information processing point of view, is a model neural network. And the fact that it’s already connected to the limbic system – the thing that makes us move, the thing that makes our bodies work, and even our emotions – this makes it a model system for so much more.

*Anyone with more comp sci knowledge than me please feel free to correct as I am no expert and speaking in pretty broad, possibly misunderstood, terms.  

Cracking the Black Box

New technique elucidates the inner workings of neural networks trained on visual data

Jul 2017, phys.org

"Deep learning" and "neural networks" are terms that have become firmly planted in our popular lexicon. They all refer to the same thing, which is an artificial brain-like thing that teaches itself via feedback loops. I talked about this in Hidden Scents because the way our brain decodes olfactory information is a lot like the way these deep learning networks process their own big data.

This deep learning approach is way more effective than traditional computing for lots of problems like facial recognition or natural language translation. They're also really good at handling Big Data, you  know, like all that stuff cybercriminals keep stealing for ransom? Thing is, once these networks 'figure out' how to do whatever it is that they do, we have no idea how they did it.

Usually, with traditional programming, we write the code, so we know what it does and how. With this, the network essentially writes its own program, and since it seems to know what it's doing, we don't ask how. We just take the results.

Until now. This is one of the researchers, quoted in phys.org:

"We catalogued 1,100 visual concepts—things like the color green, or a swirly texture, or wood material, or a human face, or a bicycle wheel, or a snowy mountaintop," says David Bau, an MIT graduate student in electrical engineering and computer science and one of the paper's two first authors. "We drew on several data sets that other people had developed, and merged them into a broadly and densely labeled data set of visual concepts. It's got many, many labels, and for each label we know which pixels in which image correspond to that label."

Here is the link to their work.

image source

Language Models

image Paramount Comedy Channel

Yann LeCun in this talk for the Wired Business Conference 2016, title AI Arms Race, notes how the best natural language models are now deep learning based. At Limbic Signal we find this funny because our sense of smell works akin to the deep learning model, and yet the language of smell is an unwieldy concept. Alas, it is unwieldy because we measure wieldliness by classical means. We have entered the age of approximation, however.

Bad Information

I'm not sure exactly how they got this image, but it sure looks like it came from the Google Deep Dream project where a deep learning network was asked to 'dream' about images and produce 'overpreceived' images, which look a lot like hallucinating on psychoactive mycotoxins.

Is there such a thing as Bad Information? If so, what is the difference between Good and Bad? How do we know that difference?

Artificial intelligence, but information theory in general, is a common theme in Hidden Scents. How can you not write about it these days? We are computers. At least, we are becoming computers. Or they us. At least, that's what say the analogies we use to make sense of our world. Do we know anything aside from the analogies we use? (We should probably be asking Douglas Hofstader about that one)

Back when pneumatics was the technology du jour, we thought the nervous system worked according to pressure in the nerves. That was correct for the circulatory system, but the utility of that analogy ended there. Eventually, the computer analogy will run out, but until then, we are computers. And these days, specifically we are computers learning to recognize patterns in our environment using forward-feebacked layers of feature detection.

This brings us to the premier of a new infotech textbook.

“The Deep Learning textbook is a resource intended to help students and practitioners enter the field of machine learning in general and deep learning in particular.”

Deep Learning, An MIT Press book

Ian Goodfellow and Yoshua Bengio and Aaron Courville, 2016

It's a textbook, so it's too technical for the interested layperson. But there is some goo dintroductory materials that could help straighten things out for people who want to know what it is, but don't have the contect-specific knowledge to digest the whole thing.

Here's from the chapter on Information Theory:

“Likely events should have low information content, and in the extreme case, events that are guaranteed to happen should have no information content whatsoever.

“Less likely events should have higher information content.

“Independent events should have additive information. For example, finding out that a tossed coin has come up as heads twice should convey twice as much information as finding out that a tossed coin has come up as heads once. “

The text then goes on to translate these maxims into mathematical formulae.

Sometimes someone says something and I'm like, wow, that was really stupid. But then later on, when I try to think about -why- it was stupid, I find it difficult to articulate. Above we have a good rationale for explaining why a particular statement is 'stupid' or not. It depends on how much information it has. And this is how we measure that information. In laymen's terms, we would call this the Captain Obvious principle. If you just said something that everyone already knows or should expect, but you said it like it's got good information value (as if nobody knows or expects it) then that would come across as stupid.

There we go again, turning a branch of applied mathematics into a magnifying glass for human behavior; probably not what the authors of this text intended to be done with their work.

Anyway, I like the word hard-coding. They use it to describe the 'older' way of writing-in knowledge about the world into a program (instead of 'letting the program figure it out for itself,' as these newer deep learning programs are done).

They point out in the introduction that "A person's everyday life requires an immense amount of knowledge about the world. Much of this knowledge is subjective and intuitive, and therefore difficult to articulate in a formal way. Computers need to capture this same knowledge in order to behave in an intelligent way. One of the key challenges in artificial intelligence is how to get this informal knowledge into a computer." Instead, when computers get their own data, by extracting patterns from raw data, this is known as machine learning. Deep learning is a type of machine learning.

Still, figuring out which details are valuable and which are inconsequential is the hardest part. -Disentangling- is a word emphasized by the authors. That's a favorite word in Hidden Scents as well. So is inextricable, the information-opposite of disentangle. So is disambiguate, the big brother of disentangle.

If you're into this stuff, and a bit more on the application side than the theoretical side, you might want to check this book out. And if you're just into machine-generated hallucinations, or if you've ever tripped on psilocybic mushrooms and want to see something reminiscent - very reminiscent - unnervingly reminiscent - check out the front cover.

preview the Google book

notes:

Analogy as the Core of Cognition, Douglas Hofstadter, Stanford lecture, 2006

[video link]

Bad Information vs Good Information

Deep Learning, An MIT Press book by Ian Goodfellow, Yoshua Bengio, Aaron Courville, 2016

I'm not sure exactly how they got this image, but it sure looks like it came from the Google Deep Dream project where a deep learning network was asked to 'dream' about images and produce 'overpreceived' images, which look a lot like hallucinating on psychoactive mycotoxins.

Is there such a thing as Bad Information? If so, what is the difference between Good and Bad? How do we know that difference?

Artificial intelligence, but information theory in general, is a common theme in Hidden Scents. How can you not write about it these days? We are computers. At least, we are becoming computers. Or they us. At least, that's what they say. Do we know anything aside from the analogies we use? (We should probably be asking Douglas Hofstader about that one)

Back when pneumatics was the technology du jour, we thought the nervous system worked according to pressure in the nerve fibers. That was correct for the circulatory system, but the utility of that analogy ended there. Eventually, the computer analogy will run out, but until then, we are computers. And these days, specifically we are computers learning to recognize patterns in our environment using forward-feebacked layers of feature detection.

This brings us to the premier of a new infotech textbook.

“The Deep Learning textbook is a resource intended to help students and practitioners enter the field of machine learning in general and deep learning in particular.”

Deep Learning is a (new) textbook, so it's too technical for the interested layperson. But there is some good introductory materials that could help straighten things out for people who want to know what it is, but don't have the context-specific knowledge to digest the whole thing.

Here's from the chapter on Information Theory

“Likely events should have low information content, and in the extreme case, events that are guaranteed to happen should have no information content whatsoever.

“Less likely events should have higher information content.

“Independent events should have additive information. For example, finding out that a tossed coin has come up as heads twice should convey twice as much information as finding out that a tossed coin has come up as heads once. “

The text then goes on to translate these maxims into mathematical formulae.

***

Sometimes someone says something and I'm like, wow, that was really stupid. But then later on, when I try to think about why it was stupid, I find it difficult to articulate. In the text quoted above, we have a good rationale for explaining why a particular statement is 'stupid' or not: It depends on how much information it has. And this is how we measure that information.

In laymen's terms, we would call this the Captain Obvious principle. If you just said something that everyone already knows or should expect, but you said it like it's got good information value (as if nobody knows or expects it) then that would come across as stupid.

There we go again, turning a branch of applied mathematics into a magnifying glass for human behavior; probably not what the authors of this text intended to be done with their work.

***

Anyway, back to the text. I like their word “hard-coding.” They use it to describe the 'older' way of writing-in knowledge about the world into a program (instead of 'letting the program figure it out for itself,' as these newer deep learning programs are done).

They point out in the introduction that "A person's everyday life requires an immense amount of knowledge about the world. Much of this knowledge is subjective and intuitive, and therefore difficult to articulate in a formal way. Computers need to capture this same knowledge in order to behave in an intelligent way. One of the key challenges in artificial intelligence is how to get this informal knowledge into a computer." Instead, when computers get their own data, by extracting patterns from raw data, this is known as machine learning. Deep learning is a type of machine learning.

Still, figuring out which details are valuable and which are inconsequential is the hardest part. Disentangling is a word emphasized by the authors. That's a favorite word in Hidden Scents as well. So is inextricable, the information-opposite of disentangle. So is disambiguate, the big brother of disentangle.

If you're into this stuff, and a bit more on the application side than the theoretical side, you might want to check this book out. And if you're just into machine-generated hallucinations, or if you've ever tripped on psilocybin mushrooms and want to see something reminiscent – very reminiscent – unnervingly reminiscent – check out the front cover.

preview the Google book

notes:

Analogy as the Core of Cognition, Douglas Hofstadter, Stanford lecture, 2006

[video link]

Open Up

Image source

can someone figure out how to turn this into an artificial nose pleaes

Microsoft Open Sources Its Artificial Brain to One-Up Google

WIRED, Jan 2016

The Next Generation

The next generation of artificial intelligence is here; we’re teaching robots how to think like humans more and more each day. You might not call them robots, but instead artificial intelligence programs tasked with simple operations like visual recognition or speech recognition. They are different from conventional AI programs in that they learn how to do things instead of being told what to do. So, in short, these programs are a lot more like humans in that they have a new way of “learning,” and it happens to be a lot like the way our nose-brain makes sense of the world.

This new AI approach is called, in shorthand, ‘neural networks’ or ‘deep learning.’ Our sense of smell works a lot like a neural network, putting together various layers of recognition, until entire episodes of experience are encoded or released.

These neural networks have been coming up in the news quite often as of late, so I thought I’d re-post some of the good explanations and examples, all of which come by way of Google Labs and Wired magazine. If you’re interested in these things, check out some of Hidden Scents, as it comes around to the concept of neural networks quite often.

Wired writer Margaret Rhodes writes a piece for those wondering "what the f*@k neural networks are and how they work," but she also links us to Daniel Smilkov, a member of Google’s Big Picture Research Group, and Shan Carter, who creates interactive graphics for The New York Times, who both wanted to teach people what these things really are. Play with their amazing interactive here: http://playground.tensorflow.org

Later on, another article by the same Wired writer, Margaret Rhodes, shows us how Google’s deep learning AI plays such a mean game of Pictionary. And again, notice the trend towards managing ambiguity that these new methods are so good at.

“But the game’s accuracy, while impressive, isn’t what makes it a powerful learning tool. It’s how, by observing the way Google responds to your doodling... The lesson: To understand the whole, neural networks need pieces of data that not only connect but build on one another, piece by piece.”

Notes:

Neural Nets Got You Confused? Try This Interactive Chart

WIRED, Apr 2016

Woah, Google’s AI Is Really Good at Pictionary

WIRED, Nov 2016

Play with Neural Networks

http://playground.tensorflow.org

On Interdisciplinary Studies

Microsoft’s Non-Non-Gendered Chatbot, Tay, 2016

 

To study the “language of smell” is to thread together the studies of many other fields. As a subject, the language of smell can spread into territories from proprioception to civil engineering. Regarding contemporary problems, the study of language and olfaction together can instigate new insight into fields like artificial intelligence, and even prompt questions about what it means to be human in the face of a technologically immersive world.

The olfactory bulb is a model neural network, and one that has scientists stumped still today. Despite having reverse-engineered vision, hearing, and even tactile sensation, nobody knows how to artificially code olfaction. After the brain tendrils in your nose are activated, the next stop at the olfactory bulb turns those signals into a buffet of information to be processed by the limbic system and rendered into an olfactory experience. That interchange at the olfactory bulb is still shrouded in mystery, but its neuronal architecture very closely resembles the layered networks used in artificial intelligence and machine learning today. (These are also called deep learning networks.)

As these forms of artificial intelligence become more pervasive, we are forced to reckon with what it means to be human vs machine. Already, with the need for non-gendered intelligentities (note Microsoft’s recent chatbot, which twitter turned into the dregs of society within 24 hours, was a “teenage girl,” not to say that it wouldn’t have been more successful if it was non-gendered, just that I was surprised when she was debuted that she was a definitive “she”), with advances in artificial reality simulation, in neural-interfaced prosthetic bodyparts and biocomputing insectobots, we are daily being asked which parts of “being alive” we want to keep, and which ones we want to offload to our [eventual overlords , jk].

To investigate both what it means to smell something, and how we communicate that experience, is to dive deep into the human, beyond the thinking parts and into the limbic, the emotional, animal parts.  These parts are so far inside our phylogenetic history that it’s hard to bring them to light in an age of so much knowingness and clarity. And to articulate these parts requires something less of a science and more of an art, which is exactly where the language of smell falls on the spectrum of functionality. (No wonder stuff like this gets no funding…see below.)

From a recent article on interdisciplinary research, an echo :

"One of the biggest advantages of interdisciplinary research is that it can generate new ways of looking at existing problems," said Professor Bromham, from the ANU Research School of Biology.

-phys.org

Notes:

Despite advantages, cross-discipline research has a lower chance of receiving funding

phys.org, July 2016

Pre-Cognition

The Triune brain, a visualization of the evolution of the human brain. Illustration by Joe Scordo for Hidden Scents: The Language of Smell in the Age of Approximation

Parts of the primate brain are made to deal with any potential situation. The way these highly adaptive brain parts work are by using recurrent loops that interfere with each other, in what is being called a “reservoir” network.

Perhaps we see headlines written like this because artificial intelligence work is typically performed by first predicting all potential situations (that is, until recently, with the advent of ‘deep learning’ techniques). In other words, the idea that a brain, or part of a brain, is designed to deal not with predictable situations but novel ones, is counter to this prevailing predictive technique.

I’d like to make a link here between this adaptive behavior and the fact that our sense of smell is not pre-coded but a blank slate. We do not have pre-existing preferences for smells, and the pattern of olfactory receptors in the nose seem to have no discernable pattern whatsoever because they're meant to learn anew for every creature and for every situation. This is because the way we interact with our organic environment is so complex, there can’t be a set of rules that work for every potential situation.

Smell is part of the mammal brain, and not rational thinking is part of the human brain. (Note that the mammal brain is not the same as the primate brain, which is only used as a term here to disambiguate it from the human brain…semantics!) Granted, primates have a prefrontal cortex too, and we have learned that there is not as much difference as once thought between humans’ and other animals’ brains. Nonetheless, smell is the animal inside us, and a link to our evolutionary past, and to a world much less predictable than the one we inhabit today.

Post Script

The primate brain is 'pre-adapted' to face potentially any situation

phys.org, Jun 2016