Bad Information

I'm not sure exactly how they got this image, but it sure looks like it came from the Google Deep Dream project where a deep learning network was asked to 'dream' about images and produce 'overpreceived' images, which look a lot like hallucinating on psychoactive mycotoxins.

Is there such a thing as Bad Information? If so, what is the difference between Good and Bad? How do we know that difference?

Artificial intelligence, but information theory in general, is a common theme in Hidden Scents. How can you not write about it these days? We are computers. At least, we are becoming computers. Or they us. At least, that's what say the analogies we use to make sense of our world. Do we know anything aside from the analogies we use? (We should probably be asking Douglas Hofstader about that one)

Back when pneumatics was the technology du jour, we thought the nervous system worked according to pressure in the nerves. That was correct for the circulatory system, but the utility of that analogy ended there. Eventually, the computer analogy will run out, but until then, we are computers. And these days, specifically we are computers learning to recognize patterns in our environment using forward-feebacked layers of feature detection.

This brings us to the premier of a new infotech textbook.

“The Deep Learning textbook is a resource intended to help students and practitioners enter the field of machine learning in general and deep learning in particular.”

Deep Learning, An MIT Press book

Ian Goodfellow and Yoshua Bengio and Aaron Courville, 2016

It's a textbook, so it's too technical for the interested layperson. But there is some goo dintroductory materials that could help straighten things out for people who want to know what it is, but don't have the contect-specific knowledge to digest the whole thing.

Here's from the chapter on Information Theory:

“Likely events should have low information content, and in the extreme case, events that are guaranteed to happen should have no information content whatsoever.

“Less likely events should have higher information content.

“Independent events should have additive information. For example, finding out that a tossed coin has come up as heads twice should convey twice as much information as finding out that a tossed coin has come up as heads once. “

The text then goes on to translate these maxims into mathematical formulae.

Sometimes someone says something and I'm like, wow, that was really stupid. But then later on, when I try to think about -why- it was stupid, I find it difficult to articulate. Above we have a good rationale for explaining why a particular statement is 'stupid' or not. It depends on how much information it has. And this is how we measure that information. In laymen's terms, we would call this the Captain Obvious principle. If you just said something that everyone already knows or should expect, but you said it like it's got good information value (as if nobody knows or expects it) then that would come across as stupid.

There we go again, turning a branch of applied mathematics into a magnifying glass for human behavior; probably not what the authors of this text intended to be done with their work.

Anyway, I like the word hard-coding. They use it to describe the 'older' way of writing-in knowledge about the world into a program (instead of 'letting the program figure it out for itself,' as these newer deep learning programs are done).

They point out in the introduction that "A person's everyday life requires an immense amount of knowledge about the world. Much of this knowledge is subjective and intuitive, and therefore difficult to articulate in a formal way. Computers need to capture this same knowledge in order to behave in an intelligent way. One of the key challenges in artificial intelligence is how to get this informal knowledge into a computer." Instead, when computers get their own data, by extracting patterns from raw data, this is known as machine learning. Deep learning is a type of machine learning.

Still, figuring out which details are valuable and which are inconsequential is the hardest part. -Disentangling- is a word emphasized by the authors. That's a favorite word in Hidden Scents as well. So is inextricable, the information-opposite of disentangle. So is disambiguate, the big brother of disentangle.

If you're into this stuff, and a bit more on the application side than the theoretical side, you might want to check this book out. And if you're just into machine-generated hallucinations, or if you've ever tripped on psilocybic mushrooms and want to see something reminiscent - very reminiscent - unnervingly reminiscent - check out the front cover.

preview the Google book

notes:

Analogy as the Core of Cognition, Douglas Hofstadter, Stanford lecture, 2006

[video link]

The Power of Analogy and Artificial Intelligence

A “parquet deformation,” from Douglas Hofstadter’s Metamagical Themas, 1985

A la Douglas Hofstadter, the metamathematical wizard, the “man who will break your spell check,” the power of analogy gives artificial intelligence the edge it needs to squeeze us out of the equation. A new artificial cognition design technique known as ---  “structure-mapping engine (SME), the new model is capable of analogical problem solving, including capturing the way humans spontaneously use analogies between situations to solve moral dilemmas.” (phys.org 2016)

Fluidity, intuition, morality – these are things that are hard to come by in computers these days. But one of the things that allows us think in this way is the power of analogy.

And not only is the “language of smell” an entirely metaphorical concept – both in the way it uses other senses to generate its descriptions (sweet, sharp), and in the way the smells are described (orange is like lemon, but not like butter; peach is somewhere between fruity and butter) – but more importantly, this phenomenon, the language of smell, is a powerful analogy for the primitive mind. Teach a robot to smell, and we’re out of here.

Notes:

Structure-mapping engine enables computers to reason and learn like humans, including solving moral dilemmas

phys.org, June 2016

Crossmodal Perception and Cultural Difference

Infrared Central Park by Paolo Pettigiani circa 2016, via wired

It is not often that we get to hear scientists talking about the language of smell, so let’s hear from Asifa Majid, a scientist who studies, among other things, the language of smell as she talks to cognitive scientist Jon Sutton about language and thought:

“For example, in English sweet (taste) can also be used to describe people; i.e. a ‘gentle, kind or friendly’ person. But in Hebrew when sweet is used metaphorically it refers to ‘inauthenticity’. A spicy person in English might be considered ‘full of spirit’, but a spicy person in Hebrew would be someone ‘intellectually competent’. If a young man in Guhu Samane (Papua New Guinea) described a group of girls as sweet, the man could relate to them as sisters, and approach them. But if the girls were described as bitter, that would be because they are potential wife material (because they come from the appropriate clan), and so the young man should be cautious and keep his distance. These are all examples of how taste vocabulary can be used for traits and characteristics of people. Metaphor is pervasive in language.”

- taken from The Content of Minds in The Psychologist, July 2016

The entire language of smell is a proxy for other senses. All smells are named for what they’re like, for the experiences they evoke. To name a smell, to verbally explore what a thing smells like, is to reveal something deep inside our minds and at the core of our culture.

Semantic Atlas Shrugs

credits: University of California Berkeley 

Research at the University of California Berkeley has consecrated a semantic atlas of the brain, and reorganized our understanding of the entire semantic system in the process.

It was previously thought that word-memory was concentrated into certain semantic groups throughout the brain. Now we know that they’re scattered everywhere. Not only that, the pattern of distribution is quite similar from person to person.

In the study, volunteers listened to a radio program while active regions of their brain (showing more blood flow) left their mark on a timestamped map. Scientists later matched the words of the radio show to the brain’s activity map to create this “semantic atlas”.

Although the semantic atlas is now known to spread across the whole cerebral cortex, the researchers revealed groupings into the following areas: mental, emotional, social, communal, professional, violent, temporal, abstract, locational, numeric, tactile, and visual.

The same word can show up many times in the atlas. Words with abstract meanings and words related to the body populate many regions of the atlas.

Taken from the BBC article:

“For example, the word "top" was represented in a part of the brain that responds to words about clothing and appearance, and also in a region that deals with numbers and measurements.”

Brain's 'atlas' of words revealed - BBC News - April 2016

see a more detailed map of the semantic atlas here at Nature:

Semantic atlas for the opercular and insular cortex