AI Limitations & The Illusion of Explanatory Depth

The illusion of explanatory depth (IOED) is, put simply, the idea that we tend to know less than we think we know.

I was reminded of the IOED by this thought-provoking episode of Hidden Brain.

A famous example of the IOED involves asking study participants to draw a bicycle from memory. Go ahead, try it.

It’s often said that the best way to get better at something is to try to teach it to someone else. This is how you realize your own blind spots, the holes in your knowledge base.

I keep finding new ways to explain why human limitations are our greatest assets.

In the framework of creative writing, I’ve long encouraged the idea of “cherry picking” knowledge out of context for interesting outcomes. For artists, being something of a jack-of-all-trades is actually quite beneficial. You can pull bits of information from a wide range of areas of interest… but not any and all knowledge from those areas. That would be boring. This is why LLMs (Large Language Models like ChatGPT) operate very differently from the human brain—which is, so to speak, a SLM (small language model).

To err is human. In the battle against AI, our flaws are our greatest strength.

*

*

Back to the IOED. As an exercise— a writing prompt— write down a list of subjects that you believe you have a fairly strong knowledge about.

*

Here’s a list of thoughts/ideas you may (or may not) think you comprehend:

· How a ballpoint pen dispenses ink

· How zippers work

· Why seasons change

· How you fall asleep

· How friendships form and develop

· How the flow of conversations operate

· How we decide who to trust

· Why some people appear more charismatic

· What makes a person likeable

· How we further open “the moral circle”

· How habits are formed

· Why we procrastinate

· How the paradox of choice operates

· How our mood affects our decision-making process

· What makes a meeting productive

· Why traffic jams occur

· How grocery stores determine stocking placements

· What makes a person a good listener

· Why we remember what we remember

· How to convey tone in a text message

· Why family dynamics are often static over time

· How household chores are distributed

· What makes a space feel welcoming or not

· Why does an activity energize you

· Why does an activity deplete you

· What magnetizes people to each other

· How do we form opinions

· How do we change someone’s mind

· How do we decide what is worth our time

· Why do people self-sabotage

· What is the value of worrying

· What makes a person emotionally intelligent

· What is true loyalty

· Why are we more vulnerable with certain individuals

· How do we weigh consequences

· How do we determine or values

· How does timing affect feedback reception

· Why can we only hear truths from certain people

· What are the limitations of self-optimization

· Why do we try to keep up when we know it is impossible

· Why do our energy levels fluctuate throughout the day

· What makes a space feel like work vs. leisure or “play”

· How does weather affect our choices

· Why are some drawn more to mountains or forests or water

· Why do different people seem to need more exposure to the natural world

· What makes certain sounds soothing and others unpleasant

· Why do some of us thrive under pressure

· How do invisible disabilities (or being neurodivergent) shape daily life

· Why are some people more resilient

· Why do difficult life events affect people differently

· Why are setbacks easier for some to bounce back from

· Why are life transitions easier for some

· What is the value in talking to strangers

· What is the value of boredom

· Why do we need unstructured time

· How does cognitive load operate

· Is there really value in dedicated time to “do nothing”

· Why are so-called “weak ties” important

· Who do you think performs the most invisible [emotional] labor in your life

· What is the value of difficult conversations

· What makes a person feel heard

· What makes shared silence more or less awkward

· Why is the experience of pain so subjective

· What makes someone an “energy vampire”

· What skills are required to explain something complex in a simple manner

· How do you know when to pivot

· How do you sit with unreconcilable truths

· How do you self-regulate

· What determines a person’s tolerance for ambiguity

· Why are we a poor judge of our own self-awareness

· How do our present selves sabotage our future selves

· What leads to personal growth

· What is the real cost of your time

· What are reasons our choices often do not align with our values

· How do you unlearn something (such as problematic beliefs or bad habits)

· How do we go about making judgements for ourselves

· What do we learn from those who are not like us

· How do you want to be remembered vs. how do you expect to be remembered