Embodied Wonder
Preventing Sensory Hijacking in the AI Era
I woke this morning, as I usually do, before the sunrise. The sun the Green Mountains to my east, gently easing themselves up from the basin of Lake Champlain.
The workload demand of these last few weeks has pulled hard at my attention and energy. Writing has been almost impossible, akin to lifting heavy objects with arms wrapped together in ace bandages. As a lifelong creative - performing artist, writer, entrepreneur, designer, poet, painter - this feeling is not unfamiliar. But it always raises a quiet terror when it happens. Will this feeling pass? When? Exhaustion itself can lead to a kind of fatalistic thinking: it will always be this way from now on. I have learned over the years not to believe everything I think.
And I have learned to listen, really listen, to what my body has to say. I don’t do it perfectly; probably never will. But years of trauma and other challenges catalyzed a sensibility that has been core to my decades of recovery. I don’t know whether the body keeps the score, but I am certain that our bodies contain maps, and if we are willing to explore them, they will lead us to powerful, unexpected, and wonderful places.
Recently I listened to a conversation on the podcast Possible between Reid Hoffman, Aria Finger, and their guest Kanjun Qiu, founder of Imbue, an AI lab focused in a central and unique way on questions of power, human agency, and the metrics of trustworthiness in the AI era. The conversation was wide-ranging and covered many of the core topics I am engaged with every day. I will be diving into these, one by one, in future writings.
A deep and longtime practitioner of meditation, Qiu describes AI as a “really strange intelligence.” Informed by her practice, she considers awareness, or consciousness, as a kind of substrate that generates our perceptual field, giving rise to our thoughts. Through meditation, as we become aware of the generator of our thoughts, we have the potential to be freed from its relentless thrall and the compulsivity of the chatter in our minds.
AI systems, Qiu explains, don’t have such a thing. There is no mechanism for reflection other than what is called “Reinforcement Learning from Human Feedback” (RLHF). As such, AI is not conscious, nor is it consciousness. “…in all these AI systems, the introspectability rounds to nothing.” In other words, where is the self that is observing?
As AI becomes increasingly present in our daily human lives, we are inclined as a species to trust it more. How is trust brokered, or built? It has me thinking about how we know what to trust, and what an embodied, somatic filter for truth would look or feel like - as in, how does truth feel in our bodies?
Qiu’s work at Imbue aims at this very question. She calls for her fellow technologists to center moral, ethical questions at the core of the development of the technologies we are building. “I think there’s a moral thing,” she says, “which is as technologists, we build incredibly powerful things that change people’s lives and affect their day-to-day, but we don’t have a variant of the Hippocratic oath…something I’ve been noodling on is like, what would the Ten Commandments for technologists look like?”
Qiu contemplates a sensation becoming increasingly common in our daily lives: why do some things feel more fake than others? “For example,” she says, “like if we’re scrolling Instagram at 3 am, that feels fake, but watching a great movie until 3 am feels kind of more meaningful, more real. Or if we’re eating processed food, it feels kind of fake. But if we’re eating really nutritious food, it feels more real…
“What’s going on? And I think there’s this thing that can happen to humans as we get more powerful technology that I’m calling sensory hijacking…technology hijacks these sensors that we have to determine actually how to live really meaningful and healthy lives. We have really good sensors for that. But they’re very, very hackable…and so one example of a commandment might be…thou shalt not hijack other people’s sensors. Another one might be…thou shalt not build systems that propagate false reality.”
I look up from my computer screen and out at the lake and risen sun to the east. A family of 20+ mergansers have dropped by for a visit and are scurrying along the shoreline in a coordinated dance of organized frenzy. It’s hilarious to watch. It’s always started by one duck. I am curious about what the prompt is: why does that one duck decide to take off and push through the water like a speedboat, and why do the others follow? Is the lead duck fleeing, or racing toward something? Do the others know? Or care? Are we like the mergansers, scurrying along the shore after a lead duck? Do we know why?
I feel the breath move in and out of my body. I feel a smile move across my face as I watch the mergansers huddle up by our dock.
My body relaxes. Yes. This is real.
- S
Source
Possible Podcast, “What It Takes To Trust AI” with Kanjun Qiu https://www.possible.fm/podcasts/kanjun/