The last two paragraphs apply not only to AI and ET, but also describe features of the human mind that affect decision making in many of us at times–lack of motivation and drive, and paralysis of decision making in the face of too many possible choices. But it gets worse: we know that a motivation can be hijacked by options that simulate the satisfaction that the motivation is aimed toward. Substance addictions plague tens of millions of people in the United States alone, and addictions to more subtle things, including certain kinds of information (such as e-mail), are prominent too. And few arts are more challenging than passing on motivation to the next generation, for the pursuit of a big idea. Intelligences that invent more and more interesting and absorbing technologies, that can better grab and hold their attention, while reducing impact on the world, might enter the opposite of a singularity.
What is the opposite of a singularity? The singularity depends on a mathematical recursion: invent a superintelligence, and then it will invent an even more powerful superintelligence. But as any mathematics student knows, there are other outcomes of an iterated process, such as a fixed point. A fixed point is a point that, when a function is applied, gives you the same point again. Applying such a function to points near the fixed point will often send them toward the fixed point.
A “societal fixed point” might therefore be defined as a state that self-reinforces, remaining in the status quo–which could in principle be peaceful and self-sustaining, but could also be extremely boring–say, involving lots of people plugged into the Internet watching videos forever. Thus, we as humans might want, sometime soon, to start laying out design rules for technologies so that they will motivate us to some high goal or end–or at least away from dead-end societal fixed points. This process will involve thinking about how technology could help confront an old question of philosophy–namely, “What should I do, given all these possible paths?” Perhaps it is time for an empirical answer to this question, derived from the properties of our brains and the universe we live in.