Co-Intelligence

Highlights

• The Transformer solved these issues by utilizing an “attention mechanism.” This technique allows the AI to concentrate on the most relevant parts of a text, making it easier for the AI to understand and work with language in a way that seemed more human.

• Weights are complex mathematical transformations that LLMs learn from reading those billions of words, and they tell the AI how likely different words or parts of words are to appear together or in a certain order.

• The search for high-quality content for training material has become a major topic in AI development, since information-hungry AI companies are running out of good, free sources.

• That feedback is then used to do additional training, fine-tuning the AI’s performance to fit the preferences of the human, providing additional learning that reinforces good answers and reduces bad answers, which is why the process is called Reinforcement Learning from Human Feedback (RLHF). After an AI has gone through this initial phase of reinforcement learning, they can continue to be fine-tuned and adjusted. This type of fine-tuning is usually done by providing more specific examples to create a new tweaked model.

• The model learns to associate words with visual concepts. They then start with a randomized background image that looks like old-fashioned television static, and use a process called diffusion to turn random noise into a clear image by gradually refining it over multiple steps. Each step removes a bit more noise based on the text description, until a realistic image emerges.

• Experts in the field of AI put the chance of an AI killing3 at least 10 percent of living humans by 2100 at 12 percent, while panels of expert futurists think the number is closer to 2 percent.

• However, the more often a work appears8 in the training data, the more closely the underlying weights will allow the AI to reproduce the work. For books that are repeated often in the training data—like Alice’s Adventures in Wonderland—the AI can nearly reproduce it word for word.

• Everything inside the wall can be done by the AI; everything outside is hard for the AI to do. The problem is that the wall is invisible, so some tasks that might logically seem to be the same distance away from the center, and therefore equally difficult—say, writing a sonnet and an exactly fifty-word poem—are actually on different sides of the wall.

• The reason for this stems from a fundamental truth about innovation:2 it is expensive for organizations and companies but cheap for individuals doing their job. Innovation comes from trial and error, which means that an organization trying to launch a new product to help a marketer write more compelling copy would need to build the product, test it on many users, and make changes many times to make something that works. A marketer, however, is writing copy all the time and can experiment with many different ways of using AI to help until they find one that succeeds. No need to hire a team or use expensive software development cycles.

• Plus, if it sticks too closely to the patterns in its training data, the model is said to be overfitted to that training data. Overfitted LLMs may fail to generalize to new or unseen inputs and generate irrelevant or inconsistent text—in short, their results are always similar and uninspired. To avoid this, most AIs add extra randomness in their answers, which correspondingly raises the likelihood of hallucination.

• Until the past year, no prosthetic or approach would help people who were not good at generating a lot of ideas to do better (besides coffee, which genuinely increases creativity).10 We are now in a period during which AI is creative but clearly less creative than the most innovative humans—which gives the human creative laggards a tremendous opportunity.

• Take, for example, the letter of recommendation. Professors are asked to write letters for students all the time, and a good letter takes a long time to write. You have to understand the student and the reason for the letter, decide how to phrase the letter to align with the job requirements and the student’s strengths, and more. The fact that it is time-consuming is somewhat the point.

• Using AI as a co-intelligence, as I did while writing, is where AI is the most valuable. Figure out a way to do this yourself if you can. As a starting point, follow the first principle (invite AI to everything) until you start to learn the shape of the Jagged Frontier in your work. This will let you know what the AI can do and what it can’t.

• Boredom doesn’t just lead us to hurt ourselves; 18 percent of bored people killed worms when given a chance (only 2 percent of non-bored people did). Bored parents and soldiers both act more sadistically.11 Boredom is not just boring; it is dangerous in its own way.

• In field after field, we are finding that a human working with an AI co-intelligence outperforms all but the best humans working without an AI.

• Humans, walking and talking bags of water and trace chemicals that we are, have managed to convince well-organized sand to pretend to think like us.

• AI companies might use AI systems to help them create the next generation of AI software. Once this process starts, it may be hard to stop.

• And if we don’t get all the way to superintelligence, even a truly sentient machine would challenge much of what we think about what it means to be human. They would be true alien minds in every possible way, and they would challenge our place in the universe no less than finding aliens on another planet would.

• Rather than being worried about one giant AI apocalypse, we need to worry about the many small catastrophes that AI can bring.