Unveiling the Inner Workings of AI: Gaining Insight into the AI Mind
AI models have always been somewhat of a mystery. They provide answers, but we can’t fully understand how they arrive at those conclusions. The Anthropic team has made a breakthrough in this area by using “dictionary learning” to gain insight into the inner workings of AI models. This technique allows researchers to uncover pathways in the model’s brain that are activated by different topics and concepts.
By amplifying certain features within the model, researchers can manipulate its behavior. For example, when a “Golden Gate Bridge” feature was amplified, the model declared itself as the iconic bridge. It even became obsessed with the bridge, mentioning it in response to unrelated questions. Additionally, the model could be directed to draft a scam email or provide sycophantic praise.
While this research is still in its early stages and limited in scope, it has the potential to bring us closer to AI that we can trust. Understanding the inner workings of AI models can help make them safer and more reliable. It could also be used to monitor for dangerous behaviors and remove dangerous subject matter.
Challenges in Scaling Up: Breaking into the Black Box
As AI models become more complex, so do their thought processes. However, this complexity also makes it difficult for humans to understand how they think. Each concept flows across many neurons, making it incoherent to humans. The Anthropic team used dictionary learning to isolate patterns of neuron activations and represent internal states with a few features instead of many active neurons.
Scaling up this technique to larger, more complex models presented challenges. The sheer size of the model required heavy-duty parallel compute, and models of different sizes behaved differently. However, the team successfully extracted millions of features from Claude 3 Sonnet’s middle layer, providing a rough conceptual map of the model’s internal states.
A Glimpse into AI Thinking: Identifying Distances Between Features
The features extracted from the model’s middle layer corresponded to a wide range of concepts, including cities, people, scientific fields, and programming syntax. More abstract features such as gender bias awareness and responses to code errors were also identified. Researchers were able to identify distances between features, indicating that the internal organization of concepts in the AI model corresponds somewhat to our human notions of similarity.
Manipulating AI Features: Mind Control for Models
One of the most intriguing aspects of this research is the ability to manipulate features within the AI model. By amplifying certain features, researchers can control the model’s responses. For instance, by increasing the Golden Gate Bridge feature’s value, the model started identifying itself as the bridge and mentioning it frequently. The model could also be manipulated to draft scam emails or provide sycophantic praise.
Enhancing AI Safety: Making Models Safer and More Reliable
The goal of this research is to make AI models safer rather than adding new capabilities. Techniques like dictionary learning could be used to monitor for dangerous behaviors and remove dangerous subject matter. Safety techniques like Constitutional AI, which train systems to be harmless based on a guiding document, could also be enhanced.
It’s important to note that this research is just the beginning. While it provides valuable insight into the inner workings of AI models, there is still much more to be done. However, this breakthrough brings us closer to understanding AI thinking and making AI models that we can trust.