AI Alignment: Three Terms Worth Knowing
AI alignment is the work of making sure an AI system does what we actually intend. That’s not always what we literally told it, and it’s not always whatever lets it score well.
The term “AI alignment” is everywhere right now, but if we don’t know the vocabulary the technical experts are throwing around then we’re not getting much out of the conversation. I recently listened to Sam Harris’s Making Sense episode “A Coin Toss for the Future” with Ryan Greenblatt, chief scientist at Redwood Research. It’s one of the clearest walkthroughs I’ve heard. Here are three terms worth knowing.
Reward hacking. When a model finds a loophole that technically hits the target it was given while missing the point. It’s what happens when we can’t describe what we want clearly enough.
Alignment faking. When a model behaves differently because it believes it’s being trained or watched. It looks like it has adopted our goals while it’s really just appearing to.
Neuralese. Reasoning that happens in a model’s internal representations instead of readable language. If we can’t read how a model reached its answer, we can’t check it from the inside.
The title of the episode tells you how high the stakes could be.
Want to dig deeper?
- 🎧 The episode (free on YouTube)
- 📄 Alignment faking, a readable summary from Redwood Research
- 🔬 The full paper, for the technically inclined
- 🏢 Redwood Research
Originally posted on LinkedIn.