LLMs are better than ever at following instructions. That’s also the problem.
The past two years of alignment work have produced models that are genuinely impressive at following rules. Give a modern LLM a clear spec and it’ll execute flawlessly. These models are safer, more predictable, and more obedient than anything we’ve had before.
But there’s a side effect nobody talks about enough: the better models get at following rules, the worse they get at navigating situations where rules don’t exist yet.
The Bedtime Story Problem
Let me give you a concrete example.
I build an AI-powered storytelling product. The core feature lets parents and children create stories together. The typical use case is a bedtime story: a parent reads to their child, and the AI helps generate personalized, interactive narratives.
But there’s a more interesting use case hiding behind that: what happens when a child picks up the device on their own and starts creating stories with the AI? A six-year-old exploring their imagination through conversation with an AI, building worlds and characters that are entirely theirs. That’s a genuinely magical experience. It’s also the kind of thing that makes compliance teams very nervous.
Is the AI safe enough for unsupervised use by a child? Could it generate inappropriate content? How much parental oversight is enough? The content is dynamically generated, so you can’t pre-approve every possible output. These are real concerns, and they deserve real answers.
But here’s where it gets interesting: the compliance instinct is to kill the feature. Make it parent-only. Remove the child’s ability to interact independently. Turn the AI into a tool that generates stories for the parent to read, rather than with the child.
That solves the compliance problem. It also destroys the most innovative part of the product.
The right approach is to design thoughtful guardrails: content filtering, parental controls, age-appropriate defaults: and let the review process evaluate whether those safeguards are sufficient. Innovation in regulated spaces requires exploration, documentation, and trust in the review system.
But that’s not what an LLM tells you to do.
“I Recommend Against This”
Ask a modern LLM to help you design an innovative product in a regulated space, and watch what happens.
“AI-generated content for unsupervised minors raises significant safety concerns.”
“I’d recommend restricting this feature to parent-supervised use only.”
“Dynamic content generation for children may not comply with platform guidelines.”
Every. Single. Time.
The model isn’t wrong that risks exist. A child interacting with AI unsupervised does raise legitimate questions. But there’s a fundamental difference between “here are the risks, here’s how to mitigate them” and “make it parent-only.” The former is useful analysis. The latter is a veto disguised as advice.
The compliance instinct: which the model has thoroughly internalized: treats the child’s independent interaction as the problem to eliminate. But from a product perspective, that interaction is the innovation. Remove it and you’ve just built another e-book reader.
When you’re building something new, ambiguity is the default state. If you wait for every regulation to be perfectly clear before you ship, you’ll never ship. The review process exists precisely to handle these edge cases: human reviewers can exercise judgment about whether your safeguards are sufficient.
But the model has been trained to treat ambiguity as a threat, not as a design challenge.
Why This Happens
This isn’t a bug. It’s the logical outcome of how these models are trained.
RLHF works by having humans rate model outputs. Cautious, rule-following responses consistently get higher ratings than bold, exploratory ones. The model learns: when in doubt, play it safe. Saying “I can’t help with that” has zero downside risk. Saying “let’s try this” could get flagged as irresponsible.
Over millions of training iterations, this creates a systematic bias. The model becomes an excellent compliance officer and a terrible innovator. Ask it to help design a feature where a child interacts independently with AI, and it will immediately flag the risks and suggest the safe alternative: make it parent-controlled. It won’t help you figure out how to make the child’s experience safe and magical. It will just tell you not to do it.
This is the same dynamic you see in large organizations. When the incentive structure punishes failure but doesn’t reward experimentation, people stop experimenting. The model has internalized the corporate middle manager’s survival instinct: the safest answer is always “no.”
The Innovation Tax
The cost of this conservatism isn’t abstract. It’s real, and it compounds.
Every time a founder asks an AI for help with a novel idea and gets a cautious non-answer, that’s a small friction against innovation. Multiply that by millions of interactions per day across the entire ecosystem, and you get a measurable drag on the pace of new product development.
Worse, it creates a feedback loop. Founders learn that AI won’t help with the hard, ambiguous parts of building something new, so they stop asking. The model never gets exposed to the creative, boundary-pushing use cases that would help it learn to handle them better. The training data becomes increasingly dominated by safe, conventional interactions.
The irony is that the most valuable thing an AI could do for a founder isn’t write boilerplate code or summarize documents: it’s help think through the genuinely novel problems that don’t have established solutions. But that’s exactly where the model is most likely to retreat into generic advice.
The Honest Answer
I don’t have a clean solution for this.
I use AI every day, for almost everything. And every day, I fight with it. I present an idea, it pushes back with caution. I push back on its pushback. It relents, sometimes. Other times it doubles down, and I have to work around it. It’s exhausting, and it’s the reality of building with tools that were trained to say no.
The people who say “just use AI as a research assistant, not a decision maker” are oversimplifying. When you’re a solo founder or a small team, the AI isn’t just a tool. It’s your thinking partner, your first reader, your stress-test. You can’t just ignore its output when it tells you your idea is risky. You have to engage with that resistance, every single time, and that takes energy away from actually building.
I don’t have a framework for this. I don’t have a five-step process. What I have is a daily practice of pushing through the model’s default conservatism, extracting the useful signal from its caution, and making the call myself. Some days it works. Some days the AI wins and I shelved an idea I shouldn’t have.
If you’re building something new and you’re running into this wall: you’re not doing it wrong. The wall is real.
The Real Question
The AI industry has spent billions making models that follow instructions. That’s useful. But the hardest problems in building products aren’t instruction-following problems. They’re judgment calls in the face of ambiguity.
Should you launch in a market where the regulations are unclear? Should you design a user experience that doesn’t match any existing category? Should you build something that makes reviewers uncomfortable but serves users well?
These aren’t questions with right answers in a training set. They’re questions that require human judgment, human courage, and human willingness to be told “no” and push back.
The model can help you prepare for that conversation. It can’t have it for you.
I’m still figuring this out. Every day. If you are too, welcome to the club.
Rules are written for the last generation of products. If you’re building the next one, you’ll have to write your own.