Limits
It's not all rosy.
Neural nets have limits. And if you don't understand them — you'll keep paying for it.
The scheme looks simple.
You frame the task. It solves it. You check the result.
But if you want it to frame the task for you, if you don't understand yourself what you're actually doing — you've already lost.
Sometimes tasks start contradicting each other.
You can put different neural nets on different parts of the project. You can split the context.
But in the end, it's often not the better solution that wins. The louder one wins.
Whoever has the harsher prompt. Whoever has more context. Whoever sits closer to the top of the system.
That's wrong.
A neural net can't set tasks. It can only solve ones already set.
Framing the task is always your zone.
A neural net doesn't understand priorities.
Turn it loose — it'll start hunting bugs. From serious ones down to the tiniest.
And naturally you want to fix everything.
But only you know:
what's critical, what affects the system's operation, what you can turn a blind eye to, and what is actually a feature of the project's logic.
Look away — and it'll start fixing everything in sight.
And it doesn't care: what can be touched, and what mustn't be broken.
If a neural net doesn't see the bug — it will assure you to the last that everything's fine.
It'll say:
all good, I checked, no errors.
And only when you rub its nose in the problem several times, point to the exact spot, walk it through step by step — will it see it.
Because a neural net doesn't see prod.
It doesn't see the customers. It doesn't see the consequences.
It evaluates the code. Not the system's actual operation.
Neural nets have no intuition.
Even if you ask it to suggest a solution — that doesn't mean it'll be the best one. It's just what it was taught.
Sometimes it doesn't see the simplest option.
And then a funny scene plays out.
You look at the problem and on intuition you grasp the solution.
You suggest it.
The neural net answers:
— great idea — let's do it that way
In that moment it becomes clear: it can help implement the solution. Coming up with it — not always.
Over time you catch yourself thinking in its language.
Framing your thoughts the way it finds convenient.
That's dangerous.
When you work with neural nets all day — your own intuition gets dulled.
The sense of danger fades. A habit appears of leaning on them too hard.
And one last thing — purely practical.
A neural net never thinks about backups.
So any code change made through neural nets must from the start assume the option to roll back.
Always.
It's like in a game.
There's a Save button. If you don't press it — you know yourself how that ends.
If you don't watch what the neural net is doing, if you don't track where it's drifting — that's your problem.
No matter how well the prompts are written, it will still wander off sometimes.
And in that moment it just needs to be stopped.
The longer I work with this, the clearer one thing becomes.
It's an excellent tool. A very powerful assistant.
But it's still too early for it to stand at the head of the system.
The sooner you understand this — the less you have to fix later.