Sasha Mudd, a philosophy professor at the Pontifical Catholic University of Chile, published an essay in Aeon on 1 October with a title that is also its thesis: Why real intelligence is something more than optimisation. I agree with it, and I think the part I find most convincing is already happening to all of us, in a very small and very ordinary way.
What Mudd says
The essay starts at a lecture where a computer scientist called large language models intelligent because they optimise, and keep getting better at optimising from past experience. Mudd accepts that this describes today’s AI well. Her objection is to the slide from “this is how the machines work” to “this is what intelligence is”, and then to “this is what we are”.
To make the point she goes back to an old dispute:
- Hume: reason is the servant of the passions. It can find facts and efficient means, but it cannot tell us which ends are worth pursuing.
- Kant: reason can ask whether an end is worth having at all. Being able to set our own goals and judge them is what he calls autonomy, and it is the source of human dignity.
Optimisation, in her reading, is the first kind of reason: a very good way of reaching goals that someone else supplied. The risk she sees is less that machines become sovereign and more that people stop using their own judgment, accepting predictions and defaults instead of asking what is worth wanting. If we define intelligence as optimisation, questions about ends become invisible, and whoever controls the objective settles them. She also argues that the usual technical talk about aligning AI to us misses a deeper danger: that we end up too aligned with systems that treat intelligence as optimisation.
The small version of the same problem
Mudd writes about ends and society. I want to look at something much smaller, which is how we phrase a request.
Anyone who uses an AI assistant every day learns, without anyone teaching them, what works. You give the context first. You say what format you want. You split the task in steps. You paste an example of the output you expect. You remove the ambiguity that a person would resolve with a glance. When the answer is bad, you do not argue with it: you rewrite the question until the machine gives you what you meant.
A made-up but typical example:
| |
The second version is not better because the AI is “smarter” about it. It is better because I did the work of making my intent legible to a particular kind of reader. That is a perfectly good skill, and I do not think we should be romantic about vague questions.
But look at the direction of the effort. We did the same thing with search engines: for years people typed keywords instead of questions, because that is what the engine handled well. Now it is the same with models, only deeper, because the model answers in sentences and so the habit looks like conversation. We are learning the dialect of the system, and the system is not learning ours in the same way.
flowchart LR
A[What I mean] --> B[Rephrase it the way<br/>the model handles best]
B --> C[Better answer]
C --> D[Habit: ask this way<br/>from now on]
D --> AWhy I think it matters
It is easy to say “this is just learning to use a tool”. I agree that most of it is, and the strongest objection to my point is exactly that: we learned to write for compilers, for databases and for forms, and nobody lost their autonomy. Being precise is a good discipline, and a prompt that makes me state my goal clearly sometimes helps me notice I did not have one.
The difference is where the adaptation stops. A compiler asks me for correct syntax. It does not tell me what program to write. An assistant that has an opinion about the best answer, a default tone and a preferred way of splitting a problem gently shapes the questions too. It is the same thing Mudd describes at a larger scale: when the objective is set by someone else, the question “is this the right goal?” quietly disappears. I do not say it happens every time. I say it is a habit worth noticing, because habits are exactly what we stop seeing.
As developers we are going further than prompts. We now write project instruction files, README sections and comments with an AI reader in mind, we choose names and structures that models handle well, and we sometimes pick a library because the assistant knows it. Some of that is simply good engineering. Some of it is us bending the work to the shape of the tool, and it is worth asking, now and then, which one it is.
My rule of thumb, for what it is worth: use the machine’s preferred way of asking for the means, and keep the question of the ends for yourself. If I catch myself taking a suggested goal as if it were mine, I stop and ask what I wanted in the first place.
Come to think of it, don’t you ever get frustrated with your favorite AI because, after trying to rephrase the question two or three times in the hope that it’ll finally work, the answer doesn’t satisfy you and seems to be about something else?
I find it convenient and practical for reviewing code and creating commit messages—you don’t risk forgetting anything.
Sources
- Sasha Mudd, Why real intelligence is something more than optimisation, Aeon, 1 October 2026
