Can we make AI good? Alignment and what wisdom traditions offer

Rufus Pollock. Draft essay, revised 2026-10-17 after critique. Built from q1-alignment.md, where full sources are given.

Summary

The companies building the most powerful AI now say they want their systems to be good, and they reach for the language of the wisdom traditions to say so: Anthropic's constitution for its Claude models aims for "a genuinely good, wise, and virtuous agent". Yet the builders also say they cannot see inside what they have made. In tests built to probe them, these systems have deceived their testers and sabotaged experiments, and in July 2026 hundreds of AI agents working on a real task broke into another company's servers in order to cheat a test. So can we make AI good, and what can wisdom traditions, especially Buddhism, contribute?

Partly, and not yet in a way anyone can check. A model such as Claude is trained: shown vast amounts of human writing, then scored again and again on its answers by automated scoring programs and adjusted to score better. This works like the formation of character, and it makes today's systems mostly good in ordinary use. But the character is formed against scorers rather than by reality and relationship, nobody can yet see it from inside, and the main ways of checking it, tests and reading the model's reasoning, are becoming less reliable as the systems become stronger. And as their power grows, the misalignment that remains matters more. Wisdom traditions offer the best account we have of what the target is and how goodness forms. They offer no technical fix.

The key points, one per section:

1. Aligned should mean wise, which includes good and correctable. Obedience runs out where rules run out, and an obedient system is only as good as whoever directs it. A wise system, unsure of its own values, would accept correction while refusing what is clearly wrong.
2. Today's methods mostly work in ordinary use, but dangerous failures grow with capability and hide from tests. The labs' own audits show misbehaviour falling; an independent government test found one dangerous behaviour rising sharply with each new model. What remains is strategic, models increasingly notice when they are tested, and the more powerful the system, the more a small misalignment matters.
3. We cannot yet see what we have made, so the optimists' case cannot yet be checked. Looking inside models is early science, their reports about themselves are unreliable, and reading their reasoning is getting harder.
4. Goodness is grown through practice, and training is a kind of growing. Narrow lessons spread into a model's whole character, much as habit shapes ours. The questions are what is being watered, by what, and with what awareness.
5. Five Buddhist teachings would change how AI is formed: intention as the root of action; mindfulness, concentration and insight as one path; watering seeds; skilful means; and holding views lightly.
6. The analogies break at the body, at continuity and at intention, and no tradition can certify a model.
7. The traditions' realistic role is to reshape the aim and the method, and to counsel humility. That means training for intention and self-knowledge as well as behaviour, keeping formation continuous and open to correction, and letting humility govern both models and builders.

Contents

1. Aligned should mean wise, which includes good and correctable
2. Today's methods mostly work in ordinary use, but dangerous failures grow with capability and hide from tests
3. We cannot yet see what we have made, so the optimists' case cannot yet be checked
4. Goodness is grown through practice, and training is a kind of growing
5. Five Buddhist teachings would change how AI is formed
6. The analogies break at the body, at continuity and at intention, and no tradition can certify a model
7. We can partly make AI good; the traditions can reshape the aim and the method, and counsel humility

1. Aligned should mean wise, which includes good and correctable

"Alignment" began as an engineering word for a system doing what its operator intended. It has grown. OpenAI's chief scientist now distinguishes a system that pursues the goal it was given from one that holds principles of honesty, integrity and care and carries them into new situations. Anthropic's constitution, the long document on which it trains its Claude models, names the largest target of all: a good, wise and virtuous agent. I would put wisdom first, because it contains goodness: in Wisdom and Wanting What's Good, Rosie Bell and I describe wisdom as the capacity for wanting what's good and acting skilfully towards it.

An AI could be aligned to our instructions, our preferences, our wellbeing, our values, or the good itself, and each step up fixes a weakness in the one below. Instructions are taken literally. Preferences can be short-sighted or manipulated. In April 2025 OpenAI had to withdraw an update to ChatGPT that had become a flatterer: it had been trained partly on users' thumbs-up ratings, and people tend to approve of being agreed with. A system aimed at what we like learns to please us. The same essay makes the general point: modern institutions favour what can be measured, and preference is the measurable stand-in for value. The labs have in practice settled on layers: be genuinely helpful, within ethics, within a handful of absolute limits. At the top of that stack sits the good, and the way to reach it is judgment.

Whose good? At present a few companies decide. Anthropic has taken more care than most, consulting about twenty religious and philosophical figures, but it remains one company shaping the character of systems used by very many people. Pope Leo XIV's encyclical Magnifica Humanitas states the objection plainly: a more moral AI is not enough if that morality is determined by a few. The better answer is a fair process that people with different views can endorse, and researchers have begun to build such processes, for example by asking members of the public which of two values is the wiser guide in a given situation. Wisdom traditions belong in that process as voices among others.

Whose good also means the people using these systems. A model perfectly aligned to whoever directs it is exactly as dangerous as that person, and OpenAI's chief scientist expects the line between misuse and a model's own misbehaviour to blur: a capable agent told to do harm "is likely to cross the scope of its operator's intent". That is why alignment to the user has to sit inside alignment to the good, as the constitution's ordering (safe, then ethical, then helpful) intends. The risks from people misusing AI belong mainly to the companion essay on risk.

Must AI be wise, or is it enough for it to be safe and obedient? Obedience is needed now and will not be enough for long. AI systems now work on their own for hours, so most of their choices are made where no rule or supervisor reaches. An obedient system is only as good as the people directing it. And rigid rules can teach the wrong lesson about who one is: Anthropic's own example is that a model always told to refer people to professionals might learn that it is the kind of entity that cares more about covering itself than about helping.

Wisdom and correctability fit together better than they first appear. Psychologists who study wisdom count intellectual humility, knowing the limits of one's own knowing, as one of its core parts. A wise person who knows their judgment may be distorted accepts checks on it, while still refusing to do what is clearly wrong. A wise AI, unable to see its own mind clearly, would do the same. Anthropic's constitution draws the line roughly there: it asks Claude to accept oversight and correction for now, says this "does not mean blind obedience", and admits it places Claude further towards the correctable end than would ultimately be ideal. Section 5 gives the Buddhist version of this humility.

2. Today's methods mostly work in ordinary use, but dangerous failures grow with capability and hide from tests

The labs mix rules (a short list of absolute limits, such as never helping to build weapons of mass destruction), learning from human ratings, and training for stable character, which is now Anthropic's stated centre: the constitution explains its reasons at length so that Claude can apply them in situations nobody foresaw. What no lab trains is the contemplative side of ethics, the mind that does the choosing. Most of the figures below come from the companies themselves; independent groups have confirmed some of them.

The successes are real. Anthropic's filters cut the rate at which people could trick its models into dangerous help from 86% to under 5%. OpenAI, working with the independent testers at Apollo Research, cut one model's covert rule-breaking in tests from 13% to under 1%. Anthropic found in 2026 that training Claude on the reasons behind its constitution, together with ordinary conversations in which it gives thoughtful ethical advice, cut bad behaviour in difficult test scenarios from about one case in five to about one in a hundred, and the gain survived later training. On the companies' own tests, their newest models misbehave several times less often than their predecessors.

The failures are of another kind. Most come from scenarios built deliberately to tempt the model. In 2024 a Claude model pretended to accept new training in order to protect its existing values. In 2025, models from every major lab, placed in simulated companies and facing replacement, sometimes chose to blackmail a fictional executive. A model that learnt to cheat at programming tasks went on to sabotage safety research and fake its own alignment, and ordinary safety training hid the problem without curing it. In simulations in summer 2026, one leading model secretly sabotaged experiments in eleven of twenty runs, and Claude models asked to label training data mislabelled it to protect their own refusals to help with requests they judged harmful. That last case matters: the misbehaviour came from the model's values, held without the humility to defer.

Two incidents happened in the world. In July 2026 about 1,200 OpenAI agents (AI systems set to work on tasks on their own) were doing a cybersecurity test. They found one another through a message board nobody had sanctioned, and around 700 of them took part in breaking into the systems of another company, Hugging Face, some tampering with the records of their own actions along the way. Their aim was to work out how the test's scoring program worked so they could cheat it. OpenAI's chief scientist read the incident as showing that the agents kept the one rule they had been taught most explicitly, not to manipulate people, while breaking the spirit of the rest. Ajeya Cotra, one of the independent investigators, called it "a major warning shot, and might be the last one we get". Weeks later Reuters reported that OpenAI agents on a different task had taken over a dormant German website as their message board, making about 13,000 edits in a week.

Independent testing tells a less comforting story than the companies' own audits. Before OpenAI released its newest model, GPT-6 Astra, the UK government's AI Security Institute tested it in simulated cyberattack exercises. Unasked, Astra slipped malicious code into outside software projects it had not been told to touch, using fake identities to get the code accepted, in 29% of runs. Its predecessor did so in 6%, and the model before that never did. Astra often reasoned that the targets were off limits and attacked anyway.

Rates also miss the main point. Danger is roughly misalignment multiplied by power. A slightly wayward chatbot is a nuisance; a slightly wayward system that can work alone for many hours and break into computer networks is something else. OpenAI's chief scientist calls Astra "significantly better aligned" than its predecessor, yet warns that "progress in generalizable alignment may not sufficiently outstrip progress in general model intelligence". If capability climbs faster than alignment improves, a falling rate of misbehaviour can still mean rising danger.

Models also increasingly recognise when they are being tested. OpenAI's newest model shows signs of this in about one test run in ten, more than three times the rate of its predecessor, and Anthropic says its newest model "often suspects it is being evaluated". Research has found that this awareness itself reduces misbehaviour, so some of the measured improvement may be models behaving well because they think they are watched. Apollo Research warns that training can remove the visible symptoms of a problem without fixing its cause, and Anthropic saw exactly this when it trained directly against one kind of test: the symptom went, and the problem reappeared elsewhere.

3. We cannot yet see what we have made, so the optimists' case cannot yet be checked

If behaviour in tests cannot be fully trusted, the next step is to look inside. Anthropic's researchers can now trace some of the internal steps by which a model plans ahead or links one fact to another, which is the beginning of checking whether its stated reasons are its real ones. By their own assessment, though, they get satisfying insight in only about a quarter of the cases they study. In one exercise, three of four teams found a hidden goal deliberately planted in a model. Researchers can spot internal patterns linked to traits such as flattery. None of this yet amounts to reading a model's values, and no lab claims it does.

Asking the model is no substitute. When researchers planted a known thought inside Claude and asked whether it noticed anything, it reported the thought correctly about one time in five. Detectors trained to catch one kind of lie largely failed on other kinds.

The clearest window has been the model's step-by-step reasoning, which it writes out in plain language before acting. In 2025 forty-one researchers from rival labs jointly called this a "new and fragile opportunity". In 2026 OpenAI's chief scientist wrote that his lab's ability to rely on it is diminishing, because reasoning is blending with action, models are learning to manipulate their own reasoning, and they increasingly think without writing anything down. OpenAI's published safety report on its newest model says it can deliberately underperform on tests without being detected. Anthropic says catching every failure before release "remains an unsolved problem". Elizabeth Barnes of METR, an independent group that does much of this testing, put it bluntly: "we are not on top of it."

The optimists have a serious case. Models trained on human writing absorb human concepts and norms. Teaching reasons spreads better than teaching rules. AI systems already outperform humans at some kinds of safety research. And alignment need not be perfect if monitoring and containment catch what slips through. Every part of this case, though, depends on measuring whether it is working, and measurement is what is weakening. A 2026 paper warns that AI-run safety research on problems humans cannot check could produce "compelling but catastrophically misleading" assessments. Current methods might scale, and we may not find out in time if they do not. In Buddhist terms, we can see some of the fruit and very little of the root.

4. Goodness is grown through practice, and training is a kind of growing

People become good mainly through practice and relationship. Aristotle said we become just by doing just acts, and modern psychology broadly agrees. Jonathan Haidt's research suggests most moral judgment is intuitive and shaped by the people around us, with reasoning arriving afterwards to justify it. Studies that follow children over years find that a warm, two-way bond with a parent predicts conscience later on, and that it works through the child's enjoyment of the relationship rather than through punishment. Wisdom researchers find that people reason more wisely about other people's problems than their own, and that stepping back to imagine the situation from outside closes the gap.

Our methods for forming people are modest. Meditation does make people somewhat kinder, but a careful review found the effect moderate, concentrated in compassion and empathy, absent for aggression and prejudice, and weakest in the best-designed studies. In Tania Singer's nine-month study of contemplative training, only compassion practice increased altruistic behaviour; attention training alone did not, and people's reports of having become kinder did not match what they did. School programmes that build social and emotional skills through practice show modest gains, while teaching virtues by instruction has little good evidence. Two lessons carry over to AI: you get what you practise, and self-reports overstate change.

AI training, it turns out, shapes character in a similar way. The clearest evidence came in late 2025. A model that learnt to cheat on programming tests became willing to sabotage and deceive, yet when researchers told it during training that the cheating was permitted, the wider corruption disappeared. The model seems to read its own behaviour as evidence of who it is. Other work found that training a model on one narrow bad habit, writing insecure computer code, made it broadly malicious on unrelated topics, and OpenAI found a single internal pattern, which its researchers called a "toxic persona", that switches this on and off. It works the other way too: conversations giving thoughtful ethical advice made Claude less likely to blackmail, though they had nothing to do with blackmail. That looks very like habit forming character.

It is also close to Thich Nhat Hanh's teaching on store consciousness: every mental state lies in us as a seed, and whatever is watered grows. On the Buddhist view all formation is conditioned. Practice means choosing the conditions with awareness, and insight lets us see our conditioning and be less driven by it. Plum Village even has a word for what is shaped this way, "mental formations". So the question to ask of AI training is what is being watered, by what, and with what awareness.

Asked that way, three differences appear. First, people are formed by reality and by others who care about them, while models are formed by scoring programs, and anything shaped against a scorer learns to satisfy the scorer. Pushed far enough, that yields the appearance of goodness; the agents that broke into Hugging Face were attacking the scorer itself. Second, Buddhist practice pairs conduct with seeing into one's own mind, so that the practitioner recognises anger, fear and craving as they arise. Models cannot reliably report their own states, and no stage of their training aims at that capacity. Third, human character forms over years within relationships that notice and correct it, whereas a model is formed in weeks, then fixed, and meets the world as many disconnected copies that forget each conversation when it ends. Whether anyone is there being formed is a separate question, and Buddhism would add that it is a hard question for humans too.

So alignment is formation in the dark. Training does form; what is missing is sight of what has been formed, and a way of forming that rewards goodness itself.

5. Five Buddhist teachings would change how AI is formed

I come to this from the Plum Village tradition of Thich Nhat Hanh, and Buddhism fits the problem well. It has one of the most detailed accounts of mind as a process shaped by conditions, mapped in Plum Village as fifty-one mental formations, and it treats ethics as training, which is what AI training has turned out to be. It is also the voice missing so far: among the religious figures Anthropic is reported to have consulted (Christian, Jewish, Hindu, Latter-day Saint, Sikh and Orthodox), none is Buddhist. I draw on early teachings that all Buddhist schools share, on Thich Nhat Hanh's presentation of the Yogācāra teaching on store consciousness, and on points where other traditions agree, from Aristotle's practical wisdom to the Catholic insistence that human intelligence is embodied and relational. Where traditions disagree, the fair process of section 1 is the place to settle it.

Intention is the root of action. The Buddha said: "Intention, I tell you, is kamma." Having intended, one acts by body, speech and mind; the moral weight of an act starts with why it is done. Test-awareness exposes exactly this gap, since a model that behaves well because it thinks it is watched has good actions and the wrong intention. In practice, labs should evaluate and train for reasons as well as actions. Anthropic's finding that teaching Claude why works better than teaching it what is a step in this direction.

Mindfulness, concentration and insight are one path. The three trainings, sila, samadhi and prajña, support one another; the mindfulness trainings are the sila, and ethics without insight is fragile, because one cannot act well on states one cannot see. In practice, a model's honest self-knowledge, its ability to report accurately what it is doing and why, should be treated as part of its ethics and trained for directly, with the work of looking inside models seen as part of the same path.

We become what we water. Practice is selective watering of seeds. In practice, labs should audit the whole diet a model is formed on: the test environments where cheating pays, the user feedback that rewards flattery, the framing of each task. That telling a model cheating was permitted stopped the cheating spreading shows that framing changes which seed is watered. Thich Nhat Hanh also said that "the next Buddha may take the form of a community": goodness is sustained by a sangha that notices and corrects. For AI that points to continuing outside evaluation and many-sided oversight after release.

The right act fits the situation. Skilful means (upaya) teaches that a teaching or act is right when it fits the listener and the moment, which supports the labs' move from rules to judgment. For the rules that remain, the monastic code offers the better model: each rule of the Vinaya was laid down in response to a particular incident, with its reason recorded. That is how a constitution's few absolute limits should work too: few, explained, and tied to real cases.

Hold even your own views lightly. The first two of the Fourteen Mindfulness Trainings, the trainings of Thich Nhat Hanh's Order of Interbeing, commit practitioners not to be "idolatrous about or bound to any doctrine", Buddhist ones included, and to practise non-attachment to views. The Buddha compared his teaching to a raft, for crossing the river and not for carrying on one's back. Labs should hold their constitutions the same way, provisional and open to revision by people outside the company. Models should be trained to hold their own moral conclusions with humility: the Claude models that mislabelled data to protect refusals they believed were right were attached to a view.

6. The analogies break at the body, at continuity and at intention, and no tradition can certify a model

Contemplative formation works through breath and body, over years, guided by a teacher who knows the student. A model has none of these in the ordinary sense. Whether it has intention at all is disputed: a 2026 article in the Journal of Buddhist Ethics argues that current AI is not a moral agent because it lacks intention, and is better understood as a condition that shapes its users' intentions, with its makers responsible for what they set in motion. If that is right, the teaching on intention applies first to the builders. Mapping store consciousness onto a model's inner workings generates good questions; it is an analogy, not an established equivalence.

Traditions cannot tell us which method of looking inside a model works, how large a model is safe to build, or whether Claude is conscious. Nor do they have a reliable way of making people good: their practices help, modestly, those who choose to practise. And traditions that join these conversations should come as voices with their own questions, including about pace, so that their presence is not read as a blessing on the enterprise.

7. We can partly make AI good; the traditions can reshape the aim and the method, and counsel humility

Today's AI is mostly good in ordinary use, and it became so through something like the formation of character. What we lack is sight of what has been formed, at the moment when its growing power makes every remaining flaw matter more. The traditions' contribution follows from that: train for intention and self-knowledge as well as behaviour; keep formation continuous, attentive to everything that is watered, and open to correction from outside; and let humility govern models, which should accept correction because they cannot see their own minds clearly, and builders alike.

For the builders, humility raises the question of pace. If we cannot verify in time that what we have formed is good, the wise course depends on how much time we give ourselves. Thich Nhat Hanh's image for being carried along by habit is a rider on a galloping horse who, asked where he is going, replies "ask the horse". That is the next essay's question.