What Are We Actually Aligning AI With?

AI implementation is increasingly becoming an exercise in context.

Connect the company documents. Integrate the ERP. Add legislation and procedures. Build specialised agents. Define instructions. Improve retrieval. Give the system access to more organisational knowledge. All of this improves what the AI knows. It does not necessarily tell the AI what constitutes a good outcome.

That distinction becomes increasingly important as general-purpose AI moves beyond information retrieval and into analysis, recommendations and operational decision support. An AI system can understand the facts of a problem extremely well while interpreting those facts through assumptions that neither the user nor the organisation has consciously chosen.

The alignment question is therefore not only whether AI follows our instructions, but whether we understand the assumptions according to which it interprets them.

General-purpose AI does not simply receive a problem and return a neutral solution. It first has to interpret what kind of problem it is dealing with. Consider some ordinary examples:

• An unhappy employee can become an incentives, compensation, retention or performance problem.
• An inefficient process can become an automation problem.
• A struggling SME can become a growth, scalability or margin problem.
• Organisational redundancy can become waste.
• Personal uncertainty can become risk management.
• Unused capability can become untapped potential.

These are all legitimate frames. They are not universally appropriate ones.

An apparently redundant process may provide resilience. A business owner may accept lower productivity to retain an employee carrying valuable tacit knowledge. A company may deliberately sacrifice margin to protect service quality. An SME may have no interest in scaling beyond its current size. €3 million in revenue, twenty employees and a decent life may genuinely be preferable to €15 million in revenue, investors, eighty employees and continuous pressure. The issue is therefore not whether optimisation is good or bad.

Optimisation only becomes meaningful after we establish what we are optimising for — and what we are unwilling to sacrifice in the process.

This is where apparently technical AI implementation becomes a management question. I initially became interested in this through what appeared to be recognizable political and economic tendencies in general-purpose AI: individual agency, incentives, autonomy, entrepreneurship, technological solutions, proceduralism or market reasoning.

It is tempting to label these tendencies politically. The research does not support anything that simple. Political and cultural tendencies have been measured in LLM outputs, but they vary across models, languages, prompts, post-training methods and even the tests used to measure them.

The more defensible conclusion is also more useful. AI systems are developed from human-generated data and subsequently shaped through post-training, human feedback, behavioural rules, safety requirements and product design.

There is no reason to assume the resulting behaviour will be intellectually neutral. Nor is complete neutrality necessarily possible.

The practical question is therefore not what ideology an AI supposedly has, but which assumptions it introduces when interpreting a problem.

Market reasoning provides a useful example. Companies require margins. Capital has a cost. Labour has a cost. Competition exists. Resources are finite.

These are economic realities.

But markets are a terrain within which organisations operate, not a complete framework for evaluating every human decision.

I describe this distinction to myself simply:

Capitalism is terrain, not belief.

Pretending the terrain does not exist is foolish. Treating the terrain as a complete philosophy of human existence is equally limiting.

A human being is not "the market" every time they ask AI a question.

So:

• Not every capability needs to be monetised.
• Not every spare capacity needs to be utilised.
• Not every inefficiency needs to be removed.
• Not every organisation needs maximum growth.
• Not every uncertainty needs immediate resolution.
• Not everything measurable needs improvement.

The reverse is equally important.

Some inefficiencies should be removed. Some processes should be automated. Some costs are unsustainable. Some employees are genuinely underperforming. Some organisations use "culture" and "tradition" to protect poor management.

The question is not whether optimisation should occur. It is according to which objective, within which constraints, and at what cost.

Now consider broadly similar AI technology deployed inside a Dutch offshore contractor, a Greek family-owned logistics company, a German industrial manufacturer and a software company.

Each could issue exactly the same instruction: Improve operational efficiency.

It sounds precise. It isn't. Efficiency according to whom?

These companies operate under different labour structures, legislation, safety requirements, customer expectations, management cultures, risk tolerances and organisational histories.

The same technology is therefore operating against very different definitions of acceptable performance. This suggests that organisations need to think about AI context at three levels.

- First: What does the AI know?

Legislation, contracts, technical standards, procedures, policies, company documentation and system data.

This is where much enterprise AI investment currently concentrates.

- Second: Does it understand how the organisation actually works?

Not only what the procedure says.

Who actually makes the decision? Where do informal handovers occur? Why does a parallel spreadsheet exist? Where does practical authority differ from the organisational chart? Why are two people checking something that theoretically requires one?

Sometimes the answer is organisational stupidity. Sometimes it is fifteen years of undocumented experience.

- Third: Does it understand what the organisation considers a good outcome?

This is the difficult one.

- What happens when margin conflicts with resilience?

- When automation conflicts with workforce continuity?

- When standardisation conflicts with customer flexibility?

- When efficiency conflicts with safety?

- When today's cost reduction weakens tomorrow's capability?

Company documents may explain what an organisation does. KPIs may explain what it measures.

Neither necessarily explains how competing outcomes should be evaluated.

An AI can therefore know almost everything about an organisation and still become extremely competent at solving the wrong problem.

There is another side to this. Humans are not merely configuring AI systems. We are repeatedly interacting with them.

Research has already shown that AI interaction can influence subsequent human judgement, while users may underestimate that influence. The phenomenon itself is not new. Consultants, managers, economists, teachers, books, media and search engines have always influenced how problems are framed.

What changes with conversational AI is the combination of frequency, breadth and personalisation.

The same system may participate in someone's thinking about business strategy, employees, investment, politics, relationships, writing, personal decisions and risk — potentially hundreds of times every month. The influence therefore does not need to take the form of persuasion. It can operate through repetition.

If uncertainty is consistently interpreted as risk, unused capacity as opportunity, redundancy as inefficiency and organisational problems as optimisation problems, those categories gradually become easier for us to reach for as well. AI has then done more than provide answers.

It has participated in defining the vocabulary through which we understand the problem.

The quality of the answer can make the assumptions behind it harder to see.

There is an obvious trap in trying to solve this. If contextual alignment simply means teaching AI to reproduce the worldview of the user or organisation, the result may be sophisticated confirmation bias. AI sycophancy is already a documented problem: systems designed to satisfy users can become excessively agreeable.

So alignment cannot mean agreement.

A useful AI needs two things simultaneously:

• Context — enough understanding to know why particular objectives, constraints and trade-offs matter.
• Independence — enough analytical distance to challenge those assumptions when the evidence requires it.

It should be capable of recognising that a requested optimisation conflicts with another stated objective. It should distinguish possibility from probability. It should recognise when an apparently inefficient process serves a legitimate operational purpose. And it should equally recognise when management is protecting poor performance behind culture, tradition or personal preference.

That is not disagreement for its own sake. It is alignment with friction.

Enough shared context to make disagreement intelligent. Enough independence to make it useful.

And this is where the question eventually returns to us.

AI systems will continue improving. More company knowledge will become accessible to them. More workflows will be automated. More decisions will receive AI support. The scarce resource may therefore gradually become less about access to information and more about judgement.

Before asking AI what should be optimised, we need to become better at defining what actually matters. Before asking it to resolve uncertainty, we need to decide whether that uncertainty requires resolution. Before asking it to improve our organisation, we need to understand what we are trying to preserve while improving it.

And before building elaborate instructions around our worldview, we need enough critical thinking to recognise where that worldview itself may be wrong.

This starts much closer to home than enterprise AI strategy.

It starts with understanding how we make decisions in our own lives and organisations:

- What do we value?

- What are we actually trying to achieve?

- What are we unwilling to sacrifice?

- Which problems require action?

- Which ones require acceptance?

- Where does additional analysis improve the decision — and where are we simply processing because we can?

AI makes these questions more important, not less. Because as access to intelligence becomes cheaper and increasingly abundant, the human advantage may shift away from producing more analysis and toward judging what deserves analysis in the first place.

We do not need to become more intelligent than AI.

We need to become better at knowing what intelligence should be used for.

And perhaps that is what we should be aligning first.

‍