01
Look behind the scenes
Free
Half an hour, by invitation
See everything we build running in our sandbox, the Proeftuin. No sales pitch, unless you ask for one.
Request an invitationBlog
Not every system is high-risk. Look at the task, the consequences, the classification, where a person steps in and the evidence for each action.
An agent that turns an order email into a draft order is very different from a system that helps rank job applicants, assesses people's creditworthiness or allocates work based on someone's behaviour, traits or characteristics. Both may use a language model. The consequences of an error are not the same.
The AI Act does not make a system high-risk just because it influences a decision. Its intended purpose must fall under Article 6 and Annex I or III. A system listed in Annex III may not count as high-risk if it performs a narrow procedural or preparatory task and poses no significant risk, for example because it does not materially influence the outcome of the decision. That exception does not apply where the system performs profiling of natural persons.
The transparency obligation of Article 50(1) for direct interaction with natural persons has applied since 2 August 2026. The rules for high-risk AI systems under Annex III, including Article 6(2), apply from 2 December 2027. For high-risk AI systems under Annex I, including Article 6(1), the date is 2 August 2028. Have the classification of a specific application reviewed before drawing legal conclusions from it.
First describe what the agent is actually allowed to do. What data comes in? What conclusion does it draw? Which system then receives an instruction? Who can override, correct or stop it? Only once that is clear can you assess which rules, sector regulations and privacy obligations apply to the solution.
An agent that answers customer questions shows the difference. Does it only read a knowledge base and draft a reply for a service employee? Or does it send a message about a contract change on its own? In the second case, the approval threshold, the source of the information and the ability to correct an error matter far more. The technology may be almost identical, but the business responsibility is not.
This description also helps when a system is probably not high-risk. It is still sensible to know what the agent may and may not do. A vague instruction such as "handle customer questions" leaves too much room for unusual cases. A specific instruction such as "draft a reply about the status of an existing order, without changing price, contract or delivery" sets a boundary you can test.
These five points are design choices, not a universal legal checklist. For a high-risk AI system, the requirements for risk management, documentation and automatic logging rest mainly with the provider. The deployer must use the system in line with the instructions, assign human oversight, monitor its operation and keep the logs to the extent that they are under its control. Which obligations apply depends on the role and the application.
First, the purpose and its limits. Someone must be able to read what the agent is for and what it never does on its own. That stops a convenient pilot set-up from quietly being used for a different decision later.
Second, where the data comes from. For a reply about an order you want to know which system supplied the status, delivery date and customer details. If a source is incomplete, the agent must say so or send the task to an employee. A reply that cites a credible source is better than one that only sounds confident.
Third, the action and the evidence for it. For each processing run, record a task ID, the instructions or version used, the source consulted, the proposed action and the outcome. That does not mean keeping all content forever. The retention period and access rights should match the nature of the process. Do make sure an authorised colleague can trace the inputs, sources, configuration, actions and human decisions behind each outcome.
Fourth, human oversight. A person only has real oversight once it is clear which exceptions reach them, what context they see and what authority they have. An approval button without the source data is not a real control. An employee must be able to reject or change an outcome and stop the process safely.
Fifth, changes and incidents. A new model, an adjusted prompt, a different source or a changed process rule can alter the outcome. Keep track of what changed, test it against known cases and decide who investigates unexpected results. That keeps the evidence useful long after the original go-live is forgotten.
For high-risk AI systems, the AI Act sets requirements for risk management, documentation, record-keeping, transparency, human oversight and post-market monitoring, among other things. For an organisation that puts an agent to work, these legal terms translate into design choices. A threshold amount can support human oversight. Whether that is enough depends on the application and the organisation's role. A task ID in a log supports traceability. A test set of known exceptions shows whether a change affects the process.
That does not mean every order email needs the same legal documentation as a system that rejects a person for a job or a loan. Under Article 50, when a system interacts directly with natural persons, the provider must make clear to them, no later than the first interaction, that they are dealing with AI, unless that is already obvious.
An agent that does real work must leave more than an answer: a trail that shows what it did, within what limits and why an employee did or did not need to step in.
Three ways to start
Start small, with one process that comes up every week. Choose how: from half an hour behind the scenes to a session at your offices.
01
Free
Half an hour, by invitation
See everything we build running in our sandbox, the Proeftuin. No sales pitch, unless you ask for one.
Request an invitation02Recommended
Free
Half an hour on Teams
You tell us what you already do with AI. We tell you what fits, or whether it is still too early.
Call with Marcel Logtenberg
03
From €495
At your offices, excluding VAT
An AI quick scan, an AI assessment or an AI assessment plus inspiration session, at a fixed price.
Get started