Skip to content
DT Nova

AI governance: a practical framework for companies using AI under the EU AI Act

AI governance is how a company decides which AI systems it uses, for what and with which data, who is accountable, and how it checks that each system works, stays within the law and is stopped when it does not. With the EU AI Act now generally applicable and AI built into most business software, AI governance is no longer only for large companies. This guide sets out a practical framework: an inventory, risk classes, roles, policies, the duties of deployers under the AI Act, and a checklist to start.

Legal references are to Regulation (EU) 2024/1689 (the AI Act) as amended by Regulation (EU) 2026/1744, and to the European Commission's AI Act page, all read on 12 November 2026. The NIST AI Risk Management Framework page was read the same day. This is practical information, not legal advice.

Why AI governance matters now

According to the Commission, the AI Act entered into force on 1 August 2024 and became generally applicable on 2 August 2026. Prohibited practices and the AI literacy provisions have applied since 2 February 2025, and the rules for general-purpose AI models since 2 August 2025. The high-risk rules for use cases in sensitive areas, including employment, education, critical infrastructure and biometrics, apply from 2 December 2027, and those for AI built into regulated products such as lifts or toys from 2 August 2028. Meanwhile, AI arrives in companies less through big projects than through features switched on in office, CRM and helpdesk tools, often without anyone deciding it. Governance is how you find out what is in use and decide on purpose.

Step 1: an AI inventory

Agents that take actions in other systems need special attention in the inventory, because their permissions decide what can go wrong; see AI agent development.

List every AI system in use or planned, including AI features inside bought software. For each, record:

  • name, vendor or internal team, and the business owner;
  • purpose and the decisions or outputs it influences;
  • data it receives, especially personal and confidential data, and where it is processed;
  • your role: provider (you build it or put your name on it) or deployer (you use it);
  • risk class under the AI Act, and whether a DPIA exists;
  • who checks its outputs, and how it can be switched off.

Step 2: sort by risk

AI Act classExamplesWhat follows
Prohibited (Article 5)Manipulative techniques causing significant harm, social scoring, predicting crime from profiling alone, untargeted scraping of facial images, emotion recognition at work and in education, biometric categorisation by sensitive traitsDo not use. Check the inventory for these first
High-risk (Article 6 and Annex III, or regulated products)Recruitment and candidate screening, decisions on promotion or termination, credit scoring of people, access to education, some critical infrastructure usesProvider duties (risk management, data governance, documentation, conformity assessment) and deployer duties (Article 26) from 2 December 2027, or 2 August 2028 for regulated products
Transparency (Article 50)Chatbots that interact with people, generated or manipulated images, audio, video and text, deep fakesTell people they are interacting with AI; mark synthetic content
Other usesDrafting, search, summarising, translation, internal analyticsNo specific AI Act duties beyond AI literacy, but the GDPR, contracts, confidentiality and quality still apply

The Commission also lists a ninth prohibition, added by the AI Omnibus and in effect from December 2026: AI systems that generate non-consensual sexually explicit and intimate content or child sexual abuse material.

Step 3: roles and decisions

  • An accountable executive who owns AI governance and approves high-risk uses.
  • A small review group: business, IT or security, data protection, legal where needed. It reviews new use cases against a short form, not in long meetings.
  • A business owner per system who is responsible for its use, outputs and monitoring.
  • Human oversight for systems that influence decisions about people. Article 26(2) requires deployers of high-risk systems to assign oversight to people with the necessary competence, training and authority.

Step 4: policies that people can follow

Keep the policy short and concrete. It should answer: which AI tools are approved; which data may go into which tool (public, internal, confidential, personal); when a person must check the output before it is used; how AI-generated content is labelled; how new use cases are proposed; and what to do when something goes wrong. Attach the approved tools list and update it when the inventory changes.

Step 5: AI literacy and training

Article 4 of the AI Act, as replaced by Regulation (EU) 2026/1744, requires providers and deployers to take measures to support the development of AI literacy of their staff and others operating or using AI systems on their behalf, taking into account their knowledge, experience and the context of use. The amended text states that this does not require a guarantee of any specific level of literacy for any individual. In practice: a short training for everyone on the policy and the limits of AI, and deeper training for owners of higher-risk systems, with a record of who was trained.

For software teams using coding assistants, see our AI in software development guide.

Step 6: duties when you deploy high-risk AI

If you use a high-risk system, Article 26 sets the deployer duties. Among them: use the system according to its instructions for use; assign human oversight to competent people; where you control the input data, ensure it is relevant and sufficiently representative for the intended purpose; monitor the operation and inform the provider and authorities of serious risks or incidents; and keep the logs under your control. Some deployers, including bodies governed by public law, private entities providing public services, and deployers of systems for credit scoring or life and health insurance pricing, must also carry out a fundamental rights impact assessment under Article 27. Since the amendment by Regulation (EU) 2026/1744, Article 27(4) lets them cross-reference their data protection impact assessment instead of repeating it.

Step 7: monitor, log and review

Each system needs a way to measure whether it still works: a fixed set of test cases rerun after every model or prompt change, sampling of outputs by a person, user feedback, and incident records. Review the inventory every quarter and each system at least yearly. When a vendor changes the underlying model, treat it as a change that needs a quick re-test.

Frameworks to borrow from

The NIST AI Risk Management Framework (AI RMF 1.0), released on 26 January 2023, is a free, voluntary framework with four functions: govern, map, measure and manage. NIST published a Generative AI Profile (NIST-AI-600-1) on 26 July 2024, and its page notes that AI RMF 1.0 is being revised. ISO/IEC 42001 is the international standard for AI management systems, which companies can be certified against. A smaller company does not need either in full, but both are useful lists of questions.

Penalties

Under Article 99 of the AI Act, using a prohibited practice can lead to fines of up to 35 million euro or 7 % of total worldwide annual turnover, whichever is higher, and breaching deployer duties under Article 26 up to 15 million euro or 3 %. Member States set the rules and must take into account the interests and economic viability of SMEs and small mid-caps.

An AI governance checklist to start

  1. AI inventory, including AI features in bought software.
  2. Risk class and role (provider or deployer) for each system.
  3. Prohibited practices ruled out.
  4. Named accountable executive and system owners.
  5. A one-page AI policy with an approved tools list and data rules.
  6. Training for all staff, recorded.
  7. DPIAs where personal data is processed at high risk.
  8. Test sets, monitoring and an incident route per system.
  9. Contracts with AI vendors covering data use, location and model changes.
  10. Review dates in the calendar.

How we help

We build AI systems with governance built in: an evaluation set and test run for every change, logging of inputs and outputs where it is lawful, human review steps where decisions affect people, clear labelling of AI content, and EU hosting by default. Our AI consulting work includes the inventory and risk sorting, and our AI strategy guide shows where governance fits into the plan. When we build a chatbot or another AI tool for you, you own the code, the prompts and the test sets.

Questions people ask

What is AI governance?

AI governance is the set of roles, rules and controls a company uses to decide which AI systems it uses, for what, with which data, and how it checks that they work as intended, stay within the law and are stopped when they do not. It covers both AI the company builds and AI it buys inside other tools.

Is AI governance required by law?

Parts of it are. The EU AI Act requires providers and deployers to support AI literacy of their staff, bans some practices outright, requires transparency for some systems such as chatbots and synthetic content, and sets detailed duties for high-risk systems. The GDPR applies whenever personal data is involved. A governance framework is the practical way to meet these duties and prove it.

What is the difference between AI governance and AI ethics?

AI ethics is about principles: fairness, transparency, human oversight, accountability. AI governance is how a company turns those principles and the legal rules into decisions, owners, documents and checks in daily work. Without governance, ethics principles stay on a poster.

Do small companies need an AI governance framework?

Yes, but a small one. A small company needs an inventory of the AI tools in use, a short policy on which data may go into which tools, a named owner, basic training and a simple review of any new use case for risk. A heavier framework is needed only when the company builds AI products or uses AI in high-risk areas such as hiring.

Which standards exist for AI governance?

The NIST AI Risk Management Framework 1.0, released on 26 January 2023, is a free, voluntary framework organised around four functions: govern, map, measure and manage. ISO/IEC 42001 is the international standard for AI management systems, which companies can be certified against. Harmonised European standards under the AI Act are being developed.

Sources