Governing AI in an established company who decides, which data, who is accountable

Where do you start governing AI in a company?

From a list of what is already running. In an established company AI arrives before the rules: in 2025, 71% of large Italian companies had at least one AI project, against 59% in 2024, and 84% had bought licences for at least one generative AI tool, led by Microsoft Copilot, ChatGPT Plus and Gemini Advanced. The rules lag behind: four large companies in ten have usage guidelines, and only 9% have structured AI governance, “with defined responsibilities” (Artificial Intelligence Observatory of Politecnico di Milano, February 2026).

Asked by an operations manager at a 600-person company, with ChatGPT, Copilot, a chatbot and two prototypes already in use, Google and the AI assistants give the same recipe: a committee, one person who coordinates, a list of systems, traffic-light data rules, approval by levels. They disagree on who is accountable when AI gets it wrong, and almost none says what Italian law requires: of the 73 pages we read, across Google results and the sources of the AI answers, Law 132 of 2025 appears in 7 and Legislative Decree 179 of 2026 in none.

The list goes by use, not by tool. ChatGPT, asked by someone working in compliance, advises recording “the system and the use case separately”, and it is the right advice: the same Copilot that summarises a meeting can write the reply to a complaint, and the two uses carry different risks.

The NIST AI Risk Management Framework, the voluntary US reference, lists an inventory of systems among its governance practices (Govern 1.6) and asks you to plan how a system is decommissioned too (Govern 1.7), as the framework core puts it. For the list to be true, people using an unauthorised tool must be able to say so without consequences: Perplexity suggests a period of two to four weeks to declare it. Every row of the list ends with a decision and a date; otherwise it is a census and governs nothing.

Who is in charge, and with what mandate?

One person who coordinates, with a written mandate from management and dedicated time, and a small group that decides on the cases that matter. The AI answers agree on this, and ChatGPT adds that the person can already be in the company: you do not need to hire a Chief AI Officer straight away. Who chairs changes: the chief operating officer for Google's AI Mode, the head of IT or technology for the AI Overview and for Perplexity. According to the Observatory, 54% of large companies are moving towards centralised governance.

NIST asks for roles, responsibilities and lines of communication to be documented and clear (Govern 2.1), and for executive leadership to take responsibility for decisions about AI risks (Govern 2.3). The group, though, should not approve everything. A committee that reviews every request once a month becomes the reason people go back to personal accounts. It decides on yellow and red cases; the rest flows.

Who approves a new use of AI?

It depends on the risk of the use, not on the tool. The AI answers all propose three traffic-light levels and a fast lane: 5 working days for Google's AI Overview in the English question and for AI Mode in the compliance question. ChatGPT points out that the traffic light is an “internal classification”, not the AI Act's; true, but the two must talk to each other: a use that is high risk under the AI Act cannot be green for you.

The table is an example to adapt: your management sets the thresholds, on the cost an error would have in your process. ChatGPT, in the English question, adds a rule that prevents everlasting prototypes: “A prototype can be approved for a controlled test without being approved for production”. Every trial starts with a threshold to reach and an end date, both set beforehand. What the AI Act requires for red uses is in the guide on what falls to those who use AI.

Which data can go into an AI tool?

It depends on the class of the data and on the contract with whoever provides the tool. If you have an information security policy, you already have the classes; the AI rule needs to add, for every approved tool, which classes it accepts.

A company licence does not settle it. Google's AI Mode writes that Enterprise licences wipe out “the risk of data leak”; ChatGPT, to the same question, that paying for a company licence “does not automatically make any use safe”. The second is right. The contract excludes training on your data, but it does not decide who inside the company can see what: an assistant connected to company documents answers with whatever its permissions let it read, and if the permissions are broad, so are the answers. In the searchable document archive we built for an industrial company, 225 GB of regulations, HSE, safety and operations documents, the agent looks for the answer only in the documents the asker's role lets them see. Where to run the models for the most sensitive data is in the guide on AI in the cloud or on your servers.

Banning is not enough. The Observatory says so too: large companies know that banning generative AI “would lead to shadow AI and loss of control”, and only 24% ban tools not provided by the organisation. If the company tool is worse than the personal one, the rule gets bypassed: the approved tool comes before the ban.

Before checking who uses what, though, bring in HR and legal. Google's AI Overview, asked the compliance question, advises analysing network logs to find unauthorised AI, without naming the Italian rules on monitoring workers. Article 4 of the Workers' Statute lets you use the information collected only if the worker has received adequate information on how the tools are used and how checks are carried out, in line with privacy law; depending on the tool, a union agreement may also be needed. Monitoring is designed first and written into the policy.

How do you measure whether AI works?

Against the starting point, with a number chosen beforehand. Among large companies using generative AI, only 54% try to measure its benefit, almost always by collecting employees' opinions; only 11% monitor it periodically and in a structured way (Observatory, February 2026). Google's AI Overview, in the English question, puts it in one line: “Never measure AI by "usage."” How many people open Copilot tells you how many open it, not what changed in the work.

Next to the benefit you measure the cost: per case, not per licence, with spending limits per model and per period. An agent that works well but costs more than the work it replaces is not a result. The number is chosen before you start, and on the set date it decides whether the use continues, changes or stops.

Who is accountable when AI gets it wrong?

Towards customers and third parties, the company; inside the company, the owner of the process the AI works in; whoever uses the tool answers for having checked the result before using it. The AI answers disagree. For Google's AI Mode “an AI error is a company error”; for the AI Overview “the end user remains solely responsible for the output”; in the English question, “The Business Unit Head who implemented the tool owns the operational risk”. Each has a piece of the truth; none separates the levels.

The most quoted case is Canadian. In February 2024, Air Canada's chatbot had assured a customer that a bereavement fare could be obtained retroactively; the customer bought the ticket, and the airline then refused it. The British Columbia tribunal found a negligent misrepresentation and did not let the company disown what the chatbot had written (Moffatt v. Air Canada, 2024 BCCRT 149, as reported by Agenda Digitale, in Italian). The same article warns that not every chatbot answer becomes a contractual commitment: content and the other information the customer had both count. But telling a customer that the chatbot got it wrong is not a defence.

The internal level needs people who can say no. For high-risk systems the AI Act wants human oversight assigned to people “who have the necessary competence, training and authority” (article 26, paragraph 2). For employment decisions Italian law is already further ahead: since 23 October 2026, final decisions on hiring, changing and ending employment, including disciplinary measures and performance evaluations, rest with a person who has real and autonomous power (Legislative Decree 179 of 2026, article 41, in Italian).

Errors are then collected: a simple way to report them, an incident log, a person who can stop the tool. NIST asks for all three: user feedback to report problems (Measure 3.3), post-deployment monitoring with the ability to override and decommission (Manage 4.1), incidents and errors communicated, tracked and documented (Manage 4.3).

How do you build the rules into the systems?

With controls that do not depend on people's memory. A rule written in the policy holds as long as someone remembers it; the same rule in the system always holds.

This is how we work when we bring AI into established companies: before release come human approval on risky operations, the decision log and spending limits per model. In a group with several companies the separation holds between them too: on the AI agent audit platform every organisation has its own isolated space, with separate data, definitions and contacts. Automation goes as far as the cost of an error stays acceptable, and from there a person decides (how we ensure reliability). Observability tools, which we always recommend and which are switched on at your decision, leave for every operation the trace you need to reconstruct an AI decision even months later, and your data does not train models belonging to others. For management, legal and compliance we run training paths on decisions and governance, built on your company's use cases.

What does the law require, and what not?

Less than many answers say, and different things. To those who use AI, the AI Act asks for training measures for everyone, transparency in some cases and heavy obligations only for high-risk uses; Italian law adds notice to workers and a person's decision on employment. It does not require a committee, an AI officer or an internal list of systems.

Article 4 is in force since 2 February 2025 and rewritten by Regulation (EU) 2026/1744: measures supporting staff's AI literacy, without imposing a specific level. Almost every AI answer still describes it as a duty to train staff, and none says it was rewritten; only Perplexity lists the regulation that changed it among its sources. According to the Observatory, more than one large company in two has started AI literacy initiatives, but only 15% have a structured AI Act compliance project, integrated with the other rules. Article 11 of Law 132 of 2025 and article 1-bis of Legislative Decree 152 of 1997, both in Italian, require informing workers when an automated system decides or monitors something about their work.

Three claims found in the AI answers need correcting. Google's AI Mode writes that the AI Act requires creating an internal register: for a private company that uses AI it does not; article 26 asks you to keep the logs of high-risk systems, which is something else. The same surface presents the fine of up to 15 million euro or 3% of turnover as the price of operating without rules: in article 99 it is the ceiling for breaching the deployer obligations of article 26 and the transparency obligations, not for lacking a policy. And in the English question it allows “the free tier of ChatGPT” in the green level, which has no company contract on data.

The list, the group and the policy are not obligations: they are the tools with which you prove, on the day it matters, that you met the real ones. The detail of the obligations, case by case and with a self-check, is in the guide on the AI Act for deployers.

Do you need ISO/IEC 42001?

Not to get started, and it is not mandatory. ISO/IEC 42001, published in December 2023, “specifies the requirements and provides guidance for establishing, implementing, maintaining and continually improving an AI management system”, for organisations that provide or use products or services built on AI (standard page on AFNOR). The NIST AI Risk Management Framework, published on 26 January 2023, is voluntary and free, and organises the work into four functions: govern, map, measure, manage.

Among the AI answers it is mentioned by ChatGPT and Perplexity, which proposes 42001 as the structure and NIST as the checklist; none says when certification pays off. Our reading: it pays off for those selling AI systems to customers who ask for it, and for those who already have a certified management system, such as ISO 27001, because the work adds to something that exists. For everyone else, the list of uses, written roles and an incident log are worth more than a certificate.

Where do you begin?

With the list, not the policy.

1. Make the list of uses across all departments, with a few weeks in which unauthorised tools can be declared without consequences.
2. Appoint the coordinator, with a written mandate from management, and the group: IT, legal, data protection, HR, the process owners.
3. Write the data classes and, for every approved tool, which ones it accepts; give the company tool to whoever needs it before banning the personal one.
4. Set the three approval levels, with a fast lane for green and an end date for every trial.
5. For every yellow or red use, choose the number to measure, the starting point and the decision date.
6. Open the incident log and a simple way to report an error.
7. Write the policy, on what you have decided, and build the training on it.

The policy is written last, because it describes decisions already taken; written first, it is a list of good intentions. If instead AI is not in your company yet and you are looking for the first process to start from, read the guide on AI in business: where to start.

An example: a company with 600 people

A company with 600 people has ChatGPT in several departments, Copilot in administration, a customer service chatbot and two prototypes built by IT. It is the case we put to Google and the AI assistants; the details that follow are invented, to be replaced with yours.

The first month produces the list: about forty uses, half on personal accounts. Three upload customer contracts into a tool without a data processing agreement: they stop at once, and the people using them get the company tool the same week. Management appoints the operations manager as coordinator, with a written mandate and one day a week of dedicated time; the group meets every two weeks, only on yellow and red cases.

The customer service chatbot is red: it answers on its own, and what it promises binds the company. It stays on, but answers only from approved documents, hands refunds and complaints to an operator, and every week someone checks a sample of conversations. Of the two prototypes, one has a threshold and a date and becomes a project; the other, with no number to improve, stops.

HR asks to use AI for a first screening of CVs. It is a use in annex III of the AI Act, with the obligations of article 26 from 2 December 2027, and the group treats it as red. Article 41 of Legislative Decree 179 of 2026 does not count searching for and screening candidates among final decisions; when AI enters the decision to hire, performance evaluations or disciplinary measures, though, the final decision already rests with a person today. After three months the company knows how many uses it has, who answers for them, which data goes where and whether the most important uses work. Only then does it write the policy: a few pages, on decisions already taken.

Sources and further reading

• Artificial Intelligence Observatory of Politecnico di Milano, “Il mercato dell'AI in Italia cresce del 50% nel 2025, 1,8 mld di euro” (The AI market in Italy grows by 50% in 2025, 1.8 billion euro), 5 February 2026osservatori.net, in Italian, accessed 31 October 2026
• NIST, “AI Risk Management Framework”, version 1.0 of 26 January 2023nist.gov, accessed 31 October 2026
• NIST AI Resource Center, “AI RMF Core”, Govern, Map, Measure and Manage functionsairc.nist.gov, accessed 31 October 2026
• ISO/IEC 42001:2023, “Information technology, Artificial intelligence, Management system”boutique.afnor.org, accessed 31 October 2026
• AI Act, article 4, “AI literacy”ai-act-service-desk.ec.europa.eu, accessed 31 October 2026
• AI Act, article 26, “Obligations of deployers of high-risk AI systems”ai-act-service-desk.ec.europa.eu, accessed 31 October 2026
• AI Act, article 50, transparency obligations for providers and deployersai-act-service-desk.ec.europa.eu, accessed 31 October 2026
• AI Act, article 99, “Penalties”ai-act-service-desk.ec.europa.eu, accessed 31 October 2026
• Law of 23 September 2025, no. 132, article 11, “Disposizioni sull'uso dell'intelligenza artificiale in materia di lavoro” (provisions on the use of artificial intelligence at work)normattiva.it, in Italian, accessed 31 October 2026
• Legislative Decree of 7 October 2026, no. 179, article 41, “Tutela del lavoratore nei processi decisionali assistiti da sistemi di intelligenza artificiale” (protection of workers in decisions assisted by AI systems)normattiva.it, in Italian, accessed 31 October 2026
• Legislative Decree of 7 October 2026, no. 179, article 42, “Intelligenza artificiale, salute e sicurezza nei luoghi di lavoro” (artificial intelligence, health and safety at work)normattiva.it, in Italian, accessed 31 October 2026
• Legislative Decree of 26 May 1997, no. 152, article 1-bis, information on automated decision-making or monitoring systemsnormattiva.it, in Italian, accessed 31 October 2026
• Law of 20 May 1970, no. 300 (Workers' Statute), article 4, “Impianti audiovisivi e altri strumenti di controllo” (audiovisual equipment and other monitoring tools)normattiva.it, in Italian, accessed 31 October 2026
• Emiliano Marchisio, “Le parole dell'IA possono vincolare l'impresa? La risposta del diritto” (Can AI's words bind the company? The legal answer), Agenda Digitale, 28 October 2026agendadigitale.eu, in Italian, accessed 31 October 2026