Many organisations do something about AI literacy. Who checks whether it works?
"We have done something about AI literacy."
You hear this in many organisations. There was a webinar, a demo of a new AI tool, an information session or an internal guideline. The attendance list is filed away. AI literacy seems to have been ticked off the list.
Meanwhile, many of those employees work with AI every day. They use it to write texts and summarise documents, paste data into a chatbot and pass the result on. But who checks whether they also know what they are doing?
Being AI literate means having enough knowledge and skill to work with AI sensibly. You know the opportunities and you know the risks. What AI literacy covers is explained in What is AI literacy? The Council of Europe's answer.
This article asks whether your employees know more about AI after all those activities. And what about you?
What Article 4 does and does not require
This is what Article 4 of the EU AI Act, the European AI law, says:
1. Providers and deployers of AI systems shall take measures to support the development of AI literacy of their staff and other persons dealing with the operation and use of AI systems on their behalf, taking into account their technical knowledge, experience, education and training and the context the AI systems are to be used in, and considering the persons or groups of persons on whom the AI systems are to be used. This obligation does not require providers or deployers to guarantee any specific level of AI literacy of any individual.
What does this mean? If your organisation uses or provides AI, it has to take steps to help employees and others who work with AI on its behalf become AI literate. The organisation looks at what those people already know and what they use AI for. The people on whom that AI is used also count. It does not have to guarantee that everyone reaches a certain level.
This duty has applied since 2 February 2025. A new European law then amended the text of Article 4: the Digital Omnibus. The amended text has applied since 27 July 2026. Since 2 August 2026, the supervisory authority in your country can check whether your organisation complies.
Article 4 does not prescribe a particular training course, test or certificate. In its questions and answers on AI literacy, the European Commission says you do not have to measure employees' knowledge. According to the Commission, you may keep an internal record of your AI literacy activities.
But such a record does not tell you what you really want to know.
What an attendance list does not tell you
An attendance list shows who was there. Not who understands it, let alone who applies it.
Yet the risk depends precisely on understanding and applying. Think of a colleague who pastes customer data into a chatbot on the internet. Or someone who forwards a summary without checking the source. Or someone who copies an answer that sounds confident but is wrong. None of those mistakes show up on an attendance list.
Flawed work produced with AI often looks convincing. It only gets noticed when something goes wrong. In what AI training delivers you can read why poorly prepared employees cost an organisation more than it seems. What you may and may not do with customer data is explained in ChatGPT at work: what you as an employee may and may not do.
What gives the most certainty?
No method gives a guarantee. Each one shows a different piece of the picture.
Do you want to know, across the whole organisation, who knows the basics? Then a well-designed test tells you much more than an attendance list. Everyone is measured against the same bar. You can repeat the test and keep the results.
A good test is not a formality. It contains many different questions, not five questions everyone gets right. The questions are about situations at work, not about definitions. They fit the participant's role and the way they use AI. And the pass mark is set before anyone starts.
Do you want to know whether employees are making progress? Then compare their knowledge before and after the training with the same kind of questions. A test afterwards only shows what someone knows now. Perhaps they already knew it before.
A test does not prove that someone acts correctly in every situation. For that, you have to look at the work. Ask managers after a few weeks what they see. Or review a sample of work done with AI.
A test is not the only way. Other well-designed ways of checking knowledge can work too. But if you check nothing, you do not know whether it worked.
Do you want to take one concrete step? End your most important training courses with a test. Then at least you can see whether the basics are in place. After that, look at the work to see whether those basics are being used.
Three questions for your organisation
Do you work with AI yourself?
Then the same question applies to you. Watching a webinar or reading a guideline is a start.
Do you recognise an answer that sounds confident but is wrong? Do you know which data should not go into a chatbot? Do you check what AI gives you before you pass it on?
Unsure about any of these questions? Start there. Why many people get less out of AI than they could is explained in Most people use AI the wrong way. If you want to get better at it yourself, the Effective AI at Work course can help. You do not need any technical knowledge.
What really matters
Article 4 asks you to take measures. Doing something about AI literacy is therefore a start. What matters more is whether your employees work better with AI as a result. And whether you know it.
Want to know how to go about it? Explore our training courses or get in touch.
Sources
Regulation (EU) 2024/1689 (EU AI Act)
Regulation (EU) 2026/1744 (Digital Omnibus)
AI Literacy - Questions & Answers, European Commission
Written by Rob Ummels in collaboration with Claude (Anthropic). Final editorial responsibility: Rob Ummels.