The answer is simple! Every non-military and non-intelligence system that is used for public good: i.e. by government, should be open sourced and it's sources should be in country communication office
with all customizations made to the system. And when it's not a system, but a service is employed, it's easy to enforce compliance procedures against which such services could be tested.
So that without special clearance you could test results on your data set and or verification inputs in public test environment and results received from production.
If you would carefully think, than you'll find that government have no reason to buy system or service that could not be transparently checked. Can the EU make AI “trustworthy”? No - but they can make it just - European Digital Rights (EDRi)
European Digital Rights (EDRi) submitted its answer to the European Commission’s consultation on the AI White Paper.