The Artificial Intelligence Governance Professional (AIGP)

When anew versionof a third-party model is released, the deployer must ensure it still meets safety, performance, and compliance requirements — which calls for aformal audit.

From theAI Governance in Practice Report 2025:

“Any updates or changes to AI systems should trigger a re-evaluation to ensure continued compliance and performance.” (p. 12)

“Post-market monitoring includes reassessing the impact of updated models or retraining.” (p. 35)

The AI risk that would not have been identified during the procurement process based on the categories of information requested by the third-party consultant is security. The consultant focused on accuracy rates, diversity of training data, and system functionality, which pertain to performance and fairness but do not directly address the security aspects of the AI system. Security risks involve ensuring that the system is protected against unauthorized access, data breaches, and other vulnerabilities that could compromise its integrity. Reference: AIGP Body of Knowledge on AI Security and Risk Management.

The core ethical concern when deploying diagnostic AI in a healthcare setting is ensuringfairness and accuracy across diverse patient populations. If the AI is trained on a dataset that isnot representativeof the population it will serve, it risks reinforcing health disparities and leading to misdiagnoses.

From theAI Governance in Practice Report 2025:

“Training datasets lacking in diversity can produce outputs that systematically underperform for certain groups… this can lead to inaccurate or biased outcomes in healthcare settings.” (p. 41)

“Bias, discrimination and fairness challenge… inadequate or nonrepresentative training data can result in AI systems that propagate historical disparities.” (p. 42)

While physician oversight may reduce risk,biased data can still shape clinical decision-making.

A– Economic benefit is not central to ethical risk here.

C– Important but less critical than data representativeness.

D– Error rate matters but is addressed via validation; it’s not the core ethical issue.

The most challenging aspect of managing a data access request in this scenario is dealing with unstructured data that cannot be easily disentangled from other data, including information about other individuals. Unstructured data, such as free-text inputs or social media posts, often lacks a clear structure and may be intermingled with data from multiple individuals, making it difficult to isolate the specific data related to the requester. This complexity poses significant challenges in complying with data access requests under privacy laws. Reference: AIGP Body of Knowledge on Data Subject Rights and Data Management.

The third-party consultant asked each vendor for information about the diversity of their datasets to assist in ensuring the fairness of the AI system. Diverse datasets help prevent biases and ensure that the AI system performs equitably across different demographic groups. This is crucial for a law enforcement application, where fairness and avoiding discriminatory practices are of paramount importance. Ensuring diversity in training data helps in building a more just and unbiased AI system. Reference: AIGP Body of Knowledge on Ethical AI and Fairness.

The NIST AI Risk Management Framework’s Govern function emphasizes the importance of establishing accountability structures that empower and train cross-functional teams. This is crucial because cross-functional teams bring diverse perspectives and expertise, which are essential for effective AI governance and risk management. Training these teams ensures that they are well-equipped to handle their responsibilities and can make informed decisions that align with the organization’s AI principles and ethical standards. Reference: NIST AI Risk Management Framework documentation, Govern function section.

Once a company moves from being adeployerto also acting as aprovider or developer, it assumesnew obligationsunder regulations like the EU AI Act. One of the core requirements for providers is to produce and maintaintechnical documentation, including descriptions of the model, associated risks, and mitigation strategies.

From theAI Governance in Practice Report 2025:

“Providers of high-risk AI systems must draw up technical documentation demonstrating the system’s conformity with the requirements... including potential risks and safeguards applied.” (p. 34)

“This documentation must be available before placing the system on the market.” (p. 35)

Retraining an LLM (Large Language Model) is primarily done to improve or maintain its performance as data changes over time, to fine-tune it for specific use cases, and to incorporate new data interpretations to enhance accuracy and relevance. However, ensuring interpretability of the model's predictions is not typically a reason for retraining. Interpretability relates to how easily the outputs of the model can be understood and explained, which is generally addressed through different techniques or methods rather than through the retraining process itself. References to this can be found in the IAPP AIGP Body of Knowledge discussing model retraining and interpretability as separate concepts.

The EU AI Act outlines what must be included intechnical documentationfor high-risk systems. These requirements are designed to supportconformity assessment, transparency, and traceability.

From theAI Governance in Practice Report 2025:

“It mandates drawing up technical documentation… must include a general description of the AI system, the intended purpose, and a detailed description of the elements and development process.” (p. 34)

“Documentation… includes training, testing, evaluation procedures, andappropriateness of performance metrics.” (p. 34–35)

Therisk management systemis addressed separately through arisk management plan, not within the technical documentation itself.

Thus:

A, C, and Dare explicitly required in thetechnical documentation.

B, while important, is part of therisk management process, not a required section oftechnical documentation.