The ISTQB Certified Tester AI Testing Exam (CT-AI)

The syllabus defines clustering as:

“Clustering: This is when the problem requires the identification of similarities in input data points that allows them to be grouped based on common characteristics or attributes. For example, clustering is used to categorize different types of customers for the purpose of marketing.”

(Reference: ISTQB CT-AI Syllabus v1.0, Section 3.1.2, page 26 of 99)

According to the syllabus, pre-trained models often inherit biases and limitations from the data and processes used in their original training, which may not align with the new use case. Specifically, the syllabus states:

"When using a pre-trained model, the training data and process cannot be fully controlled or known by the user of the model. As a result, the model can inherit biases or inaccuracies that were part of its original development and training process."

(Reference: ISTQB CT-AI Syllabus v1.0, Section 1.8.3)

Section 2.2 – Data Preparation and 4.1 – Challenges in Testing AI-Based Systems describe difficulties in obtaining and managing large, representative datasets. AI-based systems require realistic, diverse, and representative data reflecting real-world variations. The syllabus emphasizes that assembling such datasets is time-consuming, resource-intensive, and often constrained by availability, privacy, or domain complexity. Option B directly corresponds to these documented challenges.

Option A is incorrect: using the same implementation risks defect masking , not preventing it; the syllabus warns against this practice. Option C is incorrect because real-world data naturally evolves, and the syllabus notes that drift is normal; expecting stable input data contradicts operational reality. Option D is incorrect: although data privacy is important, the syllabus does not claim that artificially generated data always requires legal approval, nor that sanitization/encryption is mandatory for synthetic data.

Thus, Option B accurately reflects syllabus-defined difficulties in producing representative test data.

The question asks which combination of tests would be most appropriate to include in the strategy for optimal detection in a workflow system using multiple ML models.

Pairwise testing of combinations (I) : This method is useful for testing interactions between different components in the workflow to ensure they work well together, identifying potential issues in the integration.

Testing each individual model for accuracy (II) : Ensuring that each model in the workflow performs accurately on its own is crucial before integrating them into a combined workflow.

A/B testing of different sequences of models (III) : This involves comparing different sequences to determine which configuration yields the best results. While useful, it might not be as fundamental as pairwise and individual accuracy testing in the initial stages.

The ISTQB Certified Tester AI Testing Syllabus v1.0 explicitly categorizes different AI implementation technologies in Section 1.4 – AI Technologies . Within this section, AI methods are grouped into categories, one of which is “Reasoning techniques.” These reasoning techniques include rule engines, deductive classifiers, case-based reasoning, and procedural reasoning . Because deductive classifiers are directly listed under this set of reasoning approaches, they are recognized as a reasoning-based AI technology.

Reasoning techniques differ from machine learning approaches because they rely on structured, predefined rules or logic to reach conclusions. Deductive classifiers use logical inference and symbolic reasoning to classify inputs by applying encoded knowledge. This makes them fundamentally different from statistical or data-driven ML algorithms.

The other options— Linear regression , Random Forest , and Genetic algorithms —are listed by the syllabus as machine learning techniques , not reasoning methods. Linear regression performs numerical prediction, Random Forest is an ensemble decision-tree ML model, and genetic algorithms are optimization-based ML approaches inspired by evolutionary processes. None of these involve symbolic logical deduction.

Thus, based on the authoritative definitions in the syllabus, Deductive classifiers (Option A) is the only technology classified as a reasoning technique.

 Labelling Approaches: Among the options provided, pre-labeled datasets require the least time and effort because the data has already been labeled, eliminating the need for further manual or automated labeling efforts.

 Reference: ISTQB_CT-AI_Syllabus_v1.0, Section 4.5 Data Labelling for Supervised Learning, which discusses various approaches to data labeling, including pre-labeled datasets, and their associated time and effort requirements​​.

The ISTQB CT-AI syllabus indicates in Section 5.2 – AI for Testing that AI-generated test cases may not come with predefined expected results . This is because test-case generation methods—such as evolutionary algorithms, reinforcement learning, or clustering-based sampling—produce inputs , but the tester must still determine the correct outputs. Therefore, Option C is correct: expected results may not be available, especially when AI produces novel or previously unseen input combinations.

Option A is incorrect: AI-based test generation can improve coverage by exploring large input spaces. Option B is incorrect because the need for oracles does not depend on whether the test case is AI-generated. Option D is incorrect because the syllabus allows using a model as an oracle in back-to-back testing , provided limitations are understood.

Therefore, Option C is the correct syllabus-aligned statement.

According to the syllabus, conventional AI systems are limited to specific, pre-defined tasks and do not have generalized intelligence:

"Conventional AI systems are limited in their scope and typically only perform specific tasks within the domain for which they have been designed. They do not exhibit general AI behavior."

(Reference: ISTQB CT-AI Syllabus v1.0, Section 1.2)

The ISTQB CT-AI syllabus explains ML performance metrics in Section 3.2 – Evaluating ML Models . For clustering , which is an unsupervised learning method, the syllabus lists metrics such as intra-cluster distance , inter-cluster distance , and coherence measures. Intra-cluster metrics evaluate how close data points are within a cluster, which directly corresponds to Option A.

Option B is incorrect because R-squared is a regression metric measuring goodness-of-fit, not classification performance, and has no connection to ROC curves. Option C is wrong because the silhouette coefficient is also a clustering metric, measuring cohesion vs. separation—not regression accuracy. Option D is incorrect because ROC curves evaluate binary or multiclass classification , not clustering.

Thus, Option A is the only accurate statement based on the syllabus.