AI for Testing Instruction Comprehension

As of 2026-10-19 · the AZET team. We offer this maker-authored educational guide for fun and reference. The instructions, responses, and revisions below are hypothetical; they do not describe a study or demonstrate a learning outcome.

A reader saying that instructions look clear does not tell you which action they would take. A comprehension check asks for an observable choice under a specified condition. It can expose a missing branch or an ambiguous word before you rewrite the entire document.

This article focuses on designing those checks. AI can propose cases and an answer key from supplied instructions. A person must verify that each answer follows from the text and that a reader's response is interpreted fairly.

Define the behavior the text must support

Choose a small, ordinary task with reversible consequences. Write down the decisions a reader should be able to make: select a destination, recognize an exception, stop when information is missing, or explain a sequence.

Separate these decisions from remembered vocabulary. Asking someone to repeat a label may check recall while telling you little about how they would use it. A scenario should supply enough information to make the intended decision without requiring background knowledge you have not provided.

Before generating questions, create an answer-key field for the supporting instruction. If the author cannot point to a sentence that justifies an answer, the question may be testing an unstated expectation.

A hypothetical sorting instruction

Imagine a fictional craft club preparing a tray of reusable paper pieces. Its draft says:

Put blank cards in the blue tray. Put marked cards in the yellow tray. Set damaged cards aside. Count the blue tray when finished.

The author intends damage to take priority, but has not said so. An illustrative test packet contains a blank intact card, a marked intact card, a blank torn card, and a card whose condition cannot be seen.

For the first two, the answer key can cite the blue and yellow instructions directly. The blank torn card matches two rules. The unseen card lacks information needed to decide. A model that confidently assigns both cards would conceal two different problems: rule conflict and missing observation.

A defensible draft key records the torn card as “priority unspecified” and the unseen card as “inspect before deciding; handling rule absent.” These are findings about the document, not reader failures.

The hypothetical author could revise the instruction to: “Inspect each card first. Set damaged cards aside, whether blank or marked. Place the remaining blank cards in blue and the remaining marked cards in yellow. If you cannot inspect a card, leave it unassigned. Count only the cards placed in blue.”

Now the same cases have a textual basis. The revision also clarifies that the final count excludes the unassigned card. That is a specific improvement to inspect, not proof of a general benefit from AI.

Ask for cases before suggested answers

Use this prompt with your own instruction passage:

Design a short comprehension check using only these instructions. Include an ordinary case, an exception, a case matching two rules, and a case with missing information. First present the cases without answers. Then create a separate key containing the expected action, exact supporting sentence, and any ambiguity. If the text does not determine an answer, say so. Do not fill gaps with common practice. Include one explanation question that asks the reader to name the rule they used.

Review the key before showing the cases to anyone. Remove cases whose difficulty comes from irrelevant wording or extra details. Inspect each response individually.

Run a human check without coaching

Give the reader the instructions in the form they would actually receive. State whether they may look back at the text. Keep that condition consistent when comparing a revision with an earlier version.

Ask for an action and a brief explanation. In an illustrative response, someone might place the torn blank card in blue because the blank-card rule appears first. That explanation helps locate the ambiguity. Simply recording “wrong tray” would miss the reason.

Do not explain the intended answer while the person is deciding. Record any clarification they request and distinguish a decision made independently from one made after help. Afterward, invite them to show which sentence led to their choice.

Revise the relevant instruction and try a comparable fresh case. Repeating the identical card may measure memory of your explanation. Keep the new case close enough that it still tests the same rule.

Read results as document feedback

Retain responses and ambiguity notes rather than making broad claims from a percentage. One person's difficulty does not establish how every reader will behave.

Also check the reverse problem: a correct choice made for an unsupported reason. If the explanation relies on a rule absent from the text, the instruction still needs attention. The answer key and the reader's reasoning both matter.

FAQ

Is this an assessment of the reader's ability?

The purpose here is to inspect the instructions. Avoid labeling readers from a short exercise. Their response may reflect ambiguous wording, missing context, or an unfamiliar task rather than a stable ability.

Should every question have one correct answer?

Only when the supplied instructions determine one. An unresolved answer can be the most useful finding. Mark the ambiguity explicitly and repair the source before scoring that item.

Can AI replace a reader in this check?

No human response is established by generated roleplay. Generated interpretations remain hypotheses. Observe an actual reader before drawing conclusions about human behavior.

The historical product baseline

Our introductory AZET article describes the product as of October 3, 2026, including an early-access waitlist. It does not verify today's live status. This workflow is a general illustrative exercise, not a documented AZET capability.

For the AZET waitlist, visit azet.io.