> For the complete documentation index, see [llms.txt](https://examind.gitbook.io/v1/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://examind.gitbook.io/v1/feedback-machines/review-adjust-and-approve/human-in-the-loop.md).

# Human-in-the-loop

Review every submission and evaluation yourself, correcting the machine as you go and leaving each student a comment in your own words.

In a human-in-the-loop workflow you read **every** student's work and every evaluation before any grade is released. The machine does the first pass; you confirm or correct all of it.

This is the workflow to use when a machine is new. It does two jobs at once: it gets grades right for this cohort, and it teaches the machine your judgment — every correction you make from a real submission tightens the alignment between the criteria and how you actually grade.

## When to use it

* **A machine's first semester**, before you have evidence it grades the way you would.
* **After a material change** to the assignment, the rubric, or what you covered in class.
* **Whenever a personal touch matters** — a capstone, a small cohort, a course where students expect to hear from you directly.

## The workflow

{% stepper %}
{% step %}

### Open the first submission

From the [Results page](/v1/feedback-machines/review-adjust-and-approve.md), select a student's name to open the review workspace, then use **Previous** and **Next** to move through the class without going back to the class view each time.
{% endstep %}

{% step %}

### Read the work before the verdict

Collapse the **Evaluation** pane and read the student's submission on its own first, forming your own view before you see the machine's. Your pane layout stays as you set it while you move between students, so you can go through the whole class this way and only then open the evaluations.

This is worth the discipline: reading the score first makes it much harder to notice that it's wrong.
{% endstep %}

{% step %}

### Compare, and correct what's off

Open the Evaluation pane and compare. Use the locate controls to jump from a criterion to the exact passage it refers to, so you're checking the reasoning against the evidence rather than accepting it.

Where the machine misjudged something, [adjust the criteria from that submission](/v1/feedback-machines/review-adjust-and-approve.md#adjust-the-machine-from-real-submissions) — the correction then applies to everyone, which is the point.
{% endstep %}

{% step %}

### Leave the student something of yours

Edit the **key takeaways** to add your own words: what you want this particular student to carry into the next assignment. Your text replaces the machine's and is attributed to you where the student reads it.
{% endstep %}

{% step %}

### Approve

Approving stamps the result with your name and the date, releases it to the student, and opens the next submission still awaiting approval — so a pass through the class stays one continuous loop.
{% endstep %}
{% endstepper %}

## What you get from it

* **Accuracy at scale.** You catch the cases the machine got wrong, and each correction improves every other student's grade at the same time.
* **A machine you can trust next time.** A semester of in-the-loop review is what earns the confidence to move to a [human-on-the-loop](/v1/feedback-machines/review-adjust-and-approve/human-on-the-loop.md) workflow later.
* **A personal comment on every grade.** Students see a named human's words, not only a machine's. In some courses that matters a great deal.

{% hint style="info" %}
The trade-off is time. Once a machine has demonstrated it grades in line with your intentions — and the assignment hasn't materially changed — [human-on-the-loop](/v1/feedback-machines/review-adjust-and-approve/human-on-the-loop.md) gets you most of the accuracy for a fraction of the effort.
{% endhint %}