Collect independent labels for the same text. Compare span boundaries, entity classes, or item categories and send disagreements to a reviewer with the original passage in context.
Check supported text annotations against a hidden approved answer key. Detect differences in spans, classes, relations, or properties and route the comparison through the configured quality threshold.
Connect text annotation, consensus, benchmarks, and review. Route incomplete spans or inconsistent categories back for correction, then review the revised item before completion.
Review entity highlights and relationship endpoints in context. Correct boundaries, labels, and properties before approving an annotation or requesting rework.
Independent annotations reveal where annotators interpret the same text differently. Review the disagreement against the labeling guidelines and refine the result.
It is the review of text labels for correct span boundaries, entity classes, relation structure, and category or property consistency. Unitlab keeps review connected to the exact text used to create the annotation. QA overview
QA Workflows connect text annotation, Consensus, Quality Gates, and Review. Configure Pass and Fail routes for comparison stages, then approve reviewed items or return incomplete spans, incorrect relations, and inconsistent categories for correction. Revised annotations can follow another review, while workflow history retains the attempts and decisions. QA workflows
Consensus gathers independent annotations of the same text and compares supported spans, labels, relations, and properties under your agreement settings. Reviewers can inspect the competing interpretations alongside the original passage and refine the representative result. Agreement measures consistency between submissions; it does not settle linguistic ambiguity or establish correctness by itself. Consensus guide
A Quality Gate compares supported text annotations with an approved reference key hidden from annotators. It checks the submission against that benchmark and uses the configured threshold for Pass or Fail. Keep Not evaluated separate when comparison data is unavailable, with review or rework routes defined for the relevant outcomes. Quality Gate guide
Reviewers inspect the exact source characters and surrounding sentence before correcting partial names, overlong spans, missed entities, or incorrect classes. Define whether titles, punctuation, and modifiers belong inside each entity type. Those boundary conventions help the team apply the same labeling policy to similar examples. Review stages
Reviewers check that each relation connects the intended source and target entities with the correct label and direction. The original passage and ontology definition provide the evidence for that decision. Supported annotation comparisons evaluate relations after matching endpoints, while reviewers resolve cases where the text leaves the relationship ambiguous. Review stages
Configure intent or document categories as item properties and review them against the full source text. Define the difference between neighboring categories and include examples of borderline cases in the guidelines. Reviewers can apply those definitions consistently while keeping whole-item categories separate from entity labels attached to individual spans. Review stages
Required properties and other configured ontology rules can produce validation findings for reviewers to inspect. Use those findings to complete missing values or correct inconsistent selections before approval. A finding may be advisory rather than blocking every save, so teams should also define what reviewers must resolve. Ontology documentation
Supported comparisons evaluate annotation structures and stored property values. Scalar text properties use exact equality, so differently worded values can differ even when a reader considers their meaning similar. Reviewers should apply the project's accepted wording and category rules to resolve paraphrases, ambiguity, and other cases requiring interpretation. Review stages
We use cookies to improve your experience. See our Privacy Policy and Cookie Policy.