Bring professional judgment into model development.
Professional work depends on terminology, evidence standards, and context that vary by field. A specialist may need to account for a jurisdiction, a particular audience, or the limits of the source material.
Practitioners create tasks, example responses, preference comparisons, and evaluations. We define the specialty, intended users, inputs, and review standards before collecting examples.
Scope the expertise to the actual work
Identify the decisions the model will support and the expertise needed to assess them. Specify the specialty, jurisdiction, and application where relevant.
- Specify the professional workflow and its constraints.
- Define reviewer qualifications for the task.
- Use representative documents and expected deliverables.
Preserve grounding and uncertainty
Review claims against the supplied evidence. Distinguish facts, assumptions, and missing information, including cases where the model should ask for more context.
- Require support for source-dependent statements.
- Review omissions as well as explicit errors.
- Document disagreement and escalation criteria.
What a program can include
Scope can include:
- Expert demonstrations or preference examples
- Domain-specific review rubrics
- Source references and reviewer rationales
- Evaluation tasks aligned to the agreed scope
Illustrative task / Not a production record
Make the objective concrete.
- Objective
- Extract the required terms from a supplied professional document.
- Context
- The source document, extraction schema, and explicit review instructions.
- Review
- Check field accuracy, source references, completeness, and whether missing information is clearly marked.
Common questions
Can the same domain program include training and evaluation?
Yes. Plan separate collections for model learning and held-out evaluation, with shared quality definitions but distinct examples to make the evaluation meaningful.
How are domain requirements agreed?
Bring representative tasks, source materials, and acceptance criteria. The team can discuss the required expertise, review process, and deliverable format before collection begins.