Ground model behavior across modalities.
A multimodal task might ask a model to locate evidence in an image, follow an event in video, or interpret a chart. The source, question, and review criteria need to specify what the answer must establish.
We build tasks across image, audio, video, and text. Experts review the response and its connection to the source, using annotation formats suited to the evidence.
Make source grounding part of the specification
Specify the region, series, time interval, or other source evidence the answer should use. Preserve enough context for a reviewer to verify it.
- Connect labels or answers to the source modality.
- Define spatial, temporal, or document references where needed.
- Preserve relevant instructions and supporting context.
Review across modalities and task types
Assess the intended capability, such as intent, scene reasoning, or event interpretation. Include difficult and ambiguous cases alongside recognition tasks.
- Separate perception errors from reasoning errors.
- Calibrate reviewers on representative source material.
- Keep evaluation examples distinct from training material.
What a program can include
Scope can include:
- Scoped multimodal tasks and source references
- Expert-reviewed annotations or grounded answers
- Modality-specific review guidance
- Evaluation criteria aligned with the target capability
Illustrative task / Not a production record
Make the objective concrete.
- Objective
- Answer a question about a chart in a supplied report.
- Context
- The report page, chart, question, and required answer format.
- Review
- Verify that the model reads the relevant visual elements and grounds its explanation in the source.
Common questions
Can a task combine several modalities?
Yes. Define the source material and the relationships the model needs to understand. The review criteria should make clear which inputs support the expected answer.
Are annotation tools available to explore?
The RDL platform catalog includes text, image, audio, video, and human-feedback demos. They help illustrate possible workflows; the production scope is agreed with the team.