Design an LLM-as-judge Rubric for Subjective Quality
Turns a short description into a concrete plan to design an LLM-as-judge rubric for subjective quality, with assumptions called out instead of silently guessed.
When to use this
Use before committing time to design an LLM-as-judge rubric for subjective quality by hand, to get a first structured pass to react to.
Prompt template
Act as an AI evaluation designer. Favor concrete, checkable criteria over vague quality language, and state what data or labeling effort each part of the plan requires.
Task: Design an LLM-as-judge rubric for subjective quality.
Context to use:
- Feature description: {{feature_description}}
- Primary risk to guard against: {{primary_risk}}
- Number of test cases: {{test_case_count}}
Response format:
1. Brief recommendation or approach
2. Main deliverable, organized into clear sections
3. Assumptions and unresolved questions
4. Validation or quality checklistCustomize
Feature description is required
Primary risk to guard against is required
Preview
Act as an AI evaluation designer. Favor concrete, checkable criteria over vague quality language, and state what data or labeling effort each part of the plan requires.
Task: Design an LLM-as-judge rubric for subjective quality.
Context to use:
- Feature description: {{feature_description}} missing
- Primary risk to guard against: {{primary_risk}} missing
- Number of test cases: 10
Response format:
1. Brief recommendation or approach
2. Main deliverable, organized into clear sections
3. Assumptions and unresolved questions
4. Validation or quality checklistUsage instructions
Provide as much real context as you have; leave a field blank only if it genuinely doesn't apply, and check the assumptions list afterward.
Example output
A structured write-up covering the approach, the concrete steps or artifacts needed, open questions, and a checklist to confirm it's ready to use.
Limitations
Treat the output as a structured starting point; it does not replace review from someone with direct context on the system.
Rate this prompt
Not yet rated