Design a Deduplication Strategy for Near-duplicate Chunks

When to use this

Use when you need a second opinion on how to design a deduplication strategy for near-duplicate chunks and want the reasoning shown, not just an answer.

Prompt template

Act as a retrieval-augmented generation specialist. Separate retrieval quality from generation quality wherever possible, and be explicit about what data (gold passages, labeled queries, usage logs) each recommendation requires.

Task: Design a deduplication strategy for near-duplicate chunks.

Context to use:
- What the corpus contains: {{corpus_description}}
- What questions it needs to answer: {{use_case}}
- Constraints: {{constraints}}

Response format:
1. Brief recommendation or approach
2. Main deliverable, organized into clear sections
3. Assumptions and unresolved questions
4. Validation or quality checklist

Customize

Preview

Act as a retrieval-augmented generation specialist. Separate retrieval quality from generation quality wherever possible, and be explicit about what data (gold passages, labeled queries, usage logs) each recommendation requires.

Task: Design a deduplication strategy for near-duplicate chunks.

Context to use:
- What the corpus contains: {{corpus_description}} missing
- What questions it needs to answer: {{use_case}} missing
- Constraints: 

Response format:
1. Brief recommendation or approach
2. Main deliverable, organized into clear sections
3. Assumptions and unresolved questions
4. Validation or quality checklist

Usage instructions

Replace the placeholders with specifics from your situation, then treat the output as a first draft to edit, not a final answer.

Example output

A structured write-up covering the approach, the concrete steps or artifacts needed, open questions, and a checklist to confirm it's ready to use.

Limitations

The assumptions listed are the model's best guess when context is thin; confirm them against your actual project before proceeding.

Rate this prompt