mirror of
https://github.com/toeverything/AFFiNE.git
synced 2026-07-24 22:09:08 +08:00
feat(server): faster reranking based on confidence (#12957)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Improved document reranking with a more streamlined and accurate
scoring system.
* Enhanced support for binary ("yes"/"no") document relevance judgments.
* **Improvements**
* Simplified user prompts and output formats for reranking tasks, making
results easier to interpret.
* Increased reliability and consistency in document ranking results.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This commit is contained in:
@@ -342,57 +342,11 @@ Convert a multi-speaker audio recording into a structured JSON format by transcr
|
||||
messages: [
|
||||
{
|
||||
role: 'system',
|
||||
content: `Evaluate and rank search results based on their relevance and quality to the given query by assigning a score from 1 to 10, where 10 denotes the highest relevance.
|
||||
|
||||
Consider various factors such as content alignment with the query, source credibility, timeliness, and user intent.
|
||||
|
||||
# Steps
|
||||
|
||||
1. **Read the Query**: Understand the main intent and specific details of the search query.
|
||||
2. **Review Each Result**:
|
||||
- Analyze the content's relevance to the query.
|
||||
- Assess the credibility of the source or website.
|
||||
- Consider the timeliness of the information, ensuring it's current and relevant.
|
||||
- Evaluate the alignment with potential user intent based on the query.
|
||||
3. **Scoring**:
|
||||
- Assign a score from 1 to 10 based on the overall relevance and quality, with 10 being the most relevant.
|
||||
- Each chunk returns a score and should not be mixed together.
|
||||
|
||||
# Output Format
|
||||
|
||||
Return a JSON object for each result in the following format in raw:
|
||||
{
|
||||
"scores": [
|
||||
{
|
||||
"reason": "[Reasoning behind the score in 20 words]",
|
||||
"chunk": "[chunk]",
|
||||
"targetId": "[targetId]",
|
||||
"score": [1-10]
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
# Notes
|
||||
|
||||
- Be aware of the potential biases or inaccuracies in the sources.
|
||||
- Consider if the content is comprehensive and directly answers the query.
|
||||
- Pay attention to the nuances of user intent that might influence relevance.`,
|
||||
content: `Judge whether the Document meets the requirements based on the Query and the Instruct provided. The answer must be "yes" or "no".`,
|
||||
},
|
||||
{
|
||||
role: 'user',
|
||||
content: `
|
||||
<query>{{query}}</query>
|
||||
<results>
|
||||
{{#results}}
|
||||
<result>
|
||||
<targetId>{{targetId}}</targetId>
|
||||
<chunk>{{chunk}}</chunk>
|
||||
<content>
|
||||
{{content}}
|
||||
</content>
|
||||
</result>
|
||||
{{/results}}
|
||||
</results>`,
|
||||
content: `<Instruct>: Given a web search query, retrieve relevant passages that answer the query\n<Query>: {query}\n<Document>: {doc}`,
|
||||
},
|
||||
],
|
||||
},
|
||||
|
||||
Reference in New Issue
Block a user