Evaluation Methodologies

We used the RAGAS framework to evaluate our RAG systems, focusing on the following metrics:

• Faithfulness: Measures how well the generated answer aligns with the provided context
• Answer Relevance: Assesses how well the answer addresses the original question
• Context Relevance: Evaluates the relevance of retrieved context chunks
• Context Recall: Measures how well the retrieved context aligns with the ground truth answer

Evaluation Results

[This section needs to be filled with numerical results after testing]

Rationale for Choosing Hybrid RAG

We ultimately chose the Hybrid RAG approach for the following reasons:

• Comprehensive Information Capture: Captures both structured relationships and semantic nuances
• Enhanced Entity Representation: Maintains clear representation of legal entities and relationships
• Improved Semantic Understanding: Ensures deep understanding of complex legal language
• Flexible Querying: Allows for both structured and open-ended legal queries
• Improved Accuracy: Leverages both structured and unstructured information for comprehensive responses