Indian court judgments are lengthy, complex, and composed of diverse sections serving different purposes—such as presenting facts, arguments, legal reasoning, and decisions. To analyze these documents more effectively, each sentence can be labeled with a rhetorical role, helping distinguish between the various components of the legal narrative. This kind of annotation supports better organization, retrieval, and interpretation of legal texts, making it easier to automate tasks like summarization and precedent extraction.

Structuring Judicial Text through Rhetorical Roles

A dataset of over 7,000 Indian legal case documents has been annotated at the sentence level, comprising more than 300,000 sentences tagged with seven rhetorical roles. These roles include Facts, Arguments, Ratio, Ruling by Precedent, Ruling by Present Court, Precedent, and Statute. The annotations were curated by legal experts using a taxonomy grounded in legal theory. This corpus captures the structural and rhetorical richness of Indian legal documents, addressing a gap in global legal NLP resources. It enables the development and benchmarking of models for understanding the semantic flow of legal judgments in India's diverse legal system.

A Large-Scale Annotated Corpus for Indian Legal Judgments

Despite promising results, rhetorical role classification in legal documents remains a non-trivial task due to several inherent challenges. First, some rhetorical categories are semantically close, making it difficult even for human annotators to distinguish them reliably. For example, “Ratio” (reasoning) and “Ruling by Present Court” may share similar language and logical structure. Second, the dataset suffers from class imbalance, where roles such as “Facts” occur far more frequently than roles like “Precedent” or “Statute.” This imbalance can lead models to be biased toward predicting the dominant classes, thereby reducing their accuracy on the underrepresented ones. Additionally, legal language is often formal, archaic, and context-dependent, which makes it difficult for general-purpose language models to perform well without domain-specific tuning. Addressing these issues requires not just improved modeling techniques but also thoughtful annotation guidelines, better sampling strategies, and domain-aware model pretraining.

Challenges in Legal Role Classification

Various machine learning models have been evaluated for rhetorical role classification in legal texts. Traditional models like BiLSTM-CRF provide a baseline, while transformer-based models such as BERT and Longformer achieve better results by leveraging deeper contextual understanding. Advanced approaches like hierarchical and graph-based models further enhance performance by capturing inter-sentence dependencies and document structure. Instruction-tuned LLMs demonstrate strong zero-shot and few-shot capabilities but are outperformed by models that incorporate structural cues. This analysis highlights the trade-offs between computational resources, training data, and model performance.

Comparing Modeling Approaches for Role Classification

Automatically identifying rhetorical roles in legal texts enables smarter legal information systems. These tools improve summarization, precedent detection, and argument mining, helping legal professionals handle complex judgments more efficiently. Applying them to Indian cases supports broader efforts to digitize and democratize legal knowledge. This also sets the stage for future research in areas like multilingual legal processing and automated compliance. Overall, it moves us closer to making law more accessible and technology-driven.

Implications for Legal AI and Document Understanding

This study presents a large-scale annotated dataset of over 7,000 Indian legal judgments, segmented into seven rhetorical roles. It evaluates various machine learning models, including transformers and graph neural networks, for classifying these roles. The findings emphasize the importance of context and structure in improving legal document analysis.

LegalSeg: Unlocking the Structure of Indian Legal Judgments Through Rhetorical Role Classification

LegalSeg

To address the scarcity of large-scale, diverse legal datasets in India, a comprehensive resource named NyayaAnumana has been developed. It includes over 702,000 preprocessed case documents sourced from Supreme Court, High Courts, District Courts, Tribunal Courts, and daily orders. This wide coverage ensures representation across judicial levels, case types, and legal complexities. NyayaAnumana stands out as the most extensive and diverse dataset for Indian legal judgment prediction, offering a solid foundation for building robust and realistic AI models tailored to the nation's legal landscape.

Building a Diverse Legal Dataset

To harness the potential of the NyayaAnumana dataset, a domain-specific generative language model named INLegalLLaMA was introduced. Based on the LLaMA architecture, this model undergoes a two-step training process: continued pretraining on Indian legal texts to embed deep domain knowledge, followed by supervised fine-tuning on judgment prediction tasks. INLegalLLaMA is designed to understand the semantic and structural intricacies of Indian judicial language, allowing it to generate legally coherent predictions and explanations, which are vital in high-stakes applications like legal decision support systems.

Developing a Specialized Legal Language Model

The study employs a dual-task framework for legal judgment prediction and explanation. For prediction, transformer-based models like InLegalBERT and LLMs perform binary/ternary classification by splitting lengthy legal documents into 512-token chunks with overlapping windows, addressing context limitations. The explanation component leverages INLegalLlama to generate human-readable justifications, evaluated using lexical metrics (ROUGE, BLEU), semantic similarity (BERTScore), and expert reviews. Expert evaluations rated the model’s explanations 3.54/5 on a Likert scale, showcasing near-expert alignment in rationale clarity.

Methodology for Judgment Prediction and Explanation

Key findings reveal 10–15% accuracy gains for models trained on NyayaAnumana compared to ILDC-trained counterparts, underscoring the dataset’s superior quality. INLegalLlama achieved 76.05% accuracy in explanation tasks, outperforming LLaMa-2 (57.26%) and GPT-3.5, with tests confirming its generalization to future cases. Incorporating diverse court hierarchies (e.g., High Courts) enhanced model robustness, while temporal validation ensured reliable performance on unseen data from 2020–2024.

 Key Findings and Model Performance

The work addresses India’s judicial backlog by automating judgment prediction while maintaining transparency through explanations. Future steps include expanding the dataset to regional languages (e.g., Hindi, Bengali) and refining quantization techniques for larger models. The authors emphasize ethical AI deployment, advocating human oversight and open access via GitHub. The INLegalLlama model is available on Hugging Face, and the NyayaAnumana dataset is accessible through Huggingface. This initiative aims to democratize legal AI research and applications in India, fostering collaboration and innovation in the field.

 Implications and Future Directions

INLegalLlama is a specialized language model fine-tuned on Indian legal texts, designed to enhance predictive accuracy and generate coherent explanations for judgment prediction tasks. Paired with NyayaAnumana, a vast dataset of 700,000+ Indian legal cases from diverse courts, it enables robust AI models, addressing judicial inefficiencies and advancing legal AI research in India.

NyayaAnumana & INLegalLlama: The Largest Indian Legal Judgment Prediction Dataset and Specialized Language Model for Enhanced Decision Analysis

INLegalLlama

PredEx comprises 15,222 annotated Indian legal cases from the Supreme Court and High Courts, meticulously curated to include both binary judgment predictions (accept/reject) and expert-written explanations. Legal students and professionals annotated key sentences influencing decisions, ensuring balanced representation across case types and outcomes. The dataset’s preprocessing involved removing ambiguous or overly brief cases, resulting in a robust 80-20 train-test split. This resource addresses gaps in prior datasets like ILDC by prioritizing explanatory depth alongside prediction, enabling models to learn context-aware legal reasoning.

Creation of the PredEx Dataset

The methodology leverages instruction-tuning on LLMs like Llama-2-7B, tailoring prompts to predict outcomes and justify decisions using case proceedings. Training involves two phases: fine-tuning on PredEx for prediction and integrating explanation-specific instructions to mimic judicial reasoning. Models process 512-token chunks with overlapping windows to handle lengthy documents, prioritizing the final 1,000 tokens for contextual relevance. This approach reduces hallucinations and improves coherence, achieving 76.05% accuracy in explanation tasks compared to baseline models like GPT-3.5 and vanilla Llama-2.

Instruction-Tuning for Legal LLMs

A novel dual-task framework simultaneously predicts case outcomes and generates explanations by identifying pivotal legal arguments. Lexical metrics (ROUGE, BLEU) and semantic similarity scores (BERTScore) validate explanation quality, while expert evaluations using a 1–5 Likert scale assess practical utility. The framework’s strength lies in its ability to highlight statutory references, precedent citations, and judicial logic from case texts. For example, models trained on PredEx show 10–15% accuracy gains over ILDC-trained counterparts, demonstrating superior generalization across diverse court hierarchies and temporal ranges (2020–2024).

Dual-Task Framework for Prediction and Explanation

The study emphasizes ethical AI deployment, advocating human oversight to mitigate risks of erroneous or biased predictions. By open-sourcing PredEx and model code, the work encourages reproducibility and further research into regional language adaptations (e.g., Hindi, Bengali). Future directions include reinforcement learning from human feedback (RLHF) to refine explanations and quantization techniques for deploying larger models. These efforts aim to reduce India’s judicial backlog while ensuring AI complements, rather than replaces, human legal expertise.

Ethical and Practical Implications

Rigorous evaluations reveal instruction-tuned models reduce hallucination rates by ~40% compared to pre-trained LLMs, as measured by expert reviews. Post-processing filters truncate repetitive or contradictory outputs, ensuring coherence in predictions and explanations. Semantic metrics like BERTScore (0.69) confirm alignment with ground-truth reasoning, while temporal validation tests prove model robustness on future cases. The integration of domain-specific knowledge during fine-tuning addresses limitations in general-purpose LLMs, enabling reliable application in high-stakes legal contexts.

Evaluation and Mitigation of Hallucinations

PredEx introduces a large expert-annotated dataset for legal judgment prediction and explanation in India. By instruction-tuning LLMs, it enhances predictive accuracy and generates clear explanations, addressing judicial inefficiencies. The study emphasizes ethical AI deployment and advances AI's role in legal transparency and decision-making.

Legal Judgment Reimagined: PredEx and the Rise of Intelligent AI Interpretation in Indian Courts

PredEx

The study utilized a custom dataset comprising 13,000+ annotated legal documents from Indian courts, split into preamble and judgment sections. Preprocessing involved removing redundant spaces, non-alphanumeric characters, and aligning entity spans after cleaning. For L-NER, 18 entity types (e.g., statute, precedent, court) were annotated, with 80% of data used for training and 20% for validation. Unique challenges included handling lengthy documents (avg. 2,315 words) and aligning subword tokenization with part-of-speech tags for BERT-CRF models.

Dataset Curation and Preprocessing

A hybrid approach combined spaCy’s RoBERTa transformer with Law2Vec embeddings and a BERT-CRF model enhanced by part-of-speech tags. The spaCy pipeline achieved 85.5% F1 on preamble entities, while BERT-CRF scored 83.7% on judgment entities. Model fusion merged overlapping entity predictions, improving accuracy by 2%. Challenges included sparse entity distributions and misaligned tokenization, resolved through regex-based span correction and POS-tag embeddings.

Innovative Model Fusion for Legal Entity Recognition

Legal documents were processed using XLNet-large and hierarchical transformers with 512-token chunks and 100-token overlaps. Models trained on the last 510 tokens achieved 76.49% F1, outperforming InLegalBERT and RoBERTa. Despite hierarchical architectures (BiGRU + attention) underperforming due to limited data, truncation strategies prioritized context from concluding sections, aligning with findings that critical reasoning appears in final segments.

Hierarchical Transformers for Judgment Prediction

The research developed AI systems to predict court decisions and explain them in simple terms. By analyzing the concluding sections of judgments (last 300–550 words), where judges summarize key rulings, the models generated explanations matching 47% of human-written justifications. For outcome prediction, phrases like “case disposed of” were used to classify results with 83% accuracy. The system also categorized judgment text into roles like “court’s ruling,” finding 22.8% focused on final decisions. Simpler, expert-validated methods were prioritized over complex techniques to ensure clarity and practicality for real-world legal use.

Explanation Generation for Court Judgments

The models excelled in SemEval-2023 Task 6, achieving 76.49% F1 in Legal Judgment Prediction and 85.5% F1 in Legal NER using fused transformer-based approaches. Explanation extraction via span analysis yielded a ROUGE-2 score of 0.0473, with keyword-based outcome identification reaching 83% accuracy. Despite challenges like sparse entities and lengthy texts, the pipeline ranked 1st in explanation tasks, showcasing scalable solutions for India's judicial system.

Evaluation and Results

This research focuses on automating legal text analysis via tasks like Legal NER, Judgment Prediction, and Explanation Extraction. Using transformer-based models and hierarchical architectures, it achieved top rankings in SemEval-2023 Task 6. The study highlights scalable AI solutions for India's judicial backlog, emphasizing domain-specific training and practical explanation methods.

Nonet at SemEval-2023 Task 6: Methodologies for Legal Evaluation

Nonet

The foundation of our system rests on a rich dataset curated from authentic Indian bail judgments. This data includes details such as the type of bail application, statutes invoked, age and health of the accused, days in custody, and past criminal records. Given the unstructured nature of judicial documents, we employed natural language processing (NLP) techniques to extract structured features using custom rule-based parsers and Named Entity Recognition (NER). Each judgment was then converted into a standardized JSON schema capturing key legal and factual attributes. We ensured balanced representation across bail types and verdicts to avoid skewed learning, and applied preprocessing methods like token normalization, date formatting, and statute standardization to prepare the data for downstream learning tasks.

Dataset Creation, Extraction, and Preprocessing

To make accurate, context-aware predictions, we fine-tuned the *Phi-4-mini* language model using our curated dataset. The model was trained to understand and generate structured outputs from full-text judgments by learning patterns across various legal factors and judicial reasoning. We adopted a parameter-efficient fine-tuning approach (like LoRA) to minimize computational overhead while maintaining performance. During training, the model learned to predict outcomes (bail granted or denied), along with human-readable legal reasoning and conditional recommendations in case bail was granted. Rigorous evaluation on a validation set showed promising results, with the model not only predicting decisions with good accuracy but also generating coherent legal justifications that mirrored judicial logic.

Fine-Tuning

To quantify the model's confidence in its predictions, we first force the LLM to generate either 0 or 1 depending upon whether the LLM thinks the bail should be granted or denied, (1 for granted, 0 for denied). Then we measure the probability with which either 0 or 1 is generated and normalize it along with the probabilities of other tokens between 0 and 1. This score represents how sure the LLM is regarding its predictions.

How confident is the model?

We developed a seamless and intuitive chatbot interface to serve as the front end for users including lawyers, judges, and petitioners. The user begins by uploading a case document or answering guided questions. The backend system then processes the input through the fine-tuned LLM model, and displays the bail decision, legal reasoning, confidence score, and required conditions (if any) in a structured conversational format. The interface supports multilingual interaction, ensuring accessibility for a diverse user base. Additionally, we designed the interface with mobile-first principles and included options to export conversations as PDFs, supporting legal documentation and offline usage.

Interface / Front End

Moving forward, we envision several enhancements to broaden the impact of the Indian Bail Prediction System. We plan to incorporate *precedent analysis, allowing the system to draw from past landmark judgments to improve its reasoning. Expanding multilingual support (e.g., Hindi, Bengali, etc.), integrating OCR for image-based case documents, and speech-to-text interfaces will significantly increase accessibility. We're also exploring the inclusion of other legal domains such as land disputes and dowry cases. On the model side, integrating more powerful reasoning models and improving explainability remain a priority. To support long-term adoption, we plan to offer secure user authentication and session history management.

Future Scope and Improvements

IBPS introduces a large-scale dataset of 150,000+ Indian bail applications, distilled to JSON objects with 11 features. It employs a fine-tuned Phi-4 model to generate the responses for user's queries and a CNN model to generate a confidence score which signifies how sure is the system about what it is saying, achieving an accuracy of 85.3%. The study emphasizes on the use of LLMs specifically tuned on the relevant datasets to achieve state of the art result in bail judgement prediction and explanation.

Indian Bail Prediction System (IBPS): A Comprehensive Dataset and Model for Predicting Bail Outcomes in Indian Courts

IBPS

The 'Namo Namah' AI agent's dataset is built from rigorously collected information from books by major Indian publishers, covering all states and widely spoken languages like Hindi, Bengali, and Marathi. A significant challenge was the lack of digital versions for many critical texts, such as 'Vratotsav'. This was overcome using OCR technology and meticulous annotation to create a structured database. Other valuable sources include 'Pooja Parbaner Utsakatha', 'Hindu Fasts and Festivals', and 'Bhartiya Sanskritikosh', ensuring a comprehensive understanding of Indian festivals and rituals.

Our AI agent, uses an advanced Hybrid-RAG database for accurate, contextually relevant responses on festivals, rituals, and culture. Unlike other systems, it employs manually curated, highly efficient retrieval techniques that outperform conventional methods in precision and recall.Key innovations in Hybrid-RAG architecture lead to superior performance:Optimized Chunking Strategies: Sophisticated algorithms break text into semantically meaningful units, calibrated for cultural/religious texts to preserve critical information.Advanced Triplet Generation: Rich knowledge triplets create structured knowledge graphs, enabling precise connections and hierarchies beyond simple keyword matching, understanding relationships among festivals, deities, and rituals.Intelligent Retrieval Techniques: Proprietary methods integrate semantic understanding, contextual relevance, and query intent, identifying pertinent information even in extensive datasets (e.g., providing historical context, regional variations, and associated deities for a ritual).The synergy of chunking, triplet generation, and intelligent retrieval allows 'Namo Namah' to access, comprehend, and synthesize vast cultural information, delivering accurate, insightful, and contextually appropriate responses for festival and ritual queries.

Hybrid RAG

The Namo Namah Festival & Ritual Guide AI is an agentic assistant built on LlamaIndex orchestration and powered by Microsoft’s Phi-4 model. It acts autonomously to gather, verify, and present culturally sensitive festival knowledge by invoking specialized tools: search_festival_knowledge, search_historical_poem, search_festival_dates, search_youtube_videos, and a fallback search_web. A custom built Hybrid RAG supplies retrieval-augmented control logic that maps user intent to the appropriate tool, stitches evidence from multiple sources, and enforces policy rules such as language preference and strict provenance. Phi-4 provides fluent, context-aware generation that translates tool outputs into clear, respectful answers. The agent guarantees that festival dates, muhurta, and timings are presented in structured form and that cultural terms are transliterated to the user’s language. Default location is Kanpur when none is provided. Built-in validation routines re-check retrieved passages to avoid unsupported claims, and the system politely declines or redirects queries outside its tool-derived scope. 

Agentic Festival Companion

The Namo Namah Festival & Ritual Guide can be expanded by incorporating cross-cultural and multilingual datasets, enabling seamless interaction across diverse Indian languages and global contexts. This ensures inclusivity and wider cultural reach. Future work will focus on building efficient Retrieval-Augmented Generation (RAG) pipelines that optimize query understanding, retrieval accuracy, and response generation, reducing latency while handling large cultural knowledge bases. Integration of adaptive memory and fine-grained personalization can further enhance user experience. By combining scalable RAG-LLM architectures with enriched datasets, the system can evolve into a universal cultural companion bridging traditions, languages, and communities worldwide.

Namo Namo is a festival and ritual guide  multilingual AI Agent powered by Hybrid RAG with state of the art retrival techniques which can clear all your doubts about any festival that is celebrated in any nooks and corners of India.Our aim is to ignite a cultural renaissance by easy accessibility and enlightening experience of religious/cultural knowledge base. We intend to do this by providing comprehensive knowledge on daily rituals/practices (Nitya Karma), specific events like Kumbh, Ekadashi, Gudi Parva, and all festivals like Holi, Diwali,Navratri, Lohri, Ganesh Chaturthi, etc.We wish not only to guide on how to participate and perform these rituals and festivals, but also on the significance and meaning of every minute details of rituals/festivals.

Projects

LegalSeg

INLegalLlama

PredEx

Nonet

IBPS

Namo Namah