I'm building a legal AI that checks its own answers, starting with a benchmark of how current models fail.
My final-year project, LegalReliBench, benchmarks how LLMs fail on legal documents: 457 questions across five legal genres, annotated at the claim level. I am now building a self-verifying agent that checks its own claims against the contract, corrects them, and defers risky cases to a human lawyer. After graduating I plan to pursue graduate study and become a university lecturer and researcher.
Final-year design project
Status: benchmark and baseline audit complete · verification modules in development · agent and interface planned · paper in preparation
When a model answers from a long contract, a wrong claim can depend on text several pages away. FYDP 262-043 asks whether an agent can notice that kind of error and correct it before answering.
Phase 1 is complete: LegalReliBench, 457 questions across five legal genres (CUAD, ContractNLI, LegalBench, PrivacyQA, MAUD), evaluated in 25 conditions on three open models. Phase 2, in progress, builds the verification modules. I am supervised by Tahmid Mosaddeque and work in a team of five. The paper is in preparation, targeting ACL/NAACL 2027.
Evaluation protocol, Schema v2.0
One slice of the benchmark: MAUD · 99 questions per model · 297 evaluations · condition: Gold Evidence
Benchmark-wide findings (457 questions, 25 conditions)
An answer is acceptable only if it meets all four
One failure recurred across models: the universal temporal inversion fallacy, coded REA-02.
Bars show baseline models only. The self-verifying agent is still in development, so no agent results are reported yet.
Illustrative example · Hallucination vs. Self-Correction
Publications
Related work: Evaluates against legal hallucination benchmarks like Large Legal Fictions (Dahl et al., 2024) and Hallucination-Free? (Magesh et al., 2025), while extending atomic claim-decomposition frameworks including FActScore (Min et al., 2023) and SelfCheckGPT (Manakul et al., 2023) to high-stakes legal QA.
Engineering & Research
Verification modules (NLI claim checker, citation checker, grey-area classifier, confidence calibration) feeding a self-correction loop. Planned: a dual-pane legal workspace for lawyers.
A 457-question reliability benchmark across five legal genres, with double-blind claim-level annotation and third-party adjudication. Question types include hallucination traps and multi-clause reasoning.
A powerful tool to convert C code into MIPS assembly language, supporting loops, arrays, functions, and complex expressions with live validation.
A smart schedule builder that lets you upload course offerings (.xlsx), select sections with automatic conflict detection, preview your weekly timetable live, and export the final schedule as CSV or PDF.
Automatically generates professional assignment cover pages based on user input. Instant print-ready A4 formatting.
Built for a retail shoe store. Replaces manual record-keeping with a centralized platform: product lifecycle tracking, per-size inventory with auto audit logging, real-time stock-validated sales commits, Chart.js reporting, and self-bootstrapping database.
Full-stack movie review web app (CineVault) with star-rated reviews, community trending lists, user profiles with review history, crowdsourced entries, persistent theme toggle, and an MVC architecture with parameterized PDO queries.
Full stack job recruitment portal enabling employers to post opportunities, manage candidate applications, and provide job seekers with searchable career listings.
Desktop-based recruitment system designed with graphical desktop interface, applicant tracking, hiring workflow stages, and relational MySQL database persistence.
Robot picking and pathfinding simulation modeling automated item retrieval, spatial navigation, and trajectory efficiency in logistics warehousing.
Console-based management system managing retail store accounts, tenant inventories, billing calculations, and record tracking using core algorithms.
Smart wearable cooling jacket pairing a phase-change material (PCM) with a Peltier thermoelectric element for energy-efficient personal thermal comfort.
Teaching & Instruction
▶ YouTube Playlists
📁 My Course Notes
Notes for other courses are also available.
United International University
The role takes about five hours a week: two and a half in class and two and a half counselling.
Academic background
United International University, Dhaka
Current CGPA 3.91 / 4.00
Dhaka Imperial College · Science
GPA 5.00 / 5.00
Monpura School and College · Science
GPA 5.00 / 5.00
Toolkit and recognition
Contact
Open to: MS/PhD positions in NLP, VLM, LLMs, research assistantships focused on LLM reliability, and teaching assistantships in CSE.
sayeemmahmood2@gmail.com