Ehsan Kamalloo

prof_pic.jpg

My name is Ehsan (pronounced /eh☀️/ or /ih☀️/, or even /e☀️/). I am a Research Scientist in the Conversational Assistant team at ServiceNow Research.

I am broadly interested in building intelligent autonomous systems that are capable of complex reasoning (“TapeAgents”). My research focused on robust evaluation (“LLM-as-a-judge for QA” at ACL’23 and SIGIR’24, “NoMIRACL” at EMNLP’24, “Touché-2020” at SIGIR’24 and “MIRACL” at TACL’23) and data-efficient training methods for distilling a small model from large models (“kNN-KD” at ACL’21 and ACL’22) and training language models.

Prior to this, I was a Post-doctoral Fellow at the David R. Cheriton School of Computer Science, University of Waterloo, hosted by Jimmy Lin. I completed my PhD at the University of Alberta, advised by Davood Rafiei. I was also fortunate enough to collaborate with Charles Clarke and Siva Reddy. I also interned as a researcher at Huawei Noah’s Ark Lab in Montreal and worked with Mehdi Rezagholizadeh. Before PhD, I was working as a software engineer.

news

Jan 2025 The REALM Call for Paper is out. Submissions are due March 31, 2025.
Nov 2024 Organizing REALM: First Workshop for Research on Agent Language Models at ACL 2025 in Vienna :violin:.
Oct 2024 TapeAgents framework icon for LLM agents development released: .
Sep 2024 :tada: NoMIRACL got accepted to EMNLP 2024 Findings.
Jul 2024 :tada: Our demo on QA evaluation won “Best Short Paper Honorable Mention” at SIGIR 2024: .
May 2024 :mega: New paper: “LLMs Can Patch Up Missing Relevance Judgments in Evaluation” [arXiv]