Retrieval-augmented generation (RAG)

MEMERAG: A multilingual end-to-end meta-evaluation benchmark for retrieval augmented generation

Andrea Cruz, Jayasimha Talur, Bruno Charron, Dong Liu, Saab Mansour, Marcello Federico

ACL 2025

2025

Automatic evaluation of retrieval augmented generation (RAG) systems relies on fine grained dimensions like faithfulness and relevance, as judged by expert human annotators. Meta-evaluation benchmarks support the development of automatic evaluators that correlate well with human judgement. However, existing benchmarks predominantly focus on English or use translated data, which fails to capture cultural

Conversational AI

Measuring the fairness gap between retrieval and generation in RAG systems using a cognitive complexity framework

Sandeep Avula, Chia-Jung Lee, Rongting Zhang, Vanessa Murdock

SIGIR 2025

2025

In this paper, we investigate the problem of quantifying fairness in Retrieval-Augmented Generation (RAG) systems, particularly for complex cognitive tasks that go beyond factual question-answering. While RAG systems have demonstrated effectiveness in information extraction tasks, their fairness implications for cognitively complex tasks - including ideation, content creation, and analytical reasoning —

Search and information retrieval

CatalogRAG: Retrieval-guided LLM prediction for multilingual e-commerce product attributes

Bryan Zhang, Suleiman Khan, Stephan Walter

KDD 2025 Workshop on LLM4ECommerce

2025

E-commerce stores increasingly use Large Language Models (LLMs) to enhance catalog data quality through automated regeneration. A critical challenge is accurately predicting missing structured attribute values across multilingual product catalogs, where LLM performance varies significantly by language. While existing approaches leverage general knowledge through prompt engineering and external retrieval

Machine learning

LentEx: Generalizable latent entity extraction via synthetic data and instruction-tuned LLMs

Umesh Bodhwani, Yuan Ling, Cibi Chakravarthy Senthilkumar, Shujing Dong, Yarong Feng, Hongfei Li, Ayush Goyal

IJCNN 2025

2025

Latent entity extraction (LEE) tackles the challenge of identifying implicit, contextually inferred entities within free text—an area where traditional entity extraction methods fall short. In this paper, we introduce LentEx, a novel framework for latent entity extraction that leverages synthetic data generation and instruction fine-tuning to optimize smaller, efficient large language models (LLMs). Latent

Conversational AI

From unstructured communication to intelligent RAG: Multi-agent automation for supply chain knowledge bases

Yao Zhang, Zaixi Shang, Silpan Patel, Mikel Zuniga

KDD 2025 Workshop on AI for Supply Chain

2025

Supply chain operations generate vast amounts of operational data; however, critical knowledge—such as system usage practices, troubleshooting workflows, and resolution techniques—often remains buried within unstructured communications like support tickets, emails, and chat logs. While Retrieval-Augmented Generation (RAG) systems aim to leverage such communications as a knowledge base, their effectiveness

Information and knowledge management

AutoChunker: Structured text chunking and its evaluation

Arihant Jain, Purav Aggarwal, Anoop S V K K Saladi

ACL 2025

2025

Text chunking is fundamental to modern retrieval-augmented systems, yet existing methods often struggle with maintaining semantic coherence, both within and across chunks, while dealing with document structure and noise. We present AutoChunker, a bottom-up approach for text chunking that combines document structure awareness with noise elimination. AutoChunker leverages language models to identify and segregate

Conversational AI

CiteFix: Enhancing RAG accuracy through post-processing citation correction

Harsh Maheshwari, Srikanth Tenneti, Alwarappan Nakkiran

ACL 2025

2025

Retrieval Augmented Generation (RAG) has emerged as a powerful application of Large Language Models (LLMs), revolutionizing information search and consumption. RAG systems combine traditional search capabilities with LLMs to generate comprehensive answers to user queries, ideally with accurate citations. However, in our experience of developing a RAG product, LLMs often struggle with source attribution,

Machine learning

HybGrag: Hybrid retrieval-augmented generation on textual and relational knowledge bases

Jeremy Lee, Qi Zhu, Costas Mavromatis, Zhen Han, Soji Adeshina, Vassilis N. Ioannidis, Huzefa Rangwala, Christos Faloutsos

ACL 2025

2025

Given a semi-structured knowledge base (SKB), where text documents are interconnected by relations, how can we effectively retrieve relevant information to answer user questions? Retrieval-Augmented Generation (RAG) retrieves documents to assist large language models (LLMs) in question answering; while Graph RAG (GRAG) uses structured knowledge bases as its knowledge source. However, many questions require

Machine learning

Mind the semantic gap: Semantic efficiency in human computer interfaces

James Horsley

Frontiers in Artificial Intelligence

2025

As we become increasingly dependent on technology in our daily lives, the usability of HCIs is a key driver of individual empowerment for us all. A primary focus of AI systems has been to make HCIs easier to use by identifying what users need and agentively taking over some of the cognitive work users would have otherwise performed, as such, they are becoming our delegates. To become effective and reliable

Conversational AI

SimRAG: Self-improving retrieval-augmented generation for adapting large language models to specialized domains

Ran Xu, Hui Liu, Sreyashi Nag, Zhenwei Dai, Yaochen Xie, Xianfeng Tang, Chen Luo, Laurence (Yang) Li, Joyce C. Ho, Carl Yang, Qi He

NAACL 2025

2025

Retrieval-augmented generation (RAG) enhances the question answering (QA) abilities of large language models (LLMs) by integrating external knowledge. However, adapting general-purpose RAG systems to specialized fields such as science and medicine poses unique challenges due to distribution shifts and limited access to domain-specific data. To tackle this, we propose SimRAG, a self-training approach that

Conversational AI

Retrieval-augmented generation (RAG)

Work with us