LLM Hallucination Benchmark Dataset

Name: LLM Hallucination Benchmark Dataset
Creator: GTS

Datasets

File

LLM Hallucination Benchmark Dataset

Use Case

LLM Hallucination Detection, AI Evaluation, Model Benchmarking, AI Safety Research, RAG Evaluation

Description

The LLM Hallucination Benchmark Dataset contains annotated AI responses with hallucination labels, categories, severity levels, and verified corrections. It can help researchers and developers evaluate LLM reliability, compare models, and study methods for reducing hallucinations.

Description:

Large Language Models (LLMs) have transformed the way organizations interact with information, automate workflows, and deliver AI-powered services. However, one of the biggest challenges facing modern AI systems is hallucination—the generation of inaccurate, misleading, or unverifiable information that appears convincing but lacks factual support.

Download Dataset

The LLM Hallucination Benchmark Dataset is designed to help researchers, developers, and AI practitioners evaluate and understand hallucination behavior across leading language models. By providing carefully annotated prompts, responses, hallucination labels, severity assessments, and verified corrections, this dataset offers a valuable resource for building safer and more reliable AI systems.

Dataset Overview

The LLM Hallucination Benchmark Dataset is a specialized benchmark dataset created to analyze the reliability and factual accuracy of large language models.

The dataset contains 200 annotated LLM responses collected between September 2024 and April 2025. Each record includes a prompt, model-generated response, hallucination annotations, and supporting evaluation data.

Key Features of the Dataset

Annotated Hallucination Records

Each entry contains:

User prompt
LLM-generated response
Hallucination label
Hallucination category

Leading AI Models Included

The dataset covers responses generated by several advanced language models, including:

GPT-4o
Claude 3.5 Sonnet
Gemini 1.5 Pro
Llama 3.1 70B

Multiple Knowledge Domains

The dataset spans diverse subject areas:

Medicine
Technology
Science
Finance

Multilingual Coverage

Responses are available across multiple languages, including:

English
Spanish
French
Hindi

Hallucination Categories

The dataset includes seven common hallucination types:

Factual Contradiction

Responses that directly conflict with verified facts.

Overclaim

Statements that exaggerate evidence or certainty.

Entity Error

Incorrect people, organizations, locations, or entities.

Relation Error

Incorrect relationships between entities or concepts.

Dataset Specifications

Feature	Details
Dataset Name	LLM Hallucination Benchmark Dataset
Records	200 Annotated Responses
Models Covered	5 Leading LLMs
Knowledge Domains	8
Languages	7
Hallucination Types	7
Annotation Sources	Human Experts, Crowd Annotators, AI Judges
Use Cases	Hallucination Detection, AI Evaluation, AI Safety
Data Type	Text Dataset

Benefits for AI Research and Development

This dataset provides several advantages for AI teams:

Structured hallucination annotations
Cross-model evaluation capabilities
Multilingual benchmarking support
Domain-specific analysis opportunities
AI safety and trustworthiness research

Applications of the LLM Hallucination Benchmark Dataset

Hallucination Detection Systems

Train models to automatically identify hallucinated responses generated by AI systems.

AI Safety Research

Analyze risks associated with factual inaccuracies in large language models.

Model Comparison and Benchmarking

Compare hallucination rates and performance across multiple LLM architectures.

Retrieval-Augmented Generation (RAG) Evaluation

Measure how retrieval systems impact factual accuracy and reduce hallucinations.

Conclusion

The LLM Hallucination Benchmark Dataset provides a valuable foundation for studying one of the most important challenges in modern artificial intelligence. With annotated responses from leading language models, multilingual coverage, detailed hallucination classifications, and evaluation of mitigation strategies, the dataset supports the development of safer, more accurate, and more trustworthy AI systems.

This dataset is sourced from Kaggle.

Contact Us

FAQ

Question 1. What is the LLM Hallucination Benchmark Dataset?

It is an annotated dataset designed to evaluate hallucinations in large language model responses across multiple domains and languages.

Question 2. Which AI models are included?

The dataset includes responses from GPT-4o, Claude 3.5 Sonnet, Gemini 1.5 Pro, Llama 3.1 70B, and Mistral Large.

Question 3.How many records are included?

The dataset contains 200 annotated LLM responses.

Question 4. Is the dataset suitable for AI safety research?

Yes. The dataset is specifically designed to support AI reliability, trustworthiness, and hallucination mitigation research.

Question 5. What industries can benefit from this dataset?

Healthcare, legal technology, finance, education, government, research institutions, and AI product development teams can all benefit from this dataset.

Let's Discuss your Data collection Requirement With Us

To get a detailed estimation of requirements please reach us.

LLM Hallucination Benchmark Dataset