HALO: an ontology for representing and categorizing hallucinations in large language models

Navapat Nananukul; Mayank Kejriwal

doi:10.1117/12.3014048

6 June 2024 HALO: an ontology for representing and categorizing hallucinations in large language models

Navapat Nananukul, Mayank Kejriwal

Proceedings Volume 13058, Disruptive Technologies in Information Sciences VIII; 130580B (2024) https://doi.org/10.1117/12.3014048
Event: SPIE Defense + Commercial Sensing, 2024, National Harbor, Maryland, United States

Abstract

Recent progress in generative AI, including Large Language Models (LLMs) like ChatGPT, has opened up significant opportunities in fields ranging from natural language processing to knowledge discovery and data mining. However, there is also a growing awareness that the models can be prone to problems such as making information up or ‘hallucinations’, and faulty reasoning on seemingly simple problems. Because of the popularity of models like ChatGPT, both academic scholars and citizen scientists have documented hallucinations of several different types and severity. Despite this body of work, a formal model for describing and representing these hallucinations (with relevant meta-data) at a fine-grained level, is still lacking. In this paper, we address this gap by presenting the Hallucination Ontology or HALO, a formal, extensible ontology written in OWL that currently offers support for six different types of hallucinations known to arise in LLMs, along with support for provenance and experimental metadata. We also collect and publish a dataset containing hallucinations that we inductively gathered across multiple independent Web sources and show that HALO can be successfully used to model this dataset and answer competency questions.

Conference Presentation

(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Navapat Nananukul and Mayank Kejriwal "HALO: an ontology for representing and categorizing hallucinations in large language models", Proc. SPIE 13058, Disruptive Technologies in Information Sciences VIII, 130580B (6 June 2024); https://doi.org/10.1117/12.3014048

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available