AI Cultural Heritage Docent Web Prototype
Graph-RAG · Cultural Heritage · Semantic Knowledge Graphs · Adaptive Interpretation
This research develops an AI cultural heritage docent web prototype that combines a semantic knowledge graph with generative AI to support responsible, visitor-adaptive interpretation. It explores how explanations can respond to a visitor's knowledge and language while remaining grounded in documented cultural heritage information and institutional guidelines.
Research team: Song-yi Jung · Iro Lim · Baro Kim
The Academy of Korean Studies
Research Context
Physical heritage signage offers limited space and generally presents the same explanation to every visitor. General-purpose language models can offer more flexible responses, but may introduce unsupported historical claims. This project addresses both challenges by connecting generation to structured domain knowledge and explicit principles for heritage interpretation.
The case study is the Hanging Painting of Janggoksa Temple (Maitreya Buddha, 1673). Its rich iconography and hwagi (畫記, painting inscription) connect figures, monk-painters, donations, materials, and production activities, making it a useful case for modeling both terminology and historical relationships.
Knowledge Modeling & Narrative Guidelines
The research organizes four kinds of source material—basic artwork information, inscription records, person records, and a thesaurus—into RDF/OWL semantic data. SKOS structures the iconographic vocabulary, while CIDOC CRM provides modeling terms for relationships among people, works, places, and events.
This semantic representation is converted into a Neo4j property graph for retrieval. RDF/OWL supports knowledge modeling and scholarly sharing, while Neo4j supports the application's queries. Multilingual names and alternative terms retain source-identifying prefixes so that their origins can be traced.
Alongside the factual data, Korea Heritage Service signage guidelines are expressed as constraints in the system prompt. This layer guides narrative structure and terminology, with the aim of reducing unsupported statements and maintaining appropriate interpretive language.
System Architecture
The backend routes questions to predefined Cypher queries that retrieve relevant graph records. LangChain combines those results with institutional guidelines and the visitor's selected context in a prompt for Google Gemini. The generated explanation is intended to reflect both the retrieved evidence and the visitor's level of familiarity.
The web prototype uses a Next.js frontend and a FastAPI backend. Its adaptive interface supports audience and language selection, allowing the same source material to inform explanations for children, general visitors, and specialists.
Two Modes of Interaction
Mode 1: Text-driven Description Generation
The prototype provides a button-based interface for selecting question types and receiving explanations grounded in retrieved data. Audience and language preferences guide the explanation's wording and difficulty.
Mode 2: Image-driven Interactive Description
The proposed image-based mode uses SVG hotspots on significant iconographic elements. Selecting a detail is designed to trigger retrieval of its associated graph data and generate a contextual explanation. This connects visual exploration with structured knowledge and source references.
Research Status & Next Steps
The project remains in development. User studies and quantitative evaluation are needed to assess interpretive accuracy, usability, and the effectiveness of its grounding and guideline mechanisms. Future work considers extending the approach to other heritage objects and interactive exhibition settings, including AR and 3D environments.
Project Presentation
Open project presentation (PDF)