VR PLATFORM | LOCAL TOURS
Type Digital Heritage, VR Experience Team Zihao Zhang, Daoxin Chen, Peter Zhang, Ying Zhang, Kurt Qian Time March 2025 - June 2025
The VR platform serves as a bridge between local tour guides and global tourists, aiming to make authentic, locally rooted stories about cultural heritage and historic districts accessible to a wider audience. It empowers local guides to create and publish virtual tours, while allowing users around the world to explore these experiences in immersive virtual environments.
As a proof of concept, we used Manhattan’s Chinatown in New York City to demonstrate the platform. We designed the user experience for both local content creators and global audiences, and developed a functional prototype using Unity and VR headsets. The experience also integrates an AI-powered chatbot that users can interact with by taking photos, enabling dynamic, image-based information retrieval.
IDEATION
The Sketch of Ideation
The cultural tourism platform features three core components: support for professional tour guides to upload their own site interpretations, an AI companion that provides personalized information, and the ability for users to freely explore the architectural and cultural context of heritage sites.
Manhattan’s Chinatown in New York City was selected as the pilot site for designing and developing the experience prototype. In collaboration with local scholars, we identified seven key tourism spots that represent the rich narratives of Chinatown, including Kimlau Square, 5–7 Doyers Street, 21 Pell Street, 37 Mott Street, and Columbus Park.
The Map of Virtual Tour in Manhattan's Chinatown in NYC
STORYTELLING
The Brainstorm of Spatial Storytelling about Chinatown.
In the initial phase, we conducted historical research by consulting online archives to gather stories about the selected heritage sites. These stories were then embedded within 3D Gaussian Splatting models to enable spatial storytelling. The narratives were presented through a mix of media—including text, audio, and images—offering a rich, multi-sensory experience. By combining spatial and multimedia storytelling, the project transforms historical research and expert interpretation into engaging, interactive experiences for the audience.
The Chart of Spatial Storytelling in the Seven Places.
3D SCANNING
The selected locations were scanned using Luma AI, then further processed and imported into Unity to virtually reconstruct the site context.
The Gaussin Splatting of Kimlaw Sqaure.
The Gaussin Splatting of the intersection of Doyer St. and Pell St.
The Gaussin Splatting of Columbus Park.
The Gaussin Splatting of 37 Mott St.
AI Chatbot
To enable personalized experiences, the application integrates the Google Gemini API. Users can interact with an AI-powered chatbot by simply clicking on it, which automatically captures the scene in front of them and sends the image to the AI for analysis. Based on the visual input, the AI provides contextual information to enhance user engagement. An avatar is also designed to represent the AI chatbot, visually conveying its status and activity in real time.
The Animation of AI Chatbot
PGC USERFLOW
The userflow of PGC of the creators' platform
The features of PGC of the creators' platform
The user experience for professional creators is designed to empower local communities and tour guides to easily create virtual heritage experiences. The platform supports creators in generating Gaussian Splatting models of heritage sites and enables them to upload and integrate stories through images, text, and audio. This streamlined workflow makes it accessible for non-technical users to contribute meaningful cultural content.
USER INTERFACE
The Interface of PGC of Mobile Phone
The user experience for professional creators is designed to help local communities and tour guides easily create virtual heritage experiences. The platform supports intuitive editing tools that work seamlessly on both mobile devices and laptops, enabling users to build and customize experiences anytime, anywhere.
The Editing Interface of PGC on the Laptop
The Airtag Editing
The Airtag
The Airtag Editing
The Preview of the Experience
IMMERSIVE EXPERIENCE
Demo Video
Map Interaction
The Airtag with Stories
The AI Companion
EXHIBITION
The Project presentation in AWE
The prototype of the virtual heritage experience was showcased at AWE USA 2025 in Long Beach and during Asian & Pacific American Heritage Month (AAPI) at the Memor Museum. Approximately 30 attendees—including XR practitioners, designers, entrepreneurs, investors, and enthusiasts of heritage and cultural tourism—interacted with the demo and provided valuable feedback and suggestions for improvement.
The Project Presentation in AAPI
USER FEEDBACK
The User Testing
Based on feedback from participants, several insights and recommendations were identified to refine the user experience and support both users and content creators in engaging with cultural heritage through our platform.
Interaction Innovation
The current interaction method—primarily clicking on airtags—was perceived as unengaging. Participants suggested making better use of the interactive capabilities of XR technology to create more intuitive and immersive user interactions.
Information Presentation
Information displayed through flat panels was considered visually unappealing and lacked engagement. Enhancing the way content is presented—through spatial storytelling, multimedia elements, or adaptive visuals—could significantly improve user experience.
Usability of Airtags
The positioning and scaling of airtags made them difficult to interact with, limiting users’ ability to access embedded information. Optimizing their placement and usability is key to smoother navigation and interaction.
Chatbot Behavior
While the inclusion of a chatbot adds value, its movement and floating behavior were found to interfere with users’ focus on the 3D models and narrative content. A more subtle or context-aware design is recommended.
Model Quality
Feedback on the 3D models was mixed. Some users found the visuals unclear and lacking in detail, while others felt the current models effectively conveyed a sense of presence in Chinatown. This indicates the need for adjustable visual fidelity based on user preferences or device capabilities.
Scalability of the Experience
Since each heritage site's story is deeply context-specific, replicating this experience in other locations presents a challenge. Developing a flexible framework for content adaptation is essential to make the heritage platform scalable and reusable across diverse sites.
Support for Content Creators
To contribute to the platform, content creators need to acquire or create 3D models of heritage sites, which may require technical knowledge or training. Furthermore, establishing streamlined workflows and clear onboarding processes will be critical to connect and empower creators to use the platform effectively.
The User Testing
The User Testing
CONTRIBUTION
Zihao Zhang: Ideation, Historic Research, 3D Scanning, Presentation
Daoxin Chen: Ideation, Storytelling, Content Creators' Experience Design, 3D Scanning
Peter Zhang: VR Development, 3D Scanning, Worldbuilding
Ying Zhang: Ideation, Content Creators' Experience Design, VR Development
Kurt Qian: AI Chatbot Animation
You may also like
Beyond Facade | VR for Low Library
2024
