Preprint
Article

This version is not peer-reviewed.

Reshaping Museum Experiences with AI: The ReInHerit Toolkit

A peer-reviewed article of this preprint also exists.

Submitted:

28 May 2025

Posted:

04 June 2025

You are already at the latest version

Abstract
This paper presents the ReInHerit Toolkit, a collection of open-source interactive applications developed as part of the H2020 ReInHerit project. Informed by extensive surveys and focus groups with cultural professionals across Europe, the toolkit addresses key needs in the heritage sector by leveraging computer vision and artificial intelligence to enrich museum experiences through engaging, personalized interactions that enhance visitor learning. Designed to bridge the technology gap between larger institutions and smaller organizations, the ReInHerit Toolkit also promotes a sustainable, peoplecentered approach to digital innovation, supported by shared resources, training, and collaborative development opportunities accessible through the project’s Digital Hub.
Keywords: 
;  ;  ;  ;  ;  ;  ;  

1. Introduction

The ReInHerit Toolkit, developed within the Horizion 2020 ReInHerit Project, is a set of open source applications based on Computer Vision (CV) and Artificial Intelligence (AI) that provide diverse types of experiences exploiting interactions and tasks of interest for museums and cultural institutions with the goal to engage visitors with cultural content and improve learning. The ReInHerit project aims to propose an innovative model of sustainable heritage management, through a dynamic and collaborative network comprised of cultural heritage professionals, innovation and cultural heritage solution tech experts, researchers, creatives, museums, and managers of heritage sites.
The conceptual model pioneered by the project focuses on creating a digital ecosystem dedicated to cultural heritage. This ecosystem fosters cooperation and knowledge sharing between stakeholders, offering a shared, experimental environment. Central to this model is a "Digital Hub"1, an interactive platform that collects and disseminates all project-related content, including tools, learning materials, and webinar sessions aimed at supporting heritage professionals. Within this space, users can freely access and download the project’s Toolkit along with the relevant materials on digital applications, including instructional resources, reuse strategies, and curated webinars. The "Toolkit"2 and all associated documentation for the technological applications are available for free and can be downloaded from the Digital Hub. This includes a number of resources such as webinars, reuse guidelines, training material 3, a collection of reference components and free source codes hosted on ReInHerit’s GitHub page 4.
Initial studies conducted within ReInHerit provided an overview of the research in the field of cultural heritage. These investigation mapped current models, institutional needs, visitor expectations, and contemporary trends influencing museums and cultural sites. This pivotal investigation served as a key factor in shaping the project’s approach to technological innovation and digital transformation. It advised on the integration of new technologies designed to make cultural heritage more accessible, interactive and sustainable, while responding to current demands and developments in the field. It also provided valuable insight into the motivations behind the types of applications to be developed for the Toolkit, forming a foundational conceptual framework (Figure 1).
In the following sections, we will outline the motivations and conceptual framework that prompted the development of the tools, focusing in particular on the results of the research conducted with museum visitors and professionals, which provided relevant insights on needs and preferences. Based on these outcomes, the Toolkit Strategy was designed, with particular emphasis on both the opportunities and the challenges linked to the use of Artificial Intelligence within the museum context. The final part of the paper describes how an interdisciplinary and collaborative methodology guided the creation of the technological tools.

1.1. Results from ReInHerit Research on Digital Tools

ReInherit study carried out focus groups with professionals working in museums and cultural sites, as well as with experts in digital technologies. Additionally, online surveys were launched for both professionals and visitors. The objective was to identify the needs of cultural professionals and institutions, while also aligning them to the preferences of users with diverse backgrounds. The research aimed to explore the digital capabilities of heritage organizations in Europe, identifying best practices for CH management and the most useful types of ICT tools.5 This analysis provided insightful indications as to whether organizations have made significant progress towards digital transformation. It also explored the availability of human resources in the heritage sector, offering a clear picture of whether organizations have the capacity to implement and sustain digital innovation [1]. Focus group interviews were conducted with a total of 34 professionals working in the cultural heritage management sector from 12 countries in Europe, mapping the current state of the practices in the management of the Cultural Heritage sector. Additional focus groups on the current ICT tools in cultural heritage were conducted online in March 2022 with museum/heritage site professionals, academic researchers, officers from public authorities/NGOs and ICT professionals, from 10 European countries (Austria, Croatia, Cyprus, Finland, Greece, Italy, Spain, Sweden, Switzerland, the Netherlands). Online Survey on October-December 2021 collected 1746 responses by visitors and 506 responses from institutions by cultural heritage professionals of 37 European countries. Ethical approval for the survey and focus groups was obtained, and participants provided informed consent to be included 6. The results are summarized below, with a focus on key findings.

1.2. Demographics

The demographics of the survey participants provide valuable insights into the audience profile engaged in the ReInHerit project. Most of the respondents were in the 30-64 age group, 76.42%. Younger users, aged between 18 and 29, accounted for 19.32%, while the oldest group, those aged 65 and over, accounted for 4.26%. Regarding education, the most common level of qualification was a Master’s degree, held by 48.74% of respondents. This was followed by participants with a Bachelor’s degree (23.58%) and those with a PhD (13.48%). Only a small fraction (0.61%) reported having completed only primary education. In terms of employment, a significant majority were employees, representing 60.61% of the respondents. Additionally, 14.56% were self-employed, while 4.83% were unemployed. Participants hailed from a diverse array of European countries. The largest groups were from Spain (23.98%), Italy (20.84%), Austria (14.32%), Finland (11.01%), Greece (10.71%), and Cyprus (6.52%). Other represented nations included Belgium, Bulgaria, Croatia, France, Germany, Romania, and several others. A small number of participants were from countries like Andorra (0.17%), Denmark (0.06%), Kosovo (0.06%), and Slovenia (0.06%), highlighting the wide geographic reach of the survey. Lastly, the types of organizations represented in the survey were varied. Public museums and cultural heritage sites comprised the largest group at 36.10%, followed by universities and research institutes (16.39%) and private museums or cultural heritage sites (14.32%). Smaller proportions included creative industries (9.75%), NGOs (9.13%), and public authorities (8.09%). These demographic insights lay the foundation for understanding the preferences and needs of museum visitors and cultural heritage professionals, crucial for tailoring digital tools effectively. The key insights derived from the analysis of digital tool usage by both museum visitors and professionals provided essential input for shaping the Toolkit’s design and development.

1.3. Visitor Preferences

These insights reflect visitor preferences, challenges, and interests related to digital engagement during museum visits.
Reasons for not using mobile applications: the primary reason for not using mobile applications during museum visits was that respondents found them distracting. Over 50% of respondents considered mobile applications either distracting or uninteresting. This tendency became more prominent with age, as older users (65+) were particularly likely to find mobile applications distracting. Younger users (18-29), on the other hand, expressed concerns about insufficient storage space on their devices. This trend is consistent with existing research, showing that younger visitors are reluctant to download museum apps due to limited smartphone memory. This pattern was also observed across different educational levels (Figure 2).
Digital tools for improving the visitor experience: no single digital tool was considered to be the most or least useful; preferences varied by age group. Younger and middle-aged visitors preferred interactive technologies such as multi-touch tables and immersive environments, while older visitors (65+) preferred classic audio guides. This reflects the tendency of older users to find advanced technologies more difficult to navigate (Figure 3). In general, all visitors preferred to use their smartphones or personal devices (Figure 4).
Interest in interaction with exhibits: across all age groups, a significant percentage of visitors expressed interest in direct interaction with exhibits. Visitors of all ages found this to be very interesting, with no significant differences between age groups (Figure 5).
Digital games: a significant portion of older users (65+) found digital games not interesting, while younger and middle-aged visitors showed more interest, mainly motivated by curiosity, entertainment, improving knowledge and social connections. (Figure 6).
Mobile applications: the interest varied by age. Younger respondents (18-29) found mobile apps "interesting" or "very interesting." However, older users (65+) tended to remain neutral, neither finding mobile apps particularly interesting nor boring (Figure 7). Younger visitors are more likely to engage with mobile apps depending on the perceived benefits and features of the apps, mainly web-apps. This aligns with existing research on teenagers’ behaviour in museums, which shows that younger visitors are often reluctant to download museum-specific apps, as they prefer not to use up memory on their smartphones, highlighting the potential to engage museum visitors with AI-driven, more seamless and personalized experiences instead [2].
Digital tools thus emerge as crucial mediators of the museum experience, especially for younger visitors, fundamentally shaping how they perceive, learn from, and emotionally connect with cultural heritage, as evidenced by recent socio-material studies on youth-technology interaction [3].

1.4. Heritage Professionals Needs

Heritage professionals were asked about the technological services and systems available in the organizations where they work (Figure 8).
Responses were divided in two main groups 1) Emerging and Advanced ICT solutions, including Artificial Intelligence applications, chatbots, games and/or gamification, digital storytelling tools, digital tools for exhibition planning, and 2) Conventional and Standard digital technologies, such as video and audio recording equipment, web applications, mobile applications, online exhibitions, digitalization systems, analytics and feedback tools, social media management tools, ticketing systems, e-shop. The following results summarize the major findings regarding the technological services and systems currently available in cultural heritage organizations. What follows is a synthesis of the most relevant outcomes concerning the digital systems currently employed by cultural heritage institutions. These findings highlight the current digital landscape of the industry, underscore the disparity between conventional and cutting-edge technologies, and identify major obstacles organizations face in integrating innovative digital tools.
  • 67.33% of museums and cultural heritage sites rely on standard ICT tools, while only 33% use innovative ICT tools. This highlights a need for integrating more innovative tools into the sector to improve visitor engagement and experience.
  • The analysis revealed that smaller organizations are more likely to rely on standard ICT tools and face greater challenges in adopting innovative solutions. This underscores the need for sharing digital platform that can support museums of all sizes, offering tailored solutions to their specific needs.
  • AI and gamification tools (e.g., chatbots, digital storytelling) were identified as important but rarely used. These tools can enhance visitor interaction and engagement, making them crucial for future development.
  • Human Resources: Most organizations do not employ dedicated professionals for technological implementation. Instead, they rely on third-party consultants or lack the resources to develop digital tools internally. This indicates a need for training and upskilling heritage professionals to become active agents in the digital transformation of cultural heritage institutions (Figure 9).
The results of the national survey highlighted the primary technological requirements in the field of cultural heritage, emphasizing both the integration of digital solutions and the availability of skilled human resources. While all organizations showed interest in adopting technologies like AI, Computer Vision, digital games, and immersive experiences, smaller institutions emphasized the need for more technical support to implement them effectively. Despite increasing digital adoption, smaller organizations struggle to integrate these technologies and require upskilling. The survey also revealed a heavy reliance on outsourcing for digital tasks, contributing to a knowledge gap. There is strong demand for technologies such as AI, gamification, and interactive solutions to enhance visitor engagement. Smaller organizations face challenges in adopting these technologies, highlighting the need for accessible, tailored solutions and staff upskilling to drive digital transformation. Tools for ”phygital” interaction —such as participative storytelling, gamification, and multisensory engagement—are seen as key to enrich the visitor experience, fostering emotional involvement before, during, and after the visit, and contributing to the design of authentic, meaningful experiences that enhance the sense of presence and connection with cultural heritage, in line with recent studies [4]. AI can also personalise content and create dynamic exhibitions that connect audiences at a distance, extending the experience beyond the visit through shared memories.
Focus groups discussions with professionals highlighted the need for tools that foster dialogue between professionals and visitors. Community-driven initiatives like hackathons and workshops ensure that tools remain user-centered and sustainable. Digital platforms can connect museums globally, promoting best practices and a people-centered approach to cultural heritage. These insights align with ReInHerit’s goal of enhancing engagement through interactive digital tools. Digital technologies can expand cultural offerings and create personalized experiences[5]. The discussions also emphasized the importance of collaboration among museum experts, developers, and visitors. However, several challenges remain, including the high costs of developing and maintaining customized apps, which quickly become outdated due to rapid technological advancements, as well as the technology gap between large and smaller institutions. ReInHerit’s Toolkit aims to address these issues by using AI and CV to create interactive, gamified experiences that deepen visitor engagement and encourage user-generated content.

2. Materials and Methods

This section explores the integration of AI and CV in enhancing visitor engagement in museums, highlighting the key insights from recent research and the development of the ReInHerit strategy 7. In the first part, we examine how AI technologies are revolutionizing museum operations and visitor experiences, including opportunities for personalization and interactive engagement.
In recent years, the concept of the ‘smart museum’ has emerged as an innovative model, where AI-based technologies — from facial recognition to real-time translation — animate otherwise static collections, creating an experience that connects people, education, culture, and art wang2004image. These capabilities set the conceptual groundwork for the ReInHerit strategy, which operationalizes such technologies through user-centered, playful, and emotionally engaging tools that connect on-site and digital experiences.

2.1. Insights on AI and Museums

Recent literature on museums underscores both the challenges and opportunities presented by AI-driven digital innovation. For example, the 2021 Museum Innovation Barometer revealed that fewer than 20% of museums worldwide had adopted AI tools for collections management, administration, education, or financial operations [6]. However, the COVID-19 pandemic acted as a catalyst for digital transformation, prompting institutions to accelerate digitization efforts and adopt new technologies while exploring alternative revenue streams [7]. CV and AI have emerged as some of the most transformative technologies in this evolving landscape [8], with applications that extend well beyond efficiency. In the broader creative industries, AI has already demonstrated its ability to increase productivity by automating processes and optimizing workflows [9]. In the museum context, these technologies offer the potential for more personalized and engaging visitor experiences. AI and CV can analyze visitor behavior and preferences, enabling museums to tailor narratives, content, and interfaces to individual users [10]. This level of personalization makes digital tools not only more relevant and appealing but also more effective in enhancing cultural experiences. Beyond user interaction, CV allows museums to derive insights from visual data that would be difficult or impossible to obtain through manual research alone [11]. When combined with AI, it enables curators to identify patterns, anomalies, and relationships within collections, helping to overcome limitations in time, staffing, and resources. As Villaespesa and Murphy note, AI and CV are increasingly essential for enriching collections and enhancing visitor engagement through more personalized and interactive experiences [12]. However, the rapid adoption of algorithmic decision making across sectors brings with it ethical and operational challenges, ranging from bias and error to job displacement and privacy concerns. These issues make it essential to develop critical, ethical and transparent frameworks for the use of AI, particularly in sensitive public domains such as cultural heritage [13]. In this regard, projects like the Museums + AI Network and its associated guide, AI: A Museum Planning Toolkit offers valuable models for sustainable and ethically grounded AI adoption, especially relevant for small and medium-sized museums [14].
Efforts toward AI standardization are also underway. Organizations like ISO are working to establish standards for the design and deployment of AI systems, although progress is challenged by the rapid evolution of the field and unresolved research questions [15]. At the policy level, the European Commission and European Parliamentary research have identified a range of opportunities for AI in cultural institutions, from improving archival practices and digital cataloging to enhancing audience engagement and visitor experience management [16]. Techniques such as sentiment analysis, attendance tracking, and forecasting can provide real-time insights to help optimize museum operations and planning. However, these benefits often remain out of reach for smaller institutions due to high costs, limited access to funding, and a general perception that AI investments are non-essential. Addressing this disparity, recent recommendations from the EU Committee of Ministers encourage the use of AI to foster emotional engagement and social interaction in cultural contexts [17]. These recommendations also promote collaboration and training to build digital capacity across institutions [18]. Importantly, the ethical deployment of AI in museums is based on principles such as transparency, data integrity, and scientific accuracy. These are essential for building public trust and ensuring that AI enhances rather than detracts from the interpretation of cultural heritage [19]. In this direction, practical frameworks like AI FAQ Artificial Intelligence by Orlandi et al. (2025) offer museums concrete guidance on topics such as copyright, data governance, and emerging tech regulations [20]. Finally, museums themselves can play a vital role in shaping the societal implications of AI. As public institutions dedicated to knowledge and reflection, they are ideal spaces to foster critical conversations about technology. This perspective was affirmed by the three strategic recommendations of NEMO, presented at the 2024 conference Innovation and Integrity: Museums Paving the Way in an AI-driven Society. The recommendations call for: 1) integrating museums into the development of AI regulatory frameworks; 2) investing in infrastructure, training, and data management; and 3) establishing a European AI Innovation Hub for cultural heritage to coordinate expertise and promote human-centered design.8

2.2. The ReInherit Toolkit Method

The ReInHerit strategy was shaped by extensive research aimed at identifying best practices for the use of digital tools in the context of digital transformation [21]. This strategy emphasizes digital interactivity as a fundamental component in creating user-centered exhibitions. Digital tools are not considered ends in themselves but are viewed as means to access content, engage audiences, and promote inclusive, participatory learning experiences. Adopting an interdisciplinary approach, the ReInHerit Toolkit has been presented to ICT experts at major international conferences and workshops, including ACM Multimedia 2022 in Lisbon, Portugal; ACM Multimedia 2023 in Ottawa, Canada; ACM Multimedia Systems 2024 in Bari, IT; IEEE/CVF Computer Vision and Pattern Recognition Conference (CVPR) 2022 New Orleans, USA; IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) 2024 Waikoloa, Hawaii, USA; and the Human-Computer Interaction International Conference 2024 in Washington DC, USA. It has also been successfully showcased at museum-focused events such as “ExICE - Extended Intelligence for Cultural Engagement”, Bologna IT (H2020 SPICE Project), AVICOM (ICOM International Committee for Audiovisual, New Technologies and Social Media) 2024 Annual Conference "News from the Digital Museum World” and during the NEMO Members’ Meetup.9 Specifically, ReInHerit applications were developed with a user-centered approach, aligned with recent research by NEMO, including the 2023 report on "Digital Learning and Education in Museums" [22]. This report highlights how digital tools such as AI and CV can strengthen emotional connections between museums and visitors. By incorporating playful elements, museums encourage active participation, allowing visitors to engage with collections rather than being passive observers. This approach addresses the growing demand for digital innovation and audience inclusion, reflects the key topics identified through research conducted by the ReInHerit project (Table 1). The Toolkit aims to increase engagement by combining educational content with interactive, playful features that foster memorable experiences. It promotes a dynamic perspective on museum engagement, shifting the focus from the collection itself to the relationship between the visitor and the exhibited objects.
Tools were designed to create an interactive museum experience, across three key contexts - personal, social, and physical - which, as Falk and Dierking point out, significantly influence perception, comprehension, and learning [23] [24]. Emotional engagement is particularly crucial to authentic learning, and digital tools can trigger multisensory experiences or foster social connections. As Falk highlighs in the NEMO report "Emotions and Learning in Museums" [25], museum experiences are rarely linear: they do not start when people enter the museum, nor do they end when they exit. Emotions and feelings play a crucial role at every stage of the museum experience: it is “feelings,” not rational thought, that drive visitor satisfaction. Emotions are at the heart of every stage of a museum visit. Falk’s research on the visitor experience shows that it is feelings, rather than dispassionate reasoning, that ultimately determine satisfaction and the desire to return. This centrality of affect is echoed in neuroscience: A. Damasio [26] argues that emotional processes are enmeshed in the neural networks of rational thought, shaping perception, decision making and memory. More recent work by L. F. Barrett [27] reflects on emotions as context-dependent, learned constructions over the course of a lifetime, highlighting how the brain’s “salience networks” assemble sensations, past experiences and cultural expectations into the feelings we label as joy, surprise, boredom or curiosity. Understanding these layers of emotional baggage clarifies why people choose to visit, how they behave in museums, and why positive encounters generate lasting memories and enthusiastic recommendations. In short, every aspect of the museum experience - entry motivations, on-site engagement, and post-visit memories - begins and ends with emotions. Motivated by these research insights, we adopted an emotional and dynamic approach in designing the ReInHerit digital tools, so as to connect the museum experience before, during, and after the visit.
An increasing number of studies emphasize that sensory immersion and emotional engagement are critical factors for visitor engagement in digital and virtual museums environments. This underscores the necessity for close collaboration between UX designers and developers to effectively translate affective experiences into deeper participation and improved memory retention [28]. By incorporating gamification and playful elements, the Toolkit aims to trigger emotions, inspire curiosity, and enhance enjoyment. Targeting a young, interactive, and non-expert audience, this approach integrates social, relational, and digital dimensions, personalizing engagement in line with recent studies on emotional response to art. Museum literature increasingly leverages such approaches to develop tools that engage users cognitively, emotionally, and sensorially, thereby promoting active interaction and participation [29]. The design of the Toolkit also draws upon research exploring how citizen curation can shift traditional curatorial models by placing the public at the center of interpretive practices. This facilitates emotional responses and encourages community-based storytelling. For instance, a user-centered web app has been developed to utilize emotions for both the creation and interpretation of art-related narratives, promoting reflection and emotional sharing [30]. Likewise, digital applications have examined the relationship between interactive performances and audience emotional participation through computationally mediated experiences [31]. Recent research on digital engagement has also identified innovative gamification techniques as powerful tools for advanced communication and interaction, including within the field of cultural heritage [32]. Gamification has been shown to positively influence users by provoking, modifying, or reinforcing their attitudes, behaviors, and perceptions of cultural heritage. A notable source of inspiration for ReInHerit was the integration of new visual technologies by institutions such as the Cleveland Museum of Art. Their ArtLens Gallery Oneapplication enables visitors to explore the permanent collection via a tablet, enhancing the museum experience with personalized digital interaction [33,34]. In the Toolkit, this type of visual interaction is enabled through more advanced, AI-based technologies capable of processing human-generated skeleton data, rather than relying on devices such as Microsoft Kinect. We implemented a BYOD strategy offering visitors a smarter, more dynamic, and enriched interactive experience, with the aim of enhancing the educational and learning process [35].
Based on these insights and survey results, the ReInHerit Toolkit strategy (Figure 10) focused on creating web-apps using the BYOD model to facilitate hybrid and extended interaction across both physical and digital spaces. This includes the introduction of innovative ways to interpret museum collections anda data, making them more engaging and accessible through the use of AI and Machine Learning (ML) techniques [36]. Specifically, recent research and survey also highlight the evolution of ML approaches in cultural heritage, highlighting both algorithmic progress and their specific application in enhancing cultural data analysis and interpretation [37]. In addition, the development of ethical frameworks tailored to cultural heritage is crucial to address the unique risks and opportunities posed by AI applications in preserving and managing cultural assets [38].
The Toolkit features interactive tools designed to enhance visitor engagement, motivate learning, and foster stronger connections between artworks and visitors The ReInHerit Digital Hub aims to bridge the technology gap between large and small organizations by providing accessible resources to improve digital capacity in the cultural heritage sector. The toolkit, which emphasizes open-source software, ensures more accessible and sustainable maintenance and reuse by diverse organizations. Detailed documentation and webinars support continuous learning and adaptability. ReInHerit has employed a bottom-up, community-driven approach, encouraging active participation through webinars, hackathons, workshops, and training sessions to assess needs, address challenges, and develop solutions.
A core aspect of the strategy is the responsible use of AI, guided by UNESCO’s Guidance for Generative AI in Education and Research [39] and the Recommendation on the Ethics of Artificial Intelligence [40]. Additionally, computer vision and face recognition systems are central to the ethical discussions surrounding AI, in line with the AI-Act, first regulation on artificial intelligence of the European Union. These frameworks ensure transparency, fairness, and user privacy. ReInHerit tools prioritize data privacy, with no personal information stored, and all generated media provided solely to the user. Ethical considerations are addressed through the ReInHerit Ethics Card, which assesses aspects like data training and security, copyright, scientific accuracy. Specifically, the neural networks used in applications such as Strike-a-Pose (Section 3.1) and Face-fit (Section 3.2) have been demonstrated to function equitably with users from diverse global regions and across various attributes (e.g., gender, age, skin color), as evidenced by the Model cards of the respective models 10. Moreover, Strike-a-Posehas been shown to work effectively with users who have various disabilities, such as those using wheelchairs. In this case, the curator can choose to replicate only specific parts of the artwork, for example, the torso, in order to better accommodate users’ needs. By promoting open-source codes and providing related resources via the Digital Hub, ReInHerit empowers smaller organizations to implement and adapt these tools, ensuring sustainable digital practices across the cultural heritage sector.

3. Results

The following sections describe selected results from CV/AI-based applications developed within the Toolkit11 and their role in enhancing visitor engagement, accessibility, and interactions with cultural heritage: Strike-a-Pose (Section 3.1) and Face-Fit (Section 3.2), two applications designed around the principles of gamification and interactive engagement with artworks, aiming to increase visitor participation in museum experiences; Smart Retrieval, which leverages CLIP for artwork recognition and image retrieval (Section 3.3), and Smart Lens, a web app that transforms mobile devices into magnifying tools for detailed artwork observation and contextual information (Section 3.4); VIOLA Multimedia Chatbot which integrates computer vision and natural language processing to provide a conversational interface for web and mobile platforms, utilizing speech-to-text for seamless, natural interaction Section 3.5; and finally Photo Video Restorer, a cutting-edge tool for AI-driven restoration of digital heritage video and photos archives, showcasing the potential of AI in preserving cultural heritage (Section 3.6) [41].

3.1. Strike-a-Pose

Strike-a-Pose (Figure 11) is an interactive online tool designed to let users mirror the body positions seen in artworks, including paintings and sculptures. It can be accessed directly from visitors’ personal devices (following a BYOD model) or via a fixed installation equipped with a display and integrated camera. The platform leverages game-like elements to turn the exploration of art into an enjoyable and participatory experience. Users are encouraged to physically imitate poses from the museum’s collection.[42]. To assist with alignment, skeleton outlines of both the user and the selected artwork are shown side by side, helping to match body points and movements. The application includes a variety of pose challenges, some of which are adapted to be inclusive—such as those focusing on torso movements only, making them accessible for users with limited mobility. After completing all the pose-matching tasks, users can create a customized video of their session, which they have the option to download or share on social media. Users receive the results of their interaction via e-mail with additional material for further study and learning. The conceptual metaphor behind the activity is to mirror the pose of a character in an artwork, fostering shared experiences and connecting people and stories.
In Figure 12, the workflow for pose detection and matching is depicted. The system takes as input a reference image and a camera frame.
Using pose landmark detection, the skeleton structure of the individual in both inputs is analyzed. The primary focus is on detecting the orientation and position of joints. The system compares the extracted joint orientations of the reference pose and the camera input. If the orientations match, the system proceeds to generate the output video showing the alignment of the camera input with the reference pose. If the orientations do not match, the process iterates until the correct alignment is achieved. This ensures precise tracking and pose replication.
The system was built using JavaScript for the front-end interface and Python on the server side. Human pose recognition is powered by the TensorFlow.js library 12, which integrates the MoveNet pose estimation model. MoveNet identifies 17 key body landmarks and is optimized for speed and precision—especially the “Lightning” version, ideal for applications requiring low latency, as it runs in real time across most standard desktop and mobile devices. All computations for pose detection are performed client-side in the browser, ensuring responsiveness. On the server side, a SQLite database manages the collection of artworks, challenge parameters, and metadata. Communication between the front end and the database is facilitated through REST APIs developed with Flask13. Video compilation is handled by the server. The user interface is designed responsively: it adjusts to a vertical layout for smartphones and a horizontal one for larger screens in fixed setups. To isolate the pose data from background context, the coordinates of each key point are normalized. Pose similarity is assessed through a calculation of Euclidean distance between the user’s pose and the reference pose, with a match being validated when distances remain within a set threshold over a brief time interval. We used a pre-trained model for body pose detection: MediaPipe Movenet.14
ReinHerit Consortium museums contributed to the development of the image gallery. As part of dissemination activities, the app was efficiently tested at international workshops in the tech and cultural sectors, involving diverse users (Figure 13).

3.2. Face-Fit

Face-Fit (see Figure 14) is an interactive software created with a combination of JavaScript and Python, designed to allow users to engage creatively with portrait paintings by customizing and animating artworks. Optimized for both smartphones and larger museum installations, and inspired by the ‘share-your-expression’ concept, the activity challenges participants to find the perfect match by aligning their facial orientation and expressions with those depicted in historical portraits. After successfully replicating the pose, users can blend their own facial image with the artwork, generating a new visual. Upon completion of the interaction, a personalized email is sent to users containing the final image ready for social media sharing, along with supplementary curated content (textual descriptions, audio guides, or video clips) selected by the museum.
The development process relied on an iterative usability testing cycle involving three separate groups of five participants each [43], as per best practices in user-centered design. To begin, users position themselves in front of a device with a camera and select a portrait from a vertical scrollable menu. A semi-transparent overlay of their live image is superimposed onto the chosen painting, helping them align their face accurately with the original. This ’ghost image’ design choice was implemented to maintain user attention and support engagement without detracting from the overall enjoyment of the game. Earlier prototypes used visual cues to guide alignment, but these were eventually removed due to their tendency to divert focus from the artwork itself. We used a pre-trained model for face landmark detection: MediaPipe Face Mesh. 15 The app leverages MediaPipe’s Face Mesh module 16, which uses TensorFlow Lite to perform lightweight and real-time facial landmark detection even on mobile devices [44]. The detected keypoints are analyzed to extract head rotation angles in three dimensions. These angles are compared with those of the artwork using Euclidean distance calculations, confirming a match when a preset threshold is met. An additional verification step checks the resemblance of key facial regions—specifically the eyes, eyebrows, and mouth—since these convey universal emotions, as identified in Ekman’s facial expression theory et al. [45]. Face-swapping is achieved by applying affine transformations between corresponding facial mesh triangles of the user and the portrait. To match the user’s photo stylistically with the artwork, a color correction algorithm based on Reinhard’s statistical method is used et al. [46]. Figure 15 illustrates the pipeline for facial landmark detection and matching, followed by a face swapping process.
Gallery images in the demo test version are part of WikiArt - Visual Art Encyclopedia. The version also includes a limited selection of artworks provided by museums participating in the ReinHerit project. In addition, the gallery has been populated with images of artworks and additional content provided by small and medium-sized museums in Italy. As part of dissemination activities, the web-app was successfully tested in workshops within museums and the cultural sector, with enthusiastic participation of diverse users (young people, adults, families, researchers, museum experts, etc.) from different backgrounds (Figure 16).
Finally, these two computer vision applications aim to engage visitors with artworks by detecting and interpreting body movements and facial expressions. Both are based on a "play and engage" approach that encourages active visitor engagement with cultural content, moving beyond passive observation. This model is also inspired by the neurological concept of mirror neurons, which are associated with understanding others’ actions and emotions through internal simulation. By inviting users to replicate poses and expressions found in art, the applications foster an empathic and embodied interaction with cultural heritage [47].
Gamification, understood as the application of game-like dynamics in non-game contexts, has proven effective in enhancing museum sustainability and visitor engagement [48]. Within this framework, Strike-a-Pose and Face-Fit demonstrate how gamified digital tools can transform heritage experiences into more playful, social, and personalized journeys of discovery—fully embracing the “I play, I learn” principle [49]. They support a transition from the traditional "look but don’t touch" museum experience to one that invites visitors to "play and engage," making cultural learning both memorable and meaningful. Accordingly, the objectives of these two applications are:
  • To implement these experiences as challenges that enhance visitor engagement and provide personalized takeaways of the visit, encouraging post-visit exploration.
  • To generate user-created content that can amplify engagement on social media platforms.
  • To employ advanced AI methods optimized for mobile execution, supporting a BYOD strategy for widespread accessibility.

3.3. Using CLIP for Artwork Recognition and Image Retrieval

In recent years, multimodal neural models —particularly CLIP [50] — have demonstrated superior performance compared to traditional models based on hand-crafted features, especially in the field of computer vision. CLIP, trained on 400 million image–text pairs, has shown excellent zero-shot generalization capabilities, meaning it can perform well on tasks without requiring additional training data. This makes it particularly suitable for domains where annotated datasets are scarce, such as cultural heritage. In our study, we explored the applicability of CLIP in recognizing artworks using the NoisyArt dataset [51], a collection of 89,095 images divided into 3,120 classes, enriched with metadata and textual descriptions from sources such as DBpedia and Flickr. CLIP was tested in three main tasks: supervised classification, zero-shot classification, and image retrieval. In each of these scenarios, CLIP significantly outperformed unimodal pre-trained models such as ResNet-50, confirming the effectiveness of the multimodal approach even when working with noisy and imperfectly annotated datasets. For an accurate description of the results please refer to [52].
In the supervised classification task, we trained a simple classifier on top of CLIP’s visual backbone using labeled examples from NoisyArt. Despite the presence of noise and class imbalance in the dataset, the model achieved an accuracy of 86.63% on the test set, demonstrating CLIP’s robustness even with minimal additional architecture and training. In the zero-shot classificationtask, we directly leveraged the similarity between images and textual descriptions as learned by CLIP, without performing any fine-tuning on the dataset. By comparing image embeddings with textual labels, we obtained a remarkable improvement of over 20 percentage points compared to the state-of-the-art methods, highlighting CLIP’s ability to generalize and recognize concepts based on prior vision–language training alone. In the image retrieval task, we experimented with multiple configurations. We developed both a baseline retrieval system using CLIP’s pre-trained image embeddings and more advanced variants. One of these incorporated a visual similarity-based re-ranking phase, improving result relevance by reordering retrieved items according to fine-grained visual closeness. Another version involved fine-tuning the CLIP network for retrieval-specific objectives, further enhancing performance in both image-to-image and description-to-image search tasks. These configurations allowed us to assess the flexibility and extensibility of CLIP for retrieval applications in cultural datasets.
These experimental results laid the foundation for the development of the Smart Retrievalapplication, designed to address real-world search and annotation needs in museum and archival settings. In particular, the application implements an innovative approach known as Combined Image Retrieval (CIR), which enables advanced multimodal search. Users can issue queries by combining a reference image with a textual description that modifies or refines the intended visual content. This functionality—highly powerful and rarely found in practical systems—was developed and tested as part of the ReInHerit project, with scientific demonstrations presented at ACM Multimedia 2023 and ICCV 2023. The app was designed to be accessible through a web interface and usable even on low-spec hardware, making it particularly suitable for small cultural institutions. It is currently being tested in collaboration with the Europeana Fashion Heritage Association, which provided a dataset of historical images for system evaluation. The entire architecture, based on CLIP, supports not only image retrieval but also zero-shot tagging, enabling automatic image annotation even in the absence of labeled datasets. This approach extends the capabilities for access, exploration, and valorization of visual heritage, especially for cultural organizations with limited resources.
As a concrete outcome of this research, the ReInHerit project integratedSmart Retrievalinto its Toolkit to provide advanced multimedia search capabilities. The platform employs Content-Based Image Retrieval (CBIR) in two distinct modes: users may search by entering a textual description, or by combining a reference image with a natural-language prompt that specifies or modifies certain attributes (Figure 17). What sets Smart Retrieval apart is its conditional retrieval mechanism: an advanced neural model fuses text and visual inputs to enable searches based on both similarity and semantic modifications. Additionally, the system offers automatic zero-shot tagging, eliminating the need for pre-annotated datasets—a crucial feature for smaller museums with limited resources. The web application’s frontend is built in JavaScript and HTML5, while the backend is implemented in Python, leveraging PyTorch for the computer vision components and Flask to expose RESTful endpoints that connect the UI to the vision engine. A demonstrator showcases both text-to-image and image-to-image retrieval on the NoisyArt corpus, achieving state-of-the-art performance. The conditional image retrieval feature is under evaluation with the Europeana Fashion Heritage Association’s newly provided fashion image dataset.
To address the cost and effort of creating labeled training data, we extended our CIR method to a zero-shot setting. This approach maps reference image features into a pseudo-word token in CLIP’s embedding space and combines it with the text caption - allowing conditioned retrieval without any dataset-specific training. Our zero-shot CIR method surpasses previous benchmarks on both the FashionIQ and CIRR datasets. By integrating these innovations, Smart Retrieval transforms static image archives into interactive, semantically aware platforms, enhancing digital access, curation, and interpretation in the cultural heritage domain.

3.4. Smart Lens

Smart Lens (see Figure 18) is a mobile-friendly web application that turns smartphones and tablets into interactive visual tools for artwork exploration. It allows users to analyze specific details within artworks by framing them through the device’s camera. Recognition is performed in real time using a combination of computer vision techniques - namely image similarity search (CBIR), visual classification, and object detection. As soon as a visual element is identified, the app delivers curated multimedia content, such as text, images, audio, or video, linked to that detail. This transforms the viewing experience into a more dynamic and personalized interaction with the art.
The app differs from traditional guide systems based on static elements like QR codes by encouraging users to actively investigate and interpret what they see. This type of visual engagement transforms passive observation into a form of discovery, promoting a deeper connection with the artwork. At the same time, the application collects anonymized usage data—such as the details most frequently recognized or explored—offering curators a new way to understand visitor behavior and improve exhibit design. Smart Lens supports three distinct recognition modes:
  • Content-based Image Retrieval (CBIR) - This method compares visual descriptors from the user’s live camera feed with those extracted from a curated image dataset. Each artwork is not only represented as a whole but also partitioned into segments, so that fine-grained features can be detected and matched efficiently.
  • Classification - A neural network model, specifically fine-tuned for the collection, assigns a class label to the input frame based on overall appearance. The recognition result is accepted only if the confidence score exceeds a designated threshold. This lightweight solution is ideal for running directly on mobile devices.
  • Object Detection - In this mode, the system pinpoints and labels multiple details within a single artwork using bounding boxes. The underlying model, optimized for detecting artwork-specific elements, selects only those regions whose confidence level meets predefined criteria. This approach is particularly suited for complex works with multiple visual components.
The frontend is implemented in responsive HTML5 and CSS, dynamically updated via JavaScript to support real-time feedback. As the user navigates through the artwork with their camera, the app continuously compares live input to the database and presents contextual information on-screen. TensorFlow-based models17 are employed for classification and retrieval, while SSD/MobileNetV3 powers the detection functionality. For every recognized element, users receive access to an image, descriptive metadata, and any associated media content. If no audio file is available, an integrated text-to-speech engine ensures that the user can still experience the application as a full-featured audio guide. Additionally, the app can operate in a simplified mode that focuses solely on identifying whole artworks, bypassing the detail-oriented recognition pipeline. The web app and its backend are hosted on a remote server, accessible through a simple QR code to streamline entry and avoid manual URL input. By promoting interactive exploration and supporting fine-grained analysis of art details, Smart Lens offers a smart, engaging alternative to static museum guides. It invites visitors to become active participants in the exhibition experience while empowering institutions with new tools for data collection and visitor engagement.
Smart Lens app was tested during the ReInHerit ReThinking Exhibitions 18 held at the GrazMuseum (Austria), the Museum of Cycladic Art (Athens, Greece), and the Bank of Cyprus Cultural Foundation (Nicosia, Cyprus),(Figure 19).

3.5. Multimedia Chatbot: VIOLA

VIOLA (Visual Intelligence OnLine Art-assistant)19 is a web and mobile application designed to enhance interaction with cultural heritage content through a chatbot interface powered by AI. Users can engage with the system using natural language to ask questions about artworks, including both visual elements and historical or contextual information. The chatbot employs a hybrid architecture that merges computer vision (CV) and natural language processing (NLP) techniques. This enables it to interpret and respond to different types of queries, offering a more intuitive and flexible user experience. The design is inspired by the growing popularity of conversational AI platforms like ChatGPT, now widely adopted across various domains.
The backend is developed in Python, utilizing Flask to provide a REST API that connects with the frontend. Two versions of the backend are available, one of which incorporates three neural networks to handle different functionalities:
  • A neural network classifies the user’s query, determining whether it pertains to the visual content or the contextual aspects of the artwork.
  • A question-answering (QA) neural network uses contextual information about the artwork, stored in JSON format, to address questions related to its context.
  • A visual question-answering (VQA) neural network processes the visual data and the visual description of the artwork, stored in JSON format, to answer questions about the content of the artworkn[53].
To further enhance accuracy and dialogue capabilities, large language models (LLMs), including GPT-based architectures, are employed for more complex question-answering tasks that require deeper contextual understanding. An example of the app is shown in the figure below (Figure 20).
The application is entirely web-based, combining a backend system that runs the visual question answering engine with a frontend interface designed for user interaction. The interface, developed using modern web technologies, is fully responsive and adapts smoothly to both desktop and mobile environments. This flexibility allows VIOLA to be embedded into existing museum websites or used as a standalone mobile solution, offering an innovative and intelligent visitor guide. To improve usability on smartphones and tablets, the system includes voice input through speech recognition, reducing the need for manual typing. Its modular design makes it easily customizable—museums can update visual assets and metadata to reflect their own collections. Additionally, VIOLA provides a scalable framework for more advanced features, including interactive learning tools, gamified experiences, and natural language-driven retrieval systems. The integration of large-scale multimodal language models (MLLMs) enhances its ability to handle complex visual-contextual questions, combining image understanding with contextual interpretation for richer, more informative responses [54].
Chatbots, through recent technological advances, create new opportunities for museums and galleries to engage younger audiences through innovative narrative visualization [55]. Building on this approach, the VIOLA web app has been implemented in small and medium-sized museums as a BYOD tool, enabling new audiences and young visitors to access detailed information about artworks while prioritizing ethics and transparency, particularly in terms of privacy, data accuracy, and training data sources. Initially, VIOLA was tested using the ArtPedia dataset, a standard benchmark for VQA systems, but has since evolved to incorporate a curated selection of artworks from partner museums, including the GrazMuseum Museum of Cycladic Art and the Bank of Cultural Cyprus Foundation. This transition to museum-provided content ensures a higher degree of accuracy and scientific reliability, allowing the chatbot to deliver responses grounded in expert knowledge. In addition to the first VIOLA Gallery with the ReInHerit Consortium’s museum artowrks, an extended version of VIOLA was implemented in small and medium-sized museums in Italy following a BYOD approach that allows visitors to access the chatbot online and using their smartphones. A case study has been conducted at the Gipsoteca di Arte Antica (GiArA) of the University of Pisa. (Figure 21)
A dedicated VIOLA gallery has been created that features artworks from the GiAra Museum, all content validated by the museum’s curators. To minimize errors, the chatbot uses those validated content from museum experts, but continuous updates are necessary to address uncertainties, improve language processing, and personalize user interactions. A central feature of the VIOLA chatbot is its "prompt engineering", designed to prevent incorrect responses, known as "hallucinations". Curated textual data on artwork ensure that the chatbot answers are accurate and aligned with validated datasets. Prompt engineering also helps minimize unanswered questions by enabling the chatbot to address a wide range of queries, from general curiosities to more detailed historical inquiries. During the ReInHerit project testing workshops, users often began with curiosity-driven questions about the stories behind artworks and gradually explored more specific historical and scientific topics. The VIOLA content, provided by curators, often lacked information related to these user curiosity. The system was enhanced with an Admin interface that allows curators to update content, as well as a feature that tracks unanswered questions, helping to fill information gaps. This dynamic and speech-based approach to the use of chatbot in museums [56], improves the ability of the chatbot to respond to general questions driven by curiosity and more scientific or historical questions, aiming to avoid incorrect answers while providing comprehensive responses to the diverse questions of visitors. This user-oriented approach (Figure 22) promotes continuous collaboration between developers and curators. In this way, the chatbot does not replace the curators, but is an integrative tool that supports their digitisation efforts. It continuously relies on the curators to update and validate artwork descriptions, ensuring that the chatbot provides accurate, relevant information, and engaging for visitors.

3.6. Smart Video and Photo Restorer

Smart Video and Photo Restorer is an advanced system developed to recover analog videos and aged photographs from historical archives affected by substantial visual degradation due to tape deterioration. Traditionally, restoration of such content is carried out manually by expert archivists using commercial tools, editing frame by frame—a process that is labor-intensive and costly. The solution we propose adopts a multi-frame restoration strategy, which is particularly effective in handling severe tape mistracking artifacts that often result in visually corrupted or scrambled frames. To train the model, we constructed a synthetic dataset that closely emulates the types of degradation observed in real-world analog footage provided by Archivio Storico Luce, the largest historical video archive in Italy. These actual analog recordings guided the creation of high-fidelity simulations of degradation. The synthetic dataset was generated by applying various distortions—such as Gaussian noise, white speckles, chroma bleeding, and horizontal displacements—to high-quality digital video, using Adobe After Effects to replicate the effects of analog tape damage. These modified sequences were then paired with their clean versions to form ground-truth training data. The final dataset included 26,392 frames, which were divided into training and validation subsets.
The restoration architecture we developed is based on a Swin-UNet (see Figure 23, that works on videos, using a multi-frame approach to enhance T frames at once, exploiting spatio-temporal information. We employed 3D convolution for partitioning the input into patches and pixel shuffle for the patch expanding layer, allowing the model to learn the residual difference between degraded and restored frames—thereby stabilizing training and speeding up convergence [57].
The training procedure used a weighted combination of pixel-wise loss and a perceptual loss calculated in the VGG-19 feature space [58]. The model processes patches of size 256×256, cropped randomly from video frames. During both training and inference, the number of consecutive frames T was set to 5. To evaluate the effectiveness of our method, we used three standard full-reference image quality metrics: PSNR, SSIM [59], and LPIPS [60]. On the synthetic dataset, our model outperformed DeOldify [61], a well-known image and video restoration framework. We then applied the trained model to real analog recordings from Archivio Storico Luce, where it demonstrated excellent generalization and superior restoration quality—establishing it as a state-of-the-art solution for analog video enhancement.
Table 2. Evaluation results using synthetic data.
Table 2. Evaluation results using synthetic data.
Method PSNR ↑ SSIM ↑ LPIPS ↓
DeOldify [61] 11.56 0.451 0.671
Ours 34.78 0.939 0.063
Finally, to allow users to restore their own degraded photos or videos, we developed a Flask-based demo web app that users can access through a web browser (Figure 24). The platform supports the upload of video files and provides the user with the downloadable restored result, as well as a comparison with the original video/photo. Alternatively, the user can choose one of the example videos/photos to see what the model is capable of. To make this powerful restoration technology accessible, Smart Video and Photo Restorer App is an intuitive web application designed for archivists, curators, and cultural institutions working with degraded analog footage. This user-friendly tool allows professionals to upload old photos or analog videos directly through a browser and receive high-quality restored versions, eliminating the need for costly and time-intensive manual frame-by-frame editing. Leveraging our advanced neural network and multi-frame approach, the application can handle even severe degradations such as frame fragmentation due to tape mistracking. By restoring visibility to rare and fragile visual materials, Smart Video Restoration offers a practical solution for safeguarding audiovisual heritage and ensuring broader public access to cultural memory preserved in historical footage.

4. Discussion

The findings of the ReInHerit project and its Toolkit validate the evolving concept of the interactive museum, which leverages digital tools across the entire visitor journey, as seen in Section 1. This approach positions museums as spaces for unique, informal learning experiences that attract diverse audiences and deepen visitor engagement as shown in our survey analysis (Section 1.1). Recent museum studies, as discussed in Section 2.1, emphasize the importance of digital tools, in particular those based on AI and CV, in enriching visitor interactions and providing educational content through storytelling and personalized engagement, enhancing museum collections, and ultimately improving the user experience by making it more dynamic, informative, and tailored to individual interests. The research conducted by ReInHerit highlights the necessity of such digital tools for enhancing museum experiences, particularly for younger, digitally savvy audiences, who prefer interactive, BYOD-friendly tools over native apps, as shown in the ReInHerit Strategy (Section 2.2). Here, CV and AI have proven invaluable in personalizing interactions and require active and visual engagement from users, making the experience more interactive, engaging, and smarter, as elaborated in Section 3.4 and fostering a participatory, user-oriented process, as highlighted in Section 3.5. The ReInHerit Toolkit thus focuses on visitor-centered experiences that prioritize BYOD web applications for seamless interaction, designed to heighten emotional and playful engagement, which is crucial for effective and memorable digital learning. With a commitment to open-source development (Section 3), the ReInHerit project offers adaptable and sustainable digital solutions tailored to museums of all sizes, especially small to medium-sized institutions (Section 2). By fostering an open-source approach, the Toolkit enhances maintainability and reuse, empowering organizations with limited resources to implement and sustain digital innovations.
The project also advocates a multidisciplinary, collaborative approach—integral to the Digital Hub model where resources and best practices are shared, fostering community-wide advancement in digital heritage management. According to focus groups conducted with museum professionals described in Section 1.4, it is useful to overcome traditional boundaries between disciplines and it is crucial to develop tools in dialogue with visitors and to invite communities into the creation process. The main objective of digital innovation, as described in Section 2.2, is to provide not only a tool as an end product, but a transdisciplinary development process, promoting collaboration and integration of knowledge from different areas, creating a dialogue and mediation between different disciplinary fields. Following this approach, the ReInHerit Toolkit was designed and tested with a bottom-up approach, inviting communities to participate in the creation process through workshops and hackathons. Toolkit apps have been tested and studied during interdisciplinary hackathons at the AI/XR Summer School held in Matera in July 2023. During the week, international speakers and experts discussed and engaged with students and researchers with different backgrounds and skills. Young PhD students worked on the Toolkit using open-source codes shared by the Digital Hub. Multidisciplinary groups of young Ph.D. students and tailored experts worked on the on two main themes: ’Gamification and playful engagement’ and ’Smart Interaction and digital contents’, using open-source codes from Digital Hub and adapted them to artworks from some local museums This co-creation process added new technological developments for apps and user interaction scenarios, improving engagement, inclusivity, and new design features 20. ReInHerit’s impact has been recognized through presentations and demonstrations at national and international events, including the ACM Multimedia 2022 Conference, where the applications Strike-a-Pose and Face-Fit (Section 3.1, Section 3.2) received the Best Demo Honourable Mention Award for Engaging Museum Experience. The web app used to test the retrieval system (Section 3.3) received the Best Demo Award Honorable Mention at the Computer Vision and Pattern Recognition (CVPR) 2022 Conference. Further, the toolkit has been included as a case study in the European Heritage Hub 21, has been presented at several national and international conferences, and is gaining visibility as part of Italy’s “Museums + AI Toolkit" [62] initiative, exemplifying how ReInHerit bridges the technology gap and promotes equitable access to cultural heritage innovation.

5. Conclusions

The ReInHerit Toolkit represents an innovative, collaborative, and community-driven model for digital cultural heritage, effectively bridging the gap between ICT research, museums, creative industries, and Digital Humanities. Developed within the ReInHerit project, it exemplifies a forward-thinking approach to digital transformation by embracing user-centered design, interdisciplinary co-creation, and open-source accessibility. The Toolkit equips institutions—especially small and medium-sized museums—with adaptable and sustainable tools to enrich visitor engagement and foster meaningful digital innovation. Aligned with the evolving paradigm of people-centered museums, it facilitates multichannel, participatory experiences that make cultural engagement more inclusive, playful, and responsive to the diverse interests of contemporary audiences, including younger generations.
By rethinking the relationship between visitors and collections, the Toolkit introduces interactive and personalized digital tools that enhance storytelling, emotional engagement, and informal learning. These developments are grounded in a cross-disciplinary investigation of key topics such as emotional museums, playful experiences and gamification, audience diversification, sustainability, and bottom-up co-creation. Importantly, ReInHerit also provides a strong foundation for the ethical and sustainable integration of Artificial Intelligence into cultural heritage practices. It supports museums in developing new knowledge about their objects and in enriching collection data in ways that significantly enhance the user experience and interpretive possibilities. Ultimately, the ReInHerit Toolkit offers a future-oriented vision for innovation in the cultural sector, reshaping museums as dynamic, participatory, and inclusive spaces for cultural interpretation, knowledge production, and creative engagement.

Author Contributions

Conceptualization, P.M. and M.B.; methodology, M.B, P.M. and A.F, software, M.B, A.F and F.P.; validation, all authors; investigation, P.M. and M.B; resources, P.M and M.B.; data curation, M.B.; writing—original draft preparation, P.M.; writing—review and editing, P.M. and A.F.; visualization, F.P. and P.M; supervision, M.B.; project administration, P.M.; funding acquisition, M.B. All authors have read and agreed to the published version of the manuscript.

Funding

This work is supported by the European Commission under European Horizon 2020 Programme, grant number 101004545—ReInHerit.

Acknowledgments

The authors would like to thank all colleagues from the ReInHerit project and the MICC researchers who contributed to the outcomes presented in the Results section.

Conflicts of Interest

The authors declare no conflicts of interest.

Abbreviations

The following abbreviations are used in this manuscript:
CV Computer Vision
AI Artificial Intelligence
CH Cultural Heritage
ICT Information and Communication Technologies
GDPR General Data Protection Regulation
BYOD Bring Your Own Device
API Application Programming Interface
CLIP Contrastive Language-Image Pre-Training
CBIR Content-based Image Retrieval
GPU Graphics Processing Unit
VQA Visual Question Answering
PSNR Peak signal-to-noise ratio
SSIM Structural Similarity Index
LPIPS Learned Perceptual Image Patch Similarity

References

  1. Stamatoudi, I.; Roussos, K. A Sustainable Model of Cultural Heritage Management for Museums and Cultural Heritage Institutions. ACM Journal on Computing and Cultural Heritage, 2024. [CrossRef]
  2. Gaia, G.; Boiano, S.; Borda, A., Engaging Museum Visitors with AI: The Case of Chatbots. In Museums and Digital Culture: New Perspectives and Research; Giannini, T.; Bowen, J.P., Eds.; Springer International Publishing: Cham, 2019; pp. 309–329. [CrossRef]
  3. Adahl, S.; Träskman, T. Archeologies of Future Heritage: Cultural Heritage, Research Creations, and Youth. Mimesis Journal 2024, 13, 487–497. [Google Scholar] [CrossRef]
  4. Pietroni, E. Multisensory Museums, Hybrid Realities, Narration, and Technological Innovation: A Discussion Around New Perspectives in Experience Design and Sense of Authenticity. Heritage 2025, 8. [Google Scholar] [CrossRef]
  5. Ivanov, R.; Velkova, V. Analyzing Visitor Behavior to Enhance Personalized Experiences in Smart Museums: A Systematic Literature Review. Computers 2025, 14. [Google Scholar] [CrossRef]
  6. MusuemBooster. Museum Innovation Barometer, 2022. https://www.museumbooster.com/mib. Accessed on 7 May 2025.
  7. Giannini, T.; Bowen, J. Museums and Digital Culture: From Reality to Digitality in the Age of Covid-19, 2021. [CrossRef]
  8. Thiel, S.; Bernhardt, J.C. (Eds.) Reflections, Perspectives and Applications; transcript Verlag: Bielefeld, 2023; pp. 117–130. [Google Scholar] [CrossRef]
  9. Clarencia, E.; Tiranda, T.G.; Achmad, S.; Sutoyo, R. The Impact of Artificial Intelligence in the Creative Industries: Design and Editing. In Proceedings of the Proc. of International Seminar on Application for Technology of Information and Communication (iSemantic); 2024; pp. 440–444. [Google Scholar] [CrossRef]
  10. Muto, V.; Luongo, S.; Sepe, F.; Prisco, A. Enhancing Visitors’ Digital Experience in Museums through Artificial Intelligence. In Proceedings of the Proc. of Business Systems Laboratory International Symposium, Palermo, Italy; 2024. [Google Scholar]
  11. Furferi, R.; Di Angelo, L.; et al. Enhancing traditional museum fruition: Current state and emerging tendencies. Heritage Science 2024, 12. [Google Scholar] [CrossRef]
  12. Villaespesa, E.; Murphy, O. This is not an apple! Benefits and challenges of applying computer vision to museum collections. Museum Management and Curatorship 2021, 36, 362–383. [Google Scholar] [CrossRef]
  13. Osoba, O.A.; Welser IV, W.; Welser, W. An intelligence in our image: The risks of bias and errors in artificial intelligence; Rand Corporation, 2017.
  14. Villaespesa, E.; Murphy, O. THE MUSEUMS + AI NETWORK - AI: A Museum Planning Toolkit. https://themuseumsai.network/toolkit/, 2020. Accessed on 7 May 2025. [CrossRef]
  15. Zielke, T. Is Artificial Intelligence Ready for Standardization? In Proceedings of the Systems, Software and Services Process Improvement; Yilmaz, M.; Niemann, J.; Clarke, P.; Messnarz, R., Eds., Cham; 2020; pp. 259–274. [Google Scholar]
  16. Pasikowska-Schnass, M.; Young-Shin, L. Members’ Research Service, “Artificial intelligence in the context of cultural heritage and museums: Complex challenges and new opportunities” EPRS | European Parliamentary Research PE 747.120 – May 2023. https://www.europarl.europa.eu/RegData/etudes/BRIE/2023/747120/EPRS_BRI(2023)747120_EN.pdf. Accessed on 7 May 2025.
  17. CM. Recommendation CM/Rec(2022)15 Adopted by the Committee of Ministers on 20 May 2022 at the 132nd Session of the Committee of Ministers. https://search.coe.int/cm?i=0900001680a67952, 2022. Accessed on 7 May 2025.
  18. Furferi, R.; Colombini, M.P.; Seymour, K.; Pelagotti, A.; Gherardini, F. The Future of Heritage Science and Technologies: Papers from Florence Heri-Tech 2022. Heritage Science 2024, 12, 155. [Google Scholar] [CrossRef]
  19. Boiano, S.; Borda, A.; Gaia, G.; et al. Ethical AI and Museums: Challenges and New Directions. In Proceedings of the Proc. of EVA London 2024, 2024. [Google Scholar] [CrossRef]
  20. Orlandi, S.D.; De Angelis, D.; Giardini, G.; Manasse, C.; Marras, A.M.; Bolioli, A.; Rota, M. IA FAQ INTELLIGENZA ARTIFICIALE - AI FAQ ARTIFICIAL INTELLIGENCE. https://doi.org/10.5281/zenodo.15069460, 2025. Zenodo. Accessed on 7 May 2025.
  21. Nikolaou, P. Museums and the Post-Digital: Revisiting Challenges in the Digital Transformation of Museums. Heritage 2024, 7, 1784–1800. [Google Scholar] [CrossRef]
  22. Barekyan, K.; Peter, L. Digital Learning and Education in Museums: Innovative Approaches and Insights. https://www.ne-mo.org/fileadmin/Dateien/public/Publications/NEMO_Working_Group_LEM_Report_Digital_Learning_and_Education_in_Museums_12.2022.pdf, 2023 NEMO. Accessed on 7 May 2025.
  23. Falk, J.H.; Dierking, L.D. The Museum Experience, 1st ed.; Routledge, 2011. [CrossRef]
  24. Falk, J.; Dierking, L. The Museum Experience Revisited; 2016. [CrossRef]
  25. Mazzanti, P.; Sani, M. Emotions and Learning in Museums. https://www.ne-mo.org/fileadmin/Dateien/public/Publications/NEMO_Emotions_and_Learning_in_Museums_WG-LEM_02.2021.pdf, 2021 NEMO. Accessed on 7 May 2025.
  26. Damásio, A.R. Descartes’ Error: Emotion, Reason, and the Human Brain; Avon Books: New York, NY, USA, 1994. [Google Scholar]
  27. Barrett, L.F. How Emotions Are Made; Houghton Mifflin Harcourt: New York, NY, USA, 2017. [Google Scholar]
  28. Pietroni, E.; Pagano, A.; Fanini, B. UX Designer and Software Developer at the Mirror: Assessing Sensory Immersion and Emotional Involvement in Virtual Museums. Studies in Digital Heritage 2018, 2, 13–41. [Google Scholar] [CrossRef]
  29. Pescarin, S.; Città, G.; Spotti, S. Authenticity in Interactive Experiences. Heritage 2024, 7, 6213–6242. [Google Scholar] [CrossRef]
  30. Lieto, A.; Striani, M.; Gena, C.; Dolza, E.; Anna Maria, M.; Pozzato, G.L.; Damiano, R. A sensemaking system for grouping and suggesting stories from multiple affective viewpoints in museums. Human–Computer Interaction 2024, 39, 109–143. [Google Scholar] [CrossRef]
  31. Damiano, R.; Lombardo, V.; Monticone, G.; Pizzo, A. Studying and designing emotions in live interactions with the audience. Multimedia Tools and Applications 2021, 80, 6711–6736. [Google Scholar] [CrossRef]
  32. Khan, I.; Melro, A.; Amaro, A.C.; Oliveira, L., Role of Gamification in Cultural Heritage Dissemination: A Systematic Review; 2021; pp. 393–400. https://doi.org/10.1007/978-981-16-2377-6_37. [CrossRef]
  33. Alexander, J. Gallery One at the Cleveland Museum of Art. Curator: The Museum Journal 2014, 57, 347–362. [Google Scholar] [CrossRef]
  34. Alexander, J.; Barton, J.; Goeser, C. Transforming the Art Museum Experience: Gallery One. In Proceedings of the Museums and the Web 2013; Proctor, N.; Cherry, R., Eds., Silver Spring, MD; 2013. [Google Scholar]
  35. Fan, S.; Wei, J. Enhancing Art History Education Through the Application of Multimedia Devices. In Proceedings of the Proc. of International Conference on Internet, Education and Information Technology (IEIT 2024). Atlantis Press, 2024, pp. 425–436. [CrossRef]
  36. Münster, S.; Maiwald, F.; di Lenardo, I.; Henriksson, J.; Isaac, A.; Graf, M.M.; Beck, C.; Oomen, J. Artificial Intelligence for Digital Heritage Innovation: Setting up a R&D Agenda for Europe. Heritage 2024, 7, 794–816. [Google Scholar] [CrossRef]
  37. Fiorucci, M.; Khoroshiltseva, M.; Pontil, M.; Traviglia, A.; Del Bue, A.; James, S. Machine Learning for Cultural Heritage: A Survey. Pattern Recognition Letters 2020, 133, 102–108. [Google Scholar] [CrossRef]
  38. Pansoni, S.; Tiribelli, S.; Paolanti, M.; Di Stefano, F.; Frontoni, E.; Malinverni, E.S.; Giovanola, B.; et al. Artificial intelligence and cultural heritage: Design and assessment of an ethical framework. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences 2023, 48, 1149–1155. [Google Scholar] [CrossRef]
  39. UNESCO. Guidance for generative AI in education and research, 2023. Accessed on 7 May 2025. [CrossRef]
  40. UNESCO. Recommendation on the Ethics of Artificial Intelligence SHS/BIO/PI/2021/1. https://unesdoc.unesco.org/ark:/48223/pf0000381137. Accessed on 7 May 2025.
  41. Rachabathuni, P.K.; Mazzanti, P.; Principi, F.; Ferracani, A.; Bertini, M. Computer Vision and AI Tools for Enhancing User Experience in the Cultural Heritage Domain. In Proceedings of the Proc. of HCI International 2024 – Late Breaking Papers; Zaphiris, P.; Ioannou, A.; Sottilare, R.A.; Schwarz, J.; Rauterberg, M., Eds., Cham; 2025; pp. 345–354. [Google Scholar] [CrossRef]
  42. Donadio, M.G.; Principi, F.; Ferracani, A.; Bertini, M.; Del Bimbo, A. Engaging Museum Visitors with Gamification of Body and Facial Expressions. In Proceedings of the Proc. of ACM International Conference on Multimedia, New York, NY, USA, 2022; MM ’22; pp. 7000–7002. [Google Scholar] [CrossRef]
  43. Nielsen, J. Iterative user-interface design. Computer 1993, 26, 32–41. [Google Scholar] [CrossRef]
  44. Kartynnik, Y.; Ablavatski, A.; Grishchenko, I.; Grundmann, M. Real-time Facial Surface Geometry from Monocular Video on Mobile GPUs. https://arxiv.org/abs/1907.06724, 2019, [arXiv:cs.CV/1907.06724].
  45. Ekman, P.; Friesen, W.V.; O’sullivan, M.; Chan, A.; Diacoyanni-Tarlatzis, I.; Heider, K.; Krause, R.; LeCompte, W.A.; Pitcairn, T.; Ricci-Bitti, P.E.; et al. Universals and cultural differences in the judgments of facial expressions of emotion. Journal of personality and social psychology 1987, 53, 712. [Google Scholar] [CrossRef]
  46. Reinhard, E.; Adhikhmin, M.; Gooch, B.; Shirley, P. Color transfer between images. IEEE Computer graphics and applications 2001, 21, 34–41. [Google Scholar] [CrossRef]
  47. Gallese, V. Embodied Simulation. Its Bearing on Aesthetic Experience and the Dialogue Between Neuroscience and the Humanities. Gestalt Theory 2019, 41, 113–127. [Google Scholar] [CrossRef]
  48. Karahan, S.; Gül, L.F. Mapping Current Trends on Gamification of Cultural Heritage. In Proceedings of the Game + Design Education; Cordan, Ö.; Dinçay, D.A.; Yurdakul Toker, Ç.; Öksüz, E.B.; Semizoğlu, S., Eds., Cham; 2021; pp. 281–293. [Google Scholar]
  49. Bonacini, E.; and, S.C.G. Gamification and cultural institutions in cultural heritage promotion: A successful example from Italy. Cultural Trends 2022, 31, 3–22. [Google Scholar] [CrossRef]
  50. OPEN-AI. CLIP: Connecting text and images. https://openai.com/research/clip, 2021. Accessed on 7 May 2025.
  51. Chiaro, R.D.; Bagdanov, A.; Bimbo, A.D. NoisyArt: A Dataset for Webly-supervised Artwork Recognition. In Proceedings of the Proc. of International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2019). INSTICC, SciTePress; 2019; pp. 467–475. [Google Scholar] [CrossRef]
  52. Baldrati, A.; Bertini, M.; Uricchio, T.; Del Bimbo, A. , Exploiting CLIP-Based Multi-modal Approach for Artwork Classification and Retrieval. In The Future of Heritage Science and Technologies: ICT and Digital Heritage; Springer International Publishing, 2022; pp. 140–149. [CrossRef]
  53. Bongini, P.; Becattini, F.; Bimbo, A.D. Is GPT-3 all you need for Visual Question Answering in Cultural Heritage? https://arxiv.org/abs/2207.12101, 2023, [arXiv:cs.CV/2207.12101].
  54. Rachabatuni, P.K.; Principi, F.; Mazzanti, P.; Bertini, M. Context-aware chatbot using MLLMs for Cultural Heritage. In Proceedings of the Proc. of ACM Multimedia Systems Conference, New York, NY, USA, 2024; MMSys ’24; pp. 459–463. [Google Scholar] [CrossRef]
  55. Boiano, S.; Borda, A.; Gaia, G.; Rossi, S.; Cuomo, P. Chatbots and new audience opportunities for museums and heritage organisations. In Proceedings of the Proc. of Conference on Electronic Visualisation and the Arts, Swindon, GBR, 2018; EVA ’18; pp. 164–171. [Google Scholar] [CrossRef]
  56. Gustke, O.; Schaffer, S.; Ruß, A. CHIM—Chatbot in the Museum. Exploring and Explaining Museum Objects with Speech-Based AI. In AI in Museums; Thiel, S.; Bernhardt, J.C., Eds.; transcript Verlag: Bielefeld, 2023; pp. 257–264. Online ISBN: 978-3-8394-6710-7, Creative Commons Attribution 4.0 License (CC BY 4.0), Retrieved 2025-01-18 15:14:07-CET, ISBN 978-3-8394-6710-7. [CrossRef]
  57. Galteri, L.; Bertini, M.; Seidenari, L.; Uricchio, T.; Del Bimbo, A. Increasing Video Perceptual Quality with GANs and Semantic Coding. In Proceedings of the Proc. of ACM International Conference on Multimedia, New York, NY, USA, 2020; MM ’20; pp. 862–870. [Google Scholar] [CrossRef]
  58. Simonyan, K.; Zisserman, A. Very Deep Convolutional Networks for Large-Scale Image Recognition. In Proceedings of the Proc. of International Conference on Learning Representations, ICLR; Bengio, Y.; LeCun, Y., Eds. 2015. [Google Scholar]
  59. Wang, Z.; Bovik, A.C.; Sheikh, H.R.; Simoncelli, E.P. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing 2004, 13, 600–612. [Google Scholar] [CrossRef] [PubMed]
  60. Zhang, R.; Isola, P.; Efros, A.A.; Shechtman, E.; Wang, O. The Unreasonable Effectiveness of Deep Features as a Perceptual Metric. In Proceedings of the Proc. of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Los Alamitos, CA, USA; 2018; pp. 586–595. [Google Scholar] [CrossRef]
  61. Antic. DeOldify. https://github.com/jantic/DeOldify. Accessed on 7 May 2025.
  62. Murphy, O.; Villaespesa, E.; Gaia, G.; Boiano, S.; Elliott, L. MUSEI E INTELLIGENZA ARTIFICIALE: UN TOOLKIT DI PROGETTAZIONE; 2024. [CrossRef]
1
The Hub collects resources, and training material to foster and support cultural tourism in museums and heritage sites, and a networking platform to connect and exchange experiences. Website: https://reinherit-hub.eu/resources
2
3
ReInHerit D3.9 - Training Curriculum and Syllabi - https://ucarecdn.com/095df394-fad6-4f35-bdcc-09 769 931d0b8dd2/
4
5
D3.1 - ReInHerit National Surveys Report - https://ucarecdn.com/54faa991-1570-4a53-9e8a-c1dea0a33110/
6
The national surveys complied with GDPR, ensuring full anonymization of data by not requesting personal details. Participants were also informed about the survey’s purpose and how the data would be used. Similarly, the focus groups followed strict ethical guidelines, with informed consent procedures and data anonymization carefully planned according to the project’s Ethics and Data Management Plan. An additional central aim was to ensure full adherence to GDPR standards. See ReInHerit Deliverables D2.1, D2.4, D3.1 https://reinherit-hub.eu/deliverables
7
D3.2 - ReInHerit Toolkit Strategy Report - https://ucarecdn.com/71ffe888-3c0d-470d-962d-ab145edcff3f/
8
9
10
Ethical Aspects and Scientific Accuracy of AI/CV-based tools: https://reinherit-hub.eu/bestpractices/db1bd5ab-218f-480b-b709-06ac9ab72b33
11
12
13
14
15
16
17
18
19
20
Students and researchers from different international academic backgrounds participated in the international XR/AI Summer School 2023 from 17 to 22 July 2023 in Matera Italy, working on the topics of Extended Reality and Artificial Intelligence. More info on ReInHerit Hackathon and project proposals:https://reinherit-hub.eu/summerschool/
21
Figure 1. Digital Transformation and Innovation Process in ReInHerit(Source: D3.1 National Surveys Report).
Figure 1. Digital Transformation and Innovation Process in ReInHerit(Source: D3.1 National Surveys Report).
Preprints 161431 g001
Figure 2. Visitor Survey: reasons not using Mobile App.
Figure 2. Visitor Survey: reasons not using Mobile App.
Preprints 161431 g002
Figure 3. Visitor Survey: Helpful Digital Tools.
Figure 3. Visitor Survey: Helpful Digital Tools.
Preprints 161431 g003
Figure 4. Visitor Survey: Personal Smartphone vs Museum Tablet.
Figure 4. Visitor Survey: Personal Smartphone vs Museum Tablet.
Preprints 161431 g004
Figure 5. Visitor Survey: Interest on Interaction.
Figure 5. Visitor Survey: Interest on Interaction.
Preprints 161431 g005
Figure 6. Visitor Survey: Digital Game Motivations.
Figure 6. Visitor Survey: Digital Game Motivations.
Preprints 161431 g006
Figure 7. Visitor Survey: Interest on Mobile Apps.
Figure 7. Visitor Survey: Interest on Mobile Apps.
Preprints 161431 g007
Figure 8. Survey Professionals: Technological Services used.
Figure 8. Survey Professionals: Technological Services used.
Preprints 161431 g008
Figure 9. Survey Professionals: Human Resources.
Figure 9. Survey Professionals: Human Resources.
Preprints 161431 g009
Figure 10. ReInHerit Toolkit Strategy (Source: ReInHerit Digital Hub).
Figure 10. ReInHerit Toolkit Strategy (Source: ReInHerit Digital Hub).
Preprints 161431 g010
Figure 11. Strike a Pose matching: The user pose is detected by the system which checks the correspondence with the artwork’s pose. (Pictured are MB and PM authors of this paper. Source: ReInHerit Digital Hub).
Figure 11. Strike a Pose matching: The user pose is detected by the system which checks the correspondence with the artwork’s pose. (Pictured are MB and PM authors of this paper. Source: ReInHerit Digital Hub).
Preprints 161431 g011
Figure 12. Processing flow for pose detection and matching. The reference image and the camera frame input undergo pose landmark detection. If the joints’ orientations match, an output video is generated; otherwise, the flow loops back for further comparison.
Figure 12. Processing flow for pose detection and matching. The reference image and the camera frame input undergo pose landmark detection. If the joints’ orientations match, an output video is generated; otherwise, the flow loops back for further comparison.
Preprints 161431 g012
Figure 13. Strike-A-Pose testing at “ACM Multimedia 2022", Lisbon, Portugal and "Research Fair", Arcada University of Applied Sciences, Finland 2023.
Figure 13. Strike-A-Pose testing at “ACM Multimedia 2022", Lisbon, Portugal and "Research Fair", Arcada University of Applied Sciences, Finland 2023.
Preprints 161431 g013
Figure 14. Face-Fit: Match the expression, The ghost image helps the user focus on the task without distracting from the painting and thus the game. (Pictured is the result of the interaction of FP author of this article. Source: ReInHerit Digital Hub).
Figure 14. Face-Fit: Match the expression, The ghost image helps the user focus on the task without distracting from the painting and thus the game. (Pictured is the result of the interaction of FP author of this article. Source: ReInHerit Digital Hub).
Preprints 161431 g014
Figure 15. Processing flow for facial landmark matching and face swapping. The reference image and the camera frame input undergo facial landmark detection. If the landmarks and orientations match, a face swap is performed to generate the output image.
Figure 15. Processing flow for facial landmark matching and face swapping. The reference image and the camera frame input undergo facial landmark detection. If the landmarks and orientations match, a face swap is performed to generate the output image.
Preprints 161431 g015
Figure 16. Face-Fit testing during Workshops at Museo Capitolare Diocesano "CREA Cultura Festival 2024", Foligno (IT), "Humanities Festival 2023", Macerata (IT) GIARA Gipsoteca di Arte Antica, 2024 Pisa (IT).
Figure 16. Face-Fit testing during Workshops at Museo Capitolare Diocesano "CREA Cultura Festival 2024", Foligno (IT), "Humanities Festival 2023", Macerata (IT) GIARA Gipsoteca di Arte Antica, 2024 Pisa (IT).
Preprints 161431 g016
Figure 17. Example of the web interface Smart retrieval - text-to-image search and image-to-image search.
Figure 17. Example of the web interface Smart retrieval - text-to-image search and image-to-image search.
Preprints 161431 g017
Figure 18. Smart Lens App for smartphone. 1) Home. 2) The user frame the artwork with the device camera. Bounding boxes of the details and image details previews are shown. 3) After the selection of a detail the user can hace insights about it and listen to the audioguide.
Figure 18. Smart Lens App for smartphone. 1) Home. 2) The user frame the artwork with the device camera. Bounding boxes of the details and image details previews are shown. 3) After the selection of a detail the user can hace insights about it and listen to the audioguide.
Preprints 161431 g018
Figure 19. Examples of use of the Smart Lens app during ReThinking Exhibition at Cycladic Museum, Athens, Greece.
Figure 19. Examples of use of the Smart Lens app during ReThinking Exhibition at Cycladic Museum, Athens, Greece.
Preprints 161431 g019
Figure 20. VIOLA Multimedia Chatbot answering some questions about Cycladic Figurine (Museum of Cycladic Art) - artwork included in the VIOLA Gallery
Figure 20. VIOLA Multimedia Chatbot answering some questions about Cycladic Figurine (Museum of Cycladic Art) - artwork included in the VIOLA Gallery
Preprints 161431 g020
Figure 21. Testing Multimedia Chatbot VIOLA at GiArA Gipsoteca di Arte Antica, Pisa IT.
Figure 21. Testing Multimedia Chatbot VIOLA at GiArA Gipsoteca di Arte Antica, Pisa IT.
Preprints 161431 g021
Figure 22. VIOLA: Participatory Process to generate “user-oriented" and quality contents.
Figure 22. VIOLA: Participatory Process to generate “user-oriented" and quality contents.
Preprints 161431 g022
Figure 23. Video restorer network architecture. Frames of the video to be restored on the left, restored frames on the right.
Figure 23. Video restorer network architecture. Frames of the video to be restored on the left, restored frames on the right.
Preprints 161431 g023
Figure 24. ReInHerIt Demo web app Old Photo’s Restorer.
Figure 24. ReInHerIt Demo web app Old Photo’s Restorer.
Preprints 161431 g024
Table 1. Key Topics of the People-Centered Approach in the ReInHerit Project.
Table 1. Key Topics of the People-Centered Approach in the ReInHerit Project.
Topic Description
Playful Experience AI and CV tools are applied to foster learning and build a deeper connection between visitors and artworks. Interactions and gamified experiences are designed to trigger emotion, encourage creativity, and support participatory engagement.
New Audience Younger audiences, who tend to be more familiar with digital technologies, are a key target of the ReInherit Toolkit, which aims to increase their active participation in museum experiences.
Sustainability Smaller museums often lack the resources and skills to adopt digital tools, making training and capacity-building crucial for effective heritage innovation.
Bottom-Up The development process follows a community-driven model, where local participants are actively involved through workshops and hackathons. This inclusive method ensures that the tools reflect the needs and insights of the users themselves.
Co-Creation The innovative goal is to offer not just a tool as a final product, but a collaborative development process that fosters mediation between different disciplinary sectors.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.
Prerpints.org logo

Preprints.org is a free preprint server supported by MDPI in Basel, Switzerland.

Subscribe

Disclaimer

Terms of Use

Privacy Policy

Privacy Settings

© 2025 MDPI (Basel, Switzerland) unless otherwise stated