Virtual Tour Models as an Efficient Counterpart to BIMs in Spatial Immersion for IoT Distant Control

1. Introduction

The Architecture Engineering and Construction (AEC) industry increasingly implement Building Information Modelling (BIM) into the building phases of conception, construction, and renovation [1,2,3]. In turn, for the exploitation phase, asset owners become more and more interested in obtaining from AEC’s resulting Building Information Models (BIMs) the necessary information for Facility Management (FM) and Computerized Maintenance Management Systems (CMMS) [4,5]. Different Levels of Detail including Geometry (LOG) and Information aspects (LOIN) are needed for each phase, usually making necessary for AEC professionals to simplify their BIMs at submission. For instance, by exporting a room-based volumetric version including relevant room by room information [6].

Such models can be integrated into specialized platforms assisting facility management for room allocation, equipment availability, maintenance scheduling, and security aspects. In the case of CMMS, such models, or even more complex ones, can also be exploited for remote monitoring and control via Internet of Things (IoT) and other technologies, forming what is usually known as digital twins [7] . However, precisely because these models remain a digital representation of reality, they lack certain layers of detail including the visual information that a picture, a web cam or a video may provide at certain point. Moreover, certain IoT-based distant control and monitoring, becomes more accessible to non-trained users if they navigate through photorealistic environments than if they need to immerge into abstract digital models [8]. This article explores the potential of Virtual tours, defined as image-based geo-localized 3D reconstructions of real spaces and buildings, as a complement to BIMs in IoT-based FM. The main question it poses is : How can Virtual tours complement and connect to BIMs within a life cycle perspective? (Figure 1).

2. Methodology

The article crosses literature and empirical knowledge. We explore and analyze state of the art literature on BIM, virtual tours and IoT implementation into FM cases and contrast them to the results of a case study of our own.

This methodology is structured upon the following stages:

• Literature Classification: a thorough understanding of the current state of the art in BIM, virtual tours and IoT implementation, helping identifying gaps in the existing literature and conceptual functional bricks articulating a fully linked chain BIMs/ CMMS /IoT control/LCA-BIMs. It also provides the framework for the case study.
• Pilot and Demonstrator or Case Study: setting up of a case study based on the literature review and in relation to a BIMs-based future building;
• Installation of monitoring, IoT control devices and data base management: collecting real-time data using the case study. Validation of findings from the literature review, providing empirical evidence to support conclusions.
• Execution of case’s BIMs and virtual tour models: including the creation of instance semantic elements into a virtual tour model of the case study.
• Development of links: establishment of connections between different elements of the system in two phases: 1) IoT data base elements and Virtual tour instance sematic elements; 2) BIMs elements and virtual tour.

The case study is the Controlled Environments for Living Lab Studies [9] (CELLS), a twin-room HVAC autonomous pavilion allowing to control and monitor indoor comfort conditions, building management systems and multiple levels of automation. For our research, CELLS operates as an “incubator” to benchmark and validate systems that will be put in function on the future Smart Living Lab building. This building is a five-story, 5,000 m2 wooden structure designed for modular operations. Key features include the ability to dismantle façade modules for testing new envelope materials, the installation of sensors and IoT controls across the building envelope, the capability for independent HVAC system control at the smallest office base, and flexible partitions that accommodate a range of space layouts. This case is unique due to its requirement for flexible monitoring and remote control processes (Figure 2).

3. Results

Literature review: the literature review reveals that IoT technology is increasingly becoming prominent at the FM level, integrating monitoring devices that not only track and control building performance as Building Management Systems (BMS) currently do [10,11,12,13], but also introduce energy-saving and space optimization methods based on user behavior [14,15]. It also demonstrates how photogrammetry, Virtual Tours, Augmented Reality (AR) and virtual reality (VR) provide immersive and intuitive tools, allowing users without specific training to virtually navigate spaces and access information linked to connected objects [16,17,18,19]. Finally, the examples provided illustrate how these virtual photo-based models can be linked to geometry-based digital models, including basic attached information such as object properties or 2D plans.

Case study: the literature review also reveals a niche for FM exploration of virtual tours in connection to IoT and, eventually, BIMs. The case study of the CELLS provides a unique perspective on this integration, offering a practical example of how these technologies can be implemented in real-world settings. The constructive characteristics of this pavilion, including the building envelop, the double-gazing windows or the HVAC and electric systems, provide optimal and reliable conditions for the operational and measurements involved in the experiment.

IoT monitoring: The CELLS is equipped with a number of IoT elements including sensors (temperature -interior and exterior-, wind speed, rain, inlet power consumption, lighting fixtures active power, lighting intensity, etc.), as well as controllers (blinds (up/down), lighting-switches (on/off), among others). For demonstrative purposes of this research, two specific system elements have been fully integrated into the IoT-Virtual Tour-BIMs connection: lights-controlling (on/off) and historic temperature inquiry (interior and exterior). Distant control and inquiring of these elements, using an html protocol and interface is successfully set.

BIMs and virtual tour: a digital replica of the pavilion is modeled using a BIM software and according to a LOIN 400 standard, which represents more than what a typical BIM project may require for the construction of a standard building [6,20,21]. This model is successfully integrated into a VR environment and, as part of a different research, linked to the IoT connection. In turn, a Virtual Tour is set, combining aerial and ground photographs. This process encompasses a wide range of tasks from standard procedures, such as drone or 360° image capture and processing, to more tailor-made features. That is the case of JSON configuration files setting chronological versions of assets (different periods for singular space) or spotting named associated locations (referencing a given space or asset) (Figure 3).

Through a simple click, related information and available controls are prompted. Figure 4 illustrates the case of a temperature reader displaying the historic values of temperature of the specific room in relation to global external temperature. In the same way, a polygon can be linked to the specific room element defined in the BIMs (IfcSpace). In this case, the user will have access to all information attached to the elements belonging to this space, including windows, walls or floors, as well as properties directly attached to the room itself. The later may include room function, label, capacity, HVAC setting, etc. as well as attached files such 2D plans or catalogues.

4. Discussion

The integration of Building Information Modeling (BIM) and Virtual Tours in the Architecture, Engineering, and Construction (AEC) industry represents a promising development, especially within the context of Facility Management (FM) and Computerized Maintenance Management Systems (CMMS). The need for varying levels of detail in BIMs throughout different building phases, coupled with the potential of Virtual Tours to enhance BIMs in Internet of Things (IoT)-based FM, are significant areas of focus.

Our case study of the CELLS provides a unique perspective on this integration. The facility's design, which allows for flexible monitoring and remote controlremote-control processes, serves as a practical example of how these technologies can be implemented in real-world settings. The use of IoT devices for real-time data collection, along with the creation of instance elements in a virtual tour model of the case study, further illustrates the potential of this integration.

This research aligns with current trends in the field, focusing on FM and CMMS in relation to BIM and IoT [22,23] . However, our exploration of the potential of Virtual Tours to enhance BIMs in IoT-based FM represents a relatively new area of research, providing a solid foundation for further development.

The methodology we followed yielded successful results for the specific case study. Image capturing and processing remain reasonable for the service a Virtual Tour may provide for FM purposes. The definition of linked-elements, if well defined in advance by FM requirements, presupposes a more challenging but manageable task. Navigating a Virtual Tour is smooth and user-friendly. Importantly, the case shows that Virtual Tours, unlike BIMs, are simpler to keep updated. Given the ease of updating Virtual Tours, they offer superior operational efficiency for Facility Management tasks related to daily activities, requiring no significant alterations to the asset infrastructure. In contrast, Building Information Models remain static until a substantial change necessitates an update (Figure 1).

Among the challenges encountered are skill-related, maintenance, security, and scale. The necessary skills to set up and maintain an updated Virtual Tour may become a practical obstacle, especially in relation to image treatment or the addition of new elements linked to the database. However, any digital-based FM should include a dedicated person with these skills. Security is a fundamental aspect to consider as IoT systems expose vulnerabilities to cyberattacks and other unwanted third-party control. Remote control also magnifies risks related to automation in real-world scenarios. For example, alert systems might fail to prevent an automatically closing window from colliding with a misplaced object. To mitigate such accidents, alarm systems for human-operated remote control should be reinforced. Nevertheless, the most substantial challenge is related to scale. The research presented focuses on a small building with few elements and few rooms to capture and set. The next steps of our research involve using a more realistic building of at least 5,000 m². Nevertheless, the experiment presented, which interpolates results and expectations of a future building under construction, provides evidence, especially in relation to the links established among the different functional bricks connecting reality, BIMs, IoT elements, and virtual tour platforms (Figure 5).

5. Implementation and Reproducibility

Before delving into the conclusions and future prospects, we aim to provide readers with a concise overview of the key steps necessary to replicate our implementation. By doing so, we demonstrate the reproducibility of our development.

Step 1: Use case definition. This step consists on answering the questions who will utilize this development and how it will enhance their task performance. Our use case implies researchers who will employ the new building as a scientific tool. Precisely, each user group—professors, scientific collaborators, doctoral students, and the facility maintenance team—must have convenient access to information in a format that best suits their needs. For CELLS, which serves both teaching and research purposes and caters to individuals who may not always be on-site, an essential requirement is an online solution accessible remotely.

Step 2: Development strategy. It involves assessing whether existing open-source solutions can adequately address a specific use case or if an in-house solution is necessary. In our current scenario, we embarked on testing various virtual tour software options, including 3DVista VT Pro [24]. While this software demonstrated certain capabilities, it fell short in meeting our precise requirements: a) Authentication Component: Our use case necessitates an authentication component that aligns with the IT security standards of the involved institutions (EPFL, HEIAFR, and UNIFR); b) Flexibility for Marker Integration: We sought the ability to seamlessly integrate markers (such as polygons or geometric elements) into 360° images, allowing connections to IoT elements; c) Scalability: our solution needed to accommodate both detailed construction aspects (at a small scale) and broader neighborhood dimensions (at a larger scale); d) Chronological navigation: the virtual model needs to be evolutionary over time in relation to building life cycle analysis and renovation updates. Therefore, we needed to be able to add different periods of time for a given location, which was not offered by most software, and e) JavaScript integration: we needed flexibility to integrate JavaScript scripts, whereas 3D Vista was very limited and the documentation imperfect.

Step 3: Capturing devices. This step complements the development strategy by integrating the capturing devices to be used. According to the use case, we needed to reconstruct the building assets at the district level (outdoor), as well as detailed information at the room level (indoor). We opted to use a specific device for each function. For the district level, we used a DJI Mini 3 Pro drone [25], which allows 360° photo captures at various heights; at 2, 30 and 70 meters in our case, and zenithal views. For room capturing, as it is not possible to fly inside with a drone, we chose the Ricoh Theta Z1 [26], a 360° camera, which is easy to use, linked to a smartphone (See Figure 3).

Step 4: Virtual tour technical development. (See in Figure 3: Image acquisition and image files processing). Having defined use case and development strategy, we proceed to separate the development in technical modules. For this case, 6 modules were defined: 1) Capturing protocol: the referential definition of capturing points, including nodes of reference and capturing frequency; 2) Storage sizing. According to image resolution and capturing frequency we defined the size and type of storage device. The images required for the virtual tour are stored on the same server as the web pages. 3) Image chronological sorting. The configuration file indicates the tour360 server that a new date has been added, including the name of the associated locations. Without this information, the server cannot know which new images paths are available. Consequently, the date picker would not offer the new date for a given location. The configuration file plays a crucial role in ensuring that the tour360 server can effectively manage and display new dates associated with specific locations. 4) Image polar orientation. A script defining a reference point to establish a singular orientation for each capture point. This module is important for making virtual tour navigation more intuitive, preventing unwanted changes in observation direction when moving from one point to another. This also provides users with a compass and the orientation of the image in relation to north. 5) Lag prevention: the virtual tour works with two panorama layers of 128 tiles: low and high resolution. This is to optimize loading, preventing lagging during navigation. The panorama is rarely, if ever, reconstructed in full. When a user wants to view a new panorama, their browser initially requests the blurred (lighter) version of the entire panorama, which loads much faster. The sharper tiles are then loaded shortly after, depending on where the user is looking at. This process is completed with the JavaScript library PhotoSphereViewer [27] which takes care of displaying the panorama. It places the tiles in the right position and updates them in real time as the user browses the site. 6) Access security: Images are stored and accessed by the tour server behind an authentication protocol. This allows for control of access at several levels, granting different users varying rights. For example, it permits accredited members of the public to navigate exteriors and public spaces, specialized scientists to access any room, and researchers dealing with confidential data to access real-time information. Upon accessing the tour, a login interface is prompted, and certain areas of the tour may or may not be accessible based on the user’s credentials.

Step 5: Integration of IoT. (See in Figure 3: Polygon positioning). This step involves creating a reference geometry (typically a polygon or boundary representation) superimposed on a visible object within the virtual tour. For example, consider the silhouette of a temperature sensor. Once defined, the polygon becomes a georeferenced object within the virtual tour, allowing it to connect with other objects in a database. In our example, when an accredited user clicks on the temperature polygon, a graphic displaying the history of stored temperatures is prompted (see Figure 4). While this step may require manual effort to add polygons, they will remain in the correct position even if images from different time periods are added. Of course, if an element disappears after renovation, the polygon must be removed manually.

Step 6: Integration of BIMs (See Figure 5). At the current phase, we are developing the connection between the virtual tour and specific elements from the Building Information Models (BIMs). The underlying principle is similar to the previous one. After creating a polygon—for example, defining the floor border of a room—it can be linked to that room in the BIMs. Consequently, all relevant information associated with the room (such as its use, materials, and current CO2 levels) becomes accessible. This information is presented in a selectable text list, allowing users to extract and utilize it for various purposes. The BIM-connector polygon also includes a hyperlink, enabling users to access the Common Data Environment (CDE). In the CDE, they can visualize and navigate the actual 3D model of the building and access additional associated information. Since most CDEs are cloud-based, the connection between servers occurs via the internet. For example, it can be a get or post request.

6. Conclusions and Outlook

This research demonstrates the potential of integrating Internet of Things (IoT) technology, Building Information Modeling (BIM), and Virtual Tours in the Architecture, Engineering, and Construction (AEC) industry, specifically within Facility Management (FM) and Computerized Maintenance Management Systems (CMMS). The case study conducted on the CELLS infrastructure serves as a practical example, highlighting how these technologies can be effectively implemented in real-world scenarios. Notably, successful integration of system elements, such as lights control and historical temperature inquiry, highlights their practical application. Additionally, the ability to store real-time data from IoT devices in a Building Database and access it through the Virtual Tour interface underscores the efficiency of this integration. Overall, this study lays a solid foundation for future investigations in this field.

The findings of this applied research have the potential to significantly impact industry practices across several dimensions. Firstly, by integrating IoT technology, BIM, and Virtual Tours, facility managers can enhance FM operations. Real-time data from IoT devices can be stored in a Building Database, enabling informed decision-making and predictive maintenance, leading to cost savings and improved service delivery. Secondly, the successful integration of IoT controllers and sensors demonstrates the practicality of these technologies, benefiting CMMS. Quicker response times and better maintenance planning become possible through real-time data access via Virtual Tours. Thirdly, while this study lays a solid foundation, further research is necessary to explore larger and more complex buildings. Finally, the integration between Virtual Tours and BIMs opens up new research avenues aimed at automating or, at the very least, semi-automating the generation of polygons. This can be achieved by utilizing elements (such as switches) directly extracted from BIMs. Industry professionals can build upon these findings to drive future innovation and refine existing practices, bridging the gap between technology and AEC industry standards.