An Efficient Process for the Management of Deterioration and Conservation of Architectural Heritage. The HBIM Project of the Duomo of Molfetta (Italy)

1. Introduction

Every form of sculptural and graphic expression, whether artistic or architectural, carries with it the principles of semiotics and semantics. Semiotics in architecture refers to the analysis of the symbolic and significant aspects of architecture. The discipline of architectural surveying, accompanied by a scientific and technical methodology, guides the scientific knowledge of the works, as well as their form and context [1]. The study of heritage refers to the past in its context of the inheritance received, and technology represents the contribution to a future to come. In this framework, 3D modeling aims to contain and reflect the uniqueness of geometry, considering the structural deformations and construction properties they acquire throughout their life cycle.

The geometry of Cultural Heritage (CH) is captured by technology procedures that have to do with engineering and geomatics, which have become a common practice, such as digital terrestrial photogrammetry (TDP), photogrammetry with unmanned aerial vehicles (UAVs), together with the photogrammetric process by Structure from Motion (SfM), and technology by Terrestral Laser Scanning (TLS) for 3D digital reconstruction [2]. The current research agenda recommends the need for an interpretation of the set of 3D points, with the aim of identifying and quantifying patterns of elements from the decomposition of the elements, looking for functionality, semantics and topology [3]. To help conserve and document CH, researchers are also implementing multiple prospecting techniques at once. For example, Costantino et al., 2023 [4] included the "Digital Single Lens Reflex" (DLSR) camera and TLS for indoor and outdoor studios, merging active and passive sensors. Rocha et al., 2020 [5] used TLS and TDP to achieve detailed and accurate representations; others perform comprehensive surveys of DSLRs, UAVs, and TLS cameras [6]; Alessandri et al., 2019 [7] performed rapid photogrammetric surveys using a Nikon D800E DSLR; and Di Stefano et al., 2019 [8] applied TDP and spherical photogrammetry (SP) for data representation and enrichment.

Moreover, artificial intelligence (AI) is playing an important role in the preservation and study of the PC with a view to automating processes; it is used in the conservation and restoration of architectural heritage in various ways, such as virtual reconstruction of historic buildings, image scanning and recognition, remote sensing, and degradation prediction for preventive conservation. In the field of remote sensing, major advances in image matching algorithms are changing the landscape of 3D architectural and archaeological reconstruction. In addition, with the arrival of new geomatic technology equipment, such as the personal mobile laser scanner (PLS), data capture in the measurements of architectural spaces is simplified [9]. And a new development in the integration and combination of TLS equipment with Light Detection and Ranging (LDR), LiDAR scanners on board drones, are one of the most cost-effective systems that can provide spatial data. Among the most prominent, the DJI Matrice 300 RTK Zenmuse L1 UAV, used in forestry studies [10], and the Leica BLK2GO that captures moving images [11]. Each of the aforementioned techniques is associated with the registration space that is intended to be obtained. But once the record has been obtained and the processing of that data is obtained, a set of points is obtained that is not classified. Therefore, the great challenge for the scientific community is to have procedures for classifying complex elements in architecture and archaeology. This classification is valid for processing data in current BIM methodologies that allow interdisciplinary collaboration and the combination of compliance verification and analysis tools. In this sense, segmentation is a process of point cloud classification for the identification of homogeneous areas or entities that have similar characteristics, let's call them point cloud attributes. But currently one of the challenges is to manage and process all that data so that it has basic functionality, not only in 3D digital models, but also in the applicability of the Historic Building Information Model (HBIM) environment [9].

2. Literature Review

2.1. Remote Sensing in Heritage

The scientific community is very intensively involved in the efficient management of the data provided by remote sensing, photogrammetric and scanning systems, since the 3D information provided through large numbers of points (PCs) is essential to document and analyze the state of conservation of architectural heritage.

PCs allow an agile and detailed assessment of the structural condition of the historic building, reducing laborious "on-site" inspections [12], and provide precise information on the texture and geometry of the surfaces, scanned or photographed, which can then be analyzed to identify damage to the structures [13].

Laser scanning and photogrammetry are appropriate to cover large heritage sectors, reducing the time of auscultation and taking measurements, which make them very efficient technologies in places that are difficult to access [14]. The post-processed information provided (PCs) facilitates the documentation and analysis of sites located in very heterogeneous environmental conditions and in places inaccessible by more traditional methods, which helps in inspection and monitoring tasks [15]. Both the acquisition and analysis of collected data prove to be faster and less expensive compared to traditional inspection and measurement methods [16]. Another potential is the capture of minimal details and the exact identification of deteriorated surfaces, such as cracks, fractures and loss of material, thus contributing to an adequate analysis of the structural condition [17] and the state of the coatings associated with the heritage elements.

2.2. Detection of Deterioration

Automating the process of detection and semantic segmentation of deteriorated surfaces of heritage buildings based on the study and management of PCs, acquired through photogrammetric and TLS techniques, and their implementation in the modeling of heritage building information (HBIM), is nowadays a priority for a large number of researchers. The aim is to improve manual and time-consuming procedures, replacing them with efficient and precise automatic processes that support architectural heritage conservation efforts. Unsupervised machine learning and deep learning methods, based on dynamic convolutional neural networks, usually aid the most autonomous segmentation and classification. The process involves the accurate recognition of architectural elements and accelerate the modeling of historic buildings for the development of the HBIM project [18]. Machine learning methods and clustering techniques to automatically segment PCs can also help detect alterations in historical surfaces, such as color variations and biological colonization, based on color information from PCs [19]. The main objective of these methods is to automatically detect objects and data using specific algorithms to evaluate and monitor the condition of buildings and, thus, identify deteriorated surfaces. In parallel, these advanced techniques contribute to an accurate and rapid assessment of structural degradations, facilitating continuous monitoring and the development of effective conservation strategies [20]. Another benefit of PC segmentation is the improved performance of 3D data management platforms by reducing calculation time and increasing model accuracy [21]. In the specialized literature, various automatic systems have been developed for the detection of cracks and degradations. Muñoz-Pandiella [22] uses the Gaussian Filter and the Random Sample Consensus (RANSAC) algorithm for the detection of the initial shape of the cracks. Peng y Nam [23] have created a system that identifies cracks by using Gaussian Filter, Intensity Gradient, and pattern detection. While Sánchez & Quirós [24] propose a semi-automatic detection method based on an image approach to identify degradations in facades.

Classification methods that are not aided by machine learning are also very effective [25] and therefore do not rely on extensive training datasets. They are mainly based on image processing techniques and detected geometric analysis. These methods have the great advantage of being flexible and efficient in terms of time, cost and implementation, reducing the risk of overfitting [26]. For all these reasons, bearing in mind that the final objective of the research is the management of data intrinsic to the semantic and parametric elements of an HBIM model, the proposed methodological process is aided by a semi-automatic semantic segmentation, although more accessible and efficient. The operator's supervision will always ensure the accurate detection of deterioration in historic buildings.

2.3. Data Classification

Most segmentation studies use software external to the software used for processing. In these cases, they apply the RANSAC-based plane adjustment algorithm to add an additional layer of abstraction [3]. RANSAC is an iterative algorithm used to estimate the parameters of a mathematical model from a dataset containing a significant proportion of outliers. It was introduced by Fischler and Bolles in 1981 [27] and it has become a standard technique in computer vision, image processing, and point cloud. It is based on the idea of repeatedly selecting a random subset of the input data, fitting a model to this subset, and then evaluating how the model fits the entire dataset. RANSAC's key strength is its ability to cope with a high level of outliers in input data. Unlike traditional tuning methods that can be affected by outliers, this algorithm ignores outliers and focuses on most of the data that fits the model well [28]. Examples are the research that develops a classification based on model adjustments through the determination of geometric primitives, using the RANSAC algorithm and the Hough transform as classification processes. Dong et al., 2018 [29] conducted a segmentation classification based on four essential points. Methods based on the growth of the region, which extract 3D planes by progressively merging adjacent points or voxels with similar characteristics [30] among others. Methods based on model fitting, which are most commonly used for plane segmentation (RANSAC) and Hough transform. These algorithms have successfully demonstrated 2D and 3D shapes. Later there are methods based on function clustering, which characterize point clouds in primitives based on certain pre-calculated surface properties [31]. To this large set of studies on segmentation, the usefulness and functionality of classification procedures must be proposed. Many of the studies are dedicated to the classification of urban regions, others to interior spaces, selection of objects, elimination of noise in the point cloud, classification of urban furniture, or also the creation of digital twins in the field of CH and morphological semantic segmentation [32].

Some research offers more advanced segmentations in architectural heritage as they implement machine learning in parametric BIM modeling [33]. The process consists of the transition, still semi-automated, from unstructured TLS data to the semantically rich representation of buildings and historical H-BIM assets, applied mainly to recurrent architectural typologies that are supported by construction rules and/or building proportions, as is the case of a Renaissance loggia. The objective is to achieve an effective, precise and productive interdisciplinary process that can be shared by architects, engineers, conservators and restorers to update, retrieve and archive the available heritage information based on semantically enriched parametric models. For this reason, for the HBIM project of the Duomo di Molfetta, intelligent process automation is also sought, using, in this case, the classification algorithms implemented in various software tools [34] with the aim of simplifying the frequent tasks of management, point cloud classification and reverse engineering, thus reducing the times in the development of architectural projects, engineering, construction and conservation (AECO).

2.4. The Scan-to-HBIM Process

If the above is analyzed, it is essential to combine the various techniques to achieve an efficient methodological process that takes advantage of each prospecting method to achieve the best conservation and documentation requirements. This is where the Building Information Modeling (BIM) methodology provides a fundamental help to integrate the various conservation and documentation data (graphic and non-graphic) into a comprehensive historical construction model. The identification of geometric characterization in a Heritage BIM (HBIM) project is crucial for operators who are morphologically modeling architectural elements. The process initiated in Massive Data Capture Systems (MDCS) that involves the construction of 3D models with parametric objects, called Scan-to-BIM, presents great challenges due to the complexity of the historical architecture, always seeking to solve the errors of the BIM model according to a scanned work (Tzedaki and Tamara, 2013) [35]. This is posed as the challenge of the construction of the ideal model and, on the other hand, of the deformations that arise on site, admissible by the technical teams of execution of the project. In architectural heritage, this difference between the ideal model and the real model becomes more and more pronounced as time goes by in the life cycle of old buildings, due to the deterioration of the elements. Therefore, the great challenge and scientific discussion focuses on the automatic position of capturing the structural deformations that can occur in an HBIM project. There are several scientific reviews of the combination of remote capture of information, such as 3D laser scanning, with BIM applications [36,37], where the dilemma of finding operators who can solve complex geometries is raised. The problem also arises that the manual process of constructing the geometry of a historic building from the point cloud is a process that requires a considerable amount of time, since it is necessary to structure the point cloud in parts and to make an efficient classification of the data [38] and eliminate those that are unnecessary, usually according to the criteria of the rehabilitator-operator. Another aspect that should be highlighted is the amount of information that is managed, since the point cloud has an important weight where BIM platforms must overcome and manage efficiently.

For all these reasons, the challenge of this research is to make a classification and semantic segmentation of the point cloud by architectural/structural typologies, facilitating the modeling and management of the HBIM project in the conservation process; incorporating, in addition, the deformations and deterioration that the structural elements of the Duomo di Molfetta have suffered, both in their real geometric facet and in their analytical data. These peculiarities of a historic building, although they have been classified and analysed under the environment of a specialised engineering software (Cyclone 3DR Leica), are perfectly linked to the HBIM project (Archicad Graphisoft^®).

3. Methodological Process

All this study involves a deep knowledge of the historical architecture by applying a semantic segmentation of the point cloud and its processing in a common data environment managed from the HBIM project. The methodology is supported by a semi-automatic process assisted by algorithms that allows classifying and segmenting the unstructured information provided by the TLS scanner to support the HBIM project. The Duomo di San Corrado, Molfetta (Italy), an example of Apulian Romanesque architecture and an emblematic representation of Heritage, has been chosen for the study. The building was scanned in its entirety inside and main façade, while photogrammetry covered the survey of the other facades.

The paper presents the results of a semantic segmentation method based on classification algorithms that are implemented in point cloud processing software, such as Cyclone 3DR from Leica Geosystems. A direct workflow is created with the BIM platform so that the fractions of points, already identified, are introduced into the HBIM project for classification, according to architectural typologies and construction systems. This fragmentation makes BIM parametric modeling more flexible. To cover the conservation phases, specific properties are created for the identification and analysis of the deterioration detected in the building structures. Subsequently, a marking and labelling of the deterioration is carried out, which is consummated in the risk assessment.

The methodological process reinforces the interoperability of the HBIM project data with conservation and restoration specialists at all times. A comparative analysis of some defects of the Duomo columns is carried out, comparing the TLS information with the theoretical parametric object. Finally, the HBIM project incorporates the deterioration located in the structures, associated with the real geometry of the parametric elements of the model, whether in the form of groups of points segmented by categories or as 3D mesh surfaces, all of them identified and classified. They have also been enriched with the technical reports obtained from the study of pathologies (Figure 1).

3.1. Case Study

For the experimental study, a survey of the Duomo of Molfetta (Italy) is carried out using terrestrial laser scanning techniques. Built between 1150 and the end of the 13th century, the building is a unique example of Romanesque-Pugliese architecture (Figure 2. (a). With respect to the spatial configuration of its construction, it has an asymmetrical basilica floor plan, divided into three naves by cruciform columns. The central nave has three domes, supported on a hexagonal drum (Figure 2. (b). Of the large volumes, the roofs that cover the aforementioned vaults and the two bell towers that flank the main façade stand out.

3.2. Architectural Survey Process

The interior of the Duomo di Molfetta is scanned by a CAM2 FARO Focus 3D 120 Ground Laser (TLS) scanner, with an accuracy of ± 2 mm and a range of 0.6 m at 120 meters, a registration speed of 976,000 points/second and a field of view from the vertical and horizontal plane of 305º to 260º. The TLS survey was carried out by researchers from the Laboratory of Architectural and Urban Modelling (MAULab), Politecnico di Bari (Italy), and 56 positions were necessary for the complete scan of the interior of the Duomo, which includes the three main naves, the perimeter chapels, the presbytery and the east-facing altar. The west-facing façade was the only part of the exterior scanned, which has served to provide metric data and scale the photogrammetry applied to the exterior of the building, adequately coupling the point clouds of both processes and avoiding decompensation errors. Exterior capture is carried out by Digital Single Lens Reflex (DLSR) short-range photogrammetry. An 18-megapixel Canon E650D SLR camera is used, with a CMOS (APS-C 14 × 22.3 mm ²) and optical image stabilizer. The photogrammetric process by Structure from Motion (SfM) is executed in the Agisoft Metashape software (Figure 3.(a), allowing to obtain portions of walls in textured 3D meshes that will then be incorporated into the HBIM project (Figure 3.(c)). The meshes are previously optimized, reducing the number of triangles in Cyclone 3DR (Figure 3.(b)).

For the global post-processing of the data, Leica Geosystems' Cyclone Register 360 is used to register the TLS scans and, subsequently, the Cyclone 3DR software for the coupling and management of the different point clouds (TLS & SfM), as well as for the semi-automatic semantic segmentation processes using analysis and classification algorithms. The workflow developed in Cyclone 3DR has been very effective for a limitation and effective visualization of the large point cloud, using 3D section boxes to focus on areas with complex geometries or difficult to access. This procedure facilitates the correct delimitation of the architectural geometries and the subsequent precise segmentation of decorative moldings of free forms, cornices and architectural finials, with certain complexities at the intersection of the structural systems: meeting of walls with pilasters, arches of discharges under the barrel vaults and intersections of the hemispherical domes with the central vault.

The HBIM project is based on Graphisoft^®'s ArchiCAD platform, where all the structured information from TLS and SfM processing is managed. Initially, all portions of the original file (supplied in rcp format) are previously managed from Recap Autodesk^® to proceed with its appropriate orientation, georeferencing and sectorization. The global TLS point cloud is subdivided into four fractions before being incorporated into the HBIM project, and it is not necessary to reduce the number of points from post-processing (limit: 250 million points), before being converted into parametric objects. Next, all the datasets (e57 format) are exported to the ArchiCAD BIM environment. It is important to underline the importance of always guaranteeing the best visualization and operability of the data in the HBIM project, so the datasets have been managed in different layers depending on their location or category once segmented (columns, walls, vaults, etc.), which has also allowed a better performance of the computer's graphics card.

3.3. Data Organization and Classification

Data interoperability is one of the pillars of the BIM methodology workflow, so the information generated about the HBIM project has to be coordinated, structured and classified so that all the agents and actors who collaborate can transfer and share data. A BIM project must incorporate a classification system according to the requirements of the client or body responsible for its management. In the same way, one of the objectives of using classification in a Heritage BIM project is to provide the heritage sector with standardized terminology and semantics, which will allow the model to be structured with known bases to be shared by all collaborators. In addition, in order to follow the "open BIM" interoperability standards, the project must be structured in IFC format, which will contain an appropriate and consensual semantic classification. In this way, anyone who is outside the scope of being a BIM operator can visualize the model and be able to manage and identify independent elements.

Thinking about a system of structural elements of its own, and given the similarity of the classification with the work of Moyano et al. [39], a Column-type element can be classified following an order based on its architectural composition, where all the parts are identified. Classification allows the model to be structured with standardized databases to define the attributes of each element or subelements. However, before applying segmentation to a construction element or system, it is important to properly structure an HBIM project, such as a cataloguing process, in which semantic data is managed and included and organized according to a recognized classification.

4. Results

The results obtained from the applied methodology will be presented, a process that began with the planning of the TLS and photogrammetry surveys, going through the semantic segmentation of the relevant building typologies of the Duomo di Molfetta until reaching the semantically enriched parametric model.

4.1. Application of the Semantic Segmentation Process

In order to structure an HBIM project on the basis of optimal georeferencing, it is important to establish a common origin of the project in the different software used in the collaborative process, and thus inter-operationalize data efficiently. In the case of the Duomo di Molfetta, the procedure applied has allowed the geolocation conditions of all the elements to coincide, facilitating interoperability and workflow between software. The precise position of the elements in the project has been guaranteed at all times, covering new additions or subsequent substitutions in the process of semantic segmentation. Thus, the network of X,Y,Z axes in the BIM environment was adapted to the structural elements located in the point clouds, after they were inserted in the ArchiCAD HBIM project. The same X,Y,Z network system was transferred to the Cyclone 3DR environment, so that all the point cloud fragments originating from the segmentation process would go to the exact place with respect to the origin of the HBIM project (Figure 4).

4.2. Sectorization of Point Clouds in Construction Elements and Systems

Once the point cloud is georeferenced by a package fit, and the workflow system has been verified to be correct, the appropriate segmentation by structural element is carried out. For the case study, the "Scan-to-mesh-BIM" workflow has been carried out through an implementation of the Cyclone 3DR software [34]. Due to the complexity of the geometric shapes of the architecture examined, the segmentation process should be classified as semi-automatic, since the current automatic classification algorithms are not very efficient for architectural heritage, which has resulted in an eminent assistance of the human operator. The "box" tool has facilitated the sectorization of the point clouds, allowing the tasks of identifying points and cutting out delimiting surfaces until the characteristic geometry of the structural type is defined, as shown in Figure 4.(a)¡Error! No se encuentra el origen de la referencia.. In the case of some central pilasters, the level of development and knowledge (LOD-LOK) has been increased, becoming segmented into smaller units: base, shaft and capital (Figure 5). The semantic segmentation procedure has been extended to the entire interior of the Duomo di Molfetta, affecting the TLS point cloud (Figure 6.a). These portions have then been inserted and classified into construction typologies in the HBIM project: arches, vaults, columns, walls, pilasters and columns (Figure 6.b).

4.3. Point Cloud Classification

To reduce the time spent on manual semantic labeling tasks, the process is increasingly being automated with the implementation of segmentation algorithms that segregate geometric shapes of architectural and constructive interest. Cyclone 3DR (C3DR) implements automatic classification models of construction systems, trained with current typologies, and which have been tested for the case study. According to the results, it has not been very effective in adequately detecting the architectural typologies of the Apulian Romanesque. The algorithms for automatic detection and classification have only been effective when a sector has been previously delimited with a box (Figure 6.(a)). Therefore, manual pre-selection is still required to achieve valid results, calling the applied process semi-automatic. But other algorithms incorporated in C3DR classification tools have been used, such as the wall, floor and ceiling arrest, which is very effective for discriminating sets of points that are located on the same flat surface. For the case studied, it has been applied to columns that have masonry with a high level of wear (Figure 7).

The process starts with a previous segmentation of points for the columns and pilasters of the Duomo. At the beginning, the appropriate tolerance parameters are chosen so that the selection is effective and logical: an angular tolerance for walls, another tolerance for floors and ceilings and finally adjust the "Local normal smoothing" for the classification process, which will vary between sharp or smoothed extremes. Due to the irregularities of walls, floors and ceilings, the tolerance is brought to the maximum (10º) and the classification is activated for all the above types and others. The great irregularities in the walls, whether vertical or horizontal, are discarded within the group of planes such as wall (brown), floor (cyan) and roof (green). In the case of the column that rests on a pedestal, some upper points are classified as floor, although a large number incorporate them into the wall, in the case of those points identified as noise and that appear when the scanner's laser does not project well in the frontal plane due to the inclination of the horizontal plane. These elements are located at a higher height than the equipment, as is the case of the spine-pedestal, the capitals, the cornices or very high projecting strips. The tests carried out on the pilasters or columns have provided very significant data, since the finishes of the bases of columns and capitals are classified in others, due to the fact that their inclination is outside the range of the angular tolerance for walls or floors (10º). Due also to irregularities in the shaft of the column, many points have been classified as other "Others" (grey colour), including the carving or indentations of the capital decoration. Figure 8 shows the classification process of the Column-X4Y2, identifying the points of the classes walls (in brown), floors (in cyan) and ceilings (green).

If the operator deems it necessary to fine-tune the segmentation process, the process is repeated because the system allows it, although it may be the case that a greater number of points are automatically filtered to be classified as "others". Therefore, it is very important to apply the right tolerance. And if the results are not very logical, the applicability of such algorithms to the classification sought should be ruled out. Finally, for the classification process of the Column-X4Y2, one applies:

• Segmentation of the column into three fundamental parts: base, shaft and capital. The shaft contains many undulations due to wear and loss of material, so it is important for the algorithm to detect these parts (which differ from walls and floors) and apply only to deformations. In this way, the capital and the base would also be discarded as they are groups outside the wall and floor type.
• Choice of tolerance parameters for classification: a minimum floor angle of 1º (minimum slope) and an angular tolerance for the walls (maximum angle with nadir-zenith line) with a high angle of 9º (1ºmin-10ºmax) as the column has a backward slump of 0.744º (Figure 10.(a).
• The slider for normal local smoothing is applied. This slider allows you to increase or decrease the radius of the buffer in a cloud, to account for a larger portion of the cloud. An intermediate of 0.013 m is chosen.

The final results provide the following quantification of points: 1,742,768 - 89% (Paredes/Walls) and 208,711- 11% (Other) of the total analysed (1,951,479 points). Floors and ceilings are not included in the classification. As shown in Figure 9, a graphical analysis is performed in Cyclone 3DR filtering the points by classified typology. The points corresponding to the most pronounced deformations of the column (other) are evidenced with the gray color.

4.4. Analysis of Spinal Deformations

BIM platforms have the potential to compare complex forms of historical architecture with original primitives, comparisons that will determine in many cases and in the eyes of experts the transformations. And this is where the emphasis is placed in this research, since by exporting each of the segmented models as portions of the point cloud, the deformations can be compared on a millimeter scale. In the case of the column referenced to the X4Y2 axes, it has been possible to compare the collapse on both sides with the ideal plumb position. For the case study, a 3D model of the shaft element is developed, using the "pillar/column" tool of ArchiCAD and associating a constant theoretical profile throughout its height; the 3D element is finely adjusted to the collapses of the point cloud portion and also of the 3D mesh (derived from the segmented point cloud), since all of these are available within the HBIM project. The result of the comparative study shows a collapse of the column of -0.744º in the direction of the west façade (Figure 10.(a), and considerable deviations in -X, delimited in red in the horizontal section to +7.36 m (Figure 10.(b).

In turn, another comparison is made in the Cyclone 3DR (C3DR) software, taking advantage of the analysis of geometries when comparing models between types: i) 3D mesh with 3D mesh, ii) Point cloud with 3D mesh, iii) 3D BIM geometry with point cloud and iv) 3D mesh with BIM; and, in this way, to be able to extract accurate deviation reports from each of the planes. Based on this, the BIM model of the column is exported from ArchiCAD in IFC format and then incorporated into C3DR. Also, the point portions of the segmented columns are converted to C3DR 3D mesh. In the case of the Column-X4Y2, there are three 3D meshes that correspond to the base, shaft and capital.

In C3DR, once the two models have been chosen, the theoretical one adapted to the point cloud and the mesh generated by the software itself, a "best fit" is made by choosing the homologous points in both geometries. It must be said that, in this case, the common coordinate system is georeferenced to the same origin of the project, so when exporting the IFC object to the Cyclone environment, the two models are adjusted. From now on, the deformations can be analyzed, using different cross-sections (every fifteen centimeters in our case) along the entire shaft. Figure 11 shows the points taken at the base and at the crown of the shaft, which have served as a reference to adjust the theoretical and real geometries, and thus obtain an accurate report of millimetric deviations of the shaft.

In previous works, Moyano et al., 2021 [16] evaluated the quality of the 3D model compared to TLS data, employing Cloud Compare software and complemented with specific algorithms in the Dynamo environment for a porticoed structure. Now, in the new case, implementing the C3DR environment provides a great advantage of operability in the workflow between the parametric 3D model (HBIM project), using the standard IFC format, and the environment of a specific point cloud analysis and reverse engineering software.

Once the parametric shaft model has been reverted to the point cloud processing environment, C3DR's "mesh vs BIM comparison" is used to obtain deviation results. A graph of deviations from the reference plane of the theoretical model is generated, showing regions in color that vary from red (+) to blue (-) and that represent the positive (protruding) and negative (perforation) deformations respectively as can be seen in Figure 12. The color graph shows up to a deviation of 0.06 meters (maximum distance), giving maximum deviation and percentage values of +0.038 m (0.1%) and -0.053 m (0.2%).

The HBIM project must constitute a nucleus containing all the information generated in the processes. For this reason, the data related to the external analysis and control work have been incorporated, which in the case of the previous column exposed, will be the result of the report of its deteriorations, with the comparison between theoretical model and real model. Figure 13 incorporates the most significant sections of the Report after comparing the Point Clouds with the Theoretical Surface of the Column-X4Y2 Shaft. The data of the deviations (average, minimum, maximum) in each horizontal section given to the shaft are collected in the Table 1.

4.5. Parametric Modeling in the HBIM Project

The parametric modelling of the historic building is carried out in parallel with the process of analysing the deformations of the column. The HBIM project is supported from the beginning by the TLS and photogrammetry point clouds, initially based on the global clouds (inside and outside) and later by the fragmentation into characteristic elements of the architecture analyzed. The objective is to obtain a main typology model with a Level of Detail (LOD) 200 in its development, since the project contains a semantic segmentation of all the structural and architectural systems provided by the point clouds. To do this, the basic tools of ArchiCAD are used to model the general construction elements of the building, such as walls, floors, ceilings, roofs and openings, setting aside the typologies that incorporated irregular and complex surfaces to be later transformed into mesh surfaces, such as columns, pilasters, walls, vaults and domes. The detailed modeling process of the architectural elements is mainly satisfied with the Cyclone 3DR reverse engineering software for their subsequent precise insertion into the HBIM project. The photogrammetric process by SFM is developed in Metashape, although it only affects certain panels of exterior walls that were captured by DLSR. As for the types of doors and windows, the flexibility of Archicad Graphisoft's library of architectural objects, based on GDL programming, satisfactorily saves the parameterization and graphic representation of the characteristic geometries of the Romanesque of Puglia. Although not a priority, some of the decorative elements present in cornices, moldings and friezes are solved with the efficiency and flexibility of the "complex profile" and "morph" tools. Figure 14.(a) shows the Inner Point Cloud (TLS) reference to the Duomo parametric model in the HBIM project. It is important to note that all the fractions of the point cloud (segmented groups converted into gsm/lcf library elements) have gone through a process of identification and classification by functions and construction typologies. The resulting object is very valid because it shows the delimited geometry and the real texture, in the form of points, ready to be enriched by the semantic properties of an architectural type, structures and others typical of the conservation process. The quality of the representation will depend on the resolution taken and the conditions of the environment at the time of scanning: position of the equipment, difficulty in capturing top, back and corner areas, and the high intensity of the light or contrast in the place. In the case study, the quality has met the needs of auscultation and analysis,

Finally, the complete HBIM semantic model, fully identified and classified (Figure 14.(b), it has been exported with the "Exact Geometry Export" (Archicad) option and therefore interoperable in IFC format. Figure 14.(c) shows the model with the IFC data structure, using the BIMvision^® viewer to explore the properties of all its elements.

4.6. Heritage Conservation Work

In this section we present the entire workflow developed for the conservation of the Duomo di Molfetta, managed from the HBIM Project itself, focusing mainly on the deterioration of the historic building.

4.6.1. Management of Deterioration in the HBIM Project

For the management of conservation processes in the BIM environment, specific properties have been created for the identification and analysis of the degradation (decay) detected in the building structures. In the case of the Duomo di Molfetta, we have focused on the wear and fragmentation located in the structural elements such as columns and pilasters. It is intended that the methodology applied will reinforce the interoperability of the HBIM project data with conservation and restoration specialists. The properties have been grouped by categories: HISTORICAL INFORMATION, DECAY, DIAGNOSTIC MONITORING, RISK ASSESSMENT, ATTACHMENTS MANAGER, COST CONSERVATION. Once the parametric entities have been identified and classified, to help in the auscultation processes of the "Decay", the HBIM project has properties that are common in control and inspection activities: Bistering, Esfoliation, Lack, Detachment, Crack, .... In this way, the restoration technician can select the given 3D element and have in its editable definitions ("Classification and Properties" section) the "Tipe Decay" options, to be marked and applied to in the HBIM project. Property management can be performed in the selected element's own definition window or in the interactive schema. Subsequently, specific schemes or data tables have been formed that incorporate appropriate items for the identification of the elements, as well as particular properties of a conservation process (intervention, control or maintenance). The specialist technician involved can now make a multi-selection among the options available in the scheme. The listed parametric elements have links to be displayed in plan and 3D views.

4.6.2. Segmentation and Identification

In BIM models, classifying involves relating a BIM entity to the semantic typology or level of a classification system, assigning it its code and description. When an element is assigned different codes, a multi-classified element is obtained. For the case under study, the Duomo of Molfetta, the global point cloud of the interior has been introduced by fragments, segmented by their semantics, arranging the same elements in the HBIM project with the possibility of giving them the appropriate parametrics according to their constructive characteristics, structures, history, and other properties related to the conservation and restoration processes (Figure 15).

4.6.3. Ontology. Semantic Classification.

For conservation work, the Teamwork HBIM specialist must have all the information on the auscultation, control and diagnosis work, having at their disposal the historical references and conservation data of the property. For them, the HBIM project relies on schemas that incorporate the items of the property groups.

The point cloud portions of the Duomo have been structured into essential building sub-units. For its classification, the most advanced systems have been used, mainly based on the new classification SCFClass [40] and the Uniclass [41], classifying the structural system and coatings according to their materiality. The new classification systems (SCs), such as the SCFClass Classification System (Spain), allow several classes to be applied within the same SC table: an element can thus be multi-classified because it contains different variants of materials according to the MATERIALS table (ceramic brick masonry, ashlar masonry, lime mortar, natural pigments) and by various concepts of "activities" of the work (preparation of the brick or ashlar masonry, lime plaster, fresco painting). Although it must be recognized that we face certain difficulties here, when seeking the best adaptation of the database, made up of contemporary and recognized construction systems, to previous techniques that are not currently common; or because materials that are not currently commercialized are used. The diagram shown in Figure 16 is an example of this, where the portions corresponding to the columns of the Duomo di Molfetta have been filtered. The technician directly interacts for the assignment of the properties (Construction Type, Decription of Tecnique, Technology, Class of Material, Mortar), having for each one a series of pre-established options for its proper marking.

4.6.4. Identification of Decay

An important task in the control and diagnosis work is the appropriate auscultation, in order to classify the deterioration detected in the structure of the building and those existing pathologies in the coatings or finishes of floors, walls, columns, pilasters, arches, vaults and ceilings. In the case study, all the elements are bare of cladding, with the stone masonry of the Romanesque architecture characteristic of the Puglia region (Italy) being visible. This allows a better detection and measurement of detachments, fissures and other types of deterioration in wall masonry.

Based on the same SCFClass (V2) classification system, the Phase classification has also been used, specifically the FAS Phase. OCO Construction project. But since there is no specific category for conservation/preservation projects, the new FAS has been created. PCO.050 Execution of the Preservation Project. Thus, the Decays scheme of the Duomo columns shows all the Columns/Pilasters classified by their system (Column/Pilaster -SCFClass; Mansory column systems –Uniclass) and for its Execution of the Preservation Project Phase. In turn, the new FAS class has also been applied. PCO.050.01 Mapping of degredation. In this way, the localized Decays have been mapped, which in the case of the Column-X4Y3 have been of the type: Lack & Crack (Figure 17).

4.6.5. Marking and Labelling of Deterioration (Decays)

Visual representation tools, whether of the 2D/3D polyline type or surface with pattern and color, are very useful both in the phase of graphic marking and in identification and classification, as they also act as container elements of semantic information. In this way, implicit properties collected in the Conservation Process database are associated with them.

As mentioned above, a multiple selection of localized deteriorations (Lack & Crack) has been applied in the Column-X4Y3. But for a proper location in the HBIM project, they have also been graphically marked with a contour (3D polyline of the Morph tool) and a labeling associated with the "Note Decay" property: Fracture in the coronation of the central column, transversal to the central nave (Figure 18).

4.6.6. Risk Assessment

Another important phase in the conservation processes of a historic building is the performance of risk assessment tasks. The HBIM project should facilitate these processes in an interdisciplinary process, containing the specific properties suitable for this stage. The Duomo HBIM has gone through a phase of auscultation and previous studies, in which visual prospection has been of great importance, complemented, or not, with diagnostic tests. Continuing with the example given, the mark that outlines the detachment of ashlar has been mapped with the classification by "subdivisions" (zonings agreed by the discipline): AUE.010 Auscultation and trials, subdivision 1 (SCFClass V2). And it has been enriched with the enabled properties of the Risk Assessment category. Figure 19 has selected the Activity category property, showing all the options available for its markup: Visual, Empirical, Instrumental, Laboratory test, Action, Intervention, Preliminary activity, Preconsolidation, Cleaning (restoration), Consolidation, Integration, Protection, Disassembly/Reassembly, Cleaning (Housekeeping). And for the Activity type property the following options have been selected: Re-adhesion by pins, Stuffed same material. All the properties established with their selected items or options are collected in an HBIM Project Schema (Table 3).

4.6.7. Data Interoperability with Conservation Specialists

To make collaboration with agents external to the Teamwork HBIM team more flexible, workflow interoperability is solved with the export of schema data in Excel -xlsx- formatted tables. In this way, the specialist fills in the items of the auscultation with their variables, previously established or incorporating some new ones according to the criteria of the specialist-collaborator. The completed and revised tables are then imported into the HBIM project, automatically updating all the data. The interoperability process does not require programming processes, as it is based on simple interactive schemas that incorporate the appropriate pre-established properties. The types of Schematic will vary depending on the category of the data and the tasks to be carried out by the inspector and/or operator: inspection, control, monitoring, diagnosis of the crack or seat (decay), materials and techniques applied in the intervention, etc. Table 4 shows the risks detected in the columns and have been linked to the point cloud fractions, incorporating the items: Type Decay, Activity (type, class, category) & Risk Zone, and with the possibility of selecting the appropriate option.

5. Discussion

Nowadays, BIM platforms represent a great opportunity to serve as a collaborative workflow in the management of architectural and artistic heritage, to which it should be added that they currently satisfactorily support the data from TDP, UAV and TLS surveys, in the form of a cloud of points and processed surfaces, providing the possibility of observing and analyzing structural deformations, cavities and fissures common in a historic building. This potential must be exploited in this perpetuated scientific quest to implement mechanisms of analysis and records that can last over time. The insertion of this faithful data in the BIM project makes them parameterizable, being able to incorporate all kinds of semantic properties; therefore, the conservation of CH based on a graphic-semantic information model or HBIM becomes a very effective recording process. The methodology developed above is in line with the search for the maximum convergent direction towards the representation of a digital twin model of Architectural Heritage.

The first step to be established, before proceeding to an organizational structure of an HBIM project, is the identification and classification of the components and elements. The semantic classification, according to the standards of the area or country, structures the elements for the subsequent realization of the segmentation; although they are still limited in very specific elements of historical architecture as they are not common or standardized construction systems today. The post-processed information of the massive data is usually provided as a global point cloud, georeferenced in the same global system, so it is convenient to subdivide it for better management, both to improve the performance of the computer and to have all the elements well organized and classified. This procedure would facilitate the subsequent auscultation of the element as it is delimited, as well as the parametric modelling of it, the analytical study of its materials and deterioration, and the association of intrinsic parameters so that they can be evaluated by all the intervening agents.

One of the difficulties that has arisen in the study is that the algorithms available to automate the tasks of identification and segmentation of architectural elements, although they facilitate the tasks of selection and classification of surfaces such as walls, floors, ceilings and pure geometric shapes, present certain difficulties in detecting characteristic pieces that incorporate decorative motifs. These elements are classified in the category of others, although it is in this classification that the bulging and overhanging of surfaces that do not follow a geometric rule are also recorded. In the case study, this point discrimination has been used for the detection and proper classification of deterioration. Although it should be noted that currently the assistance and interpretation of the expert operator is still essential to achieve accurate results, as the CH is still orphaned of automatic classification algorithms that allow its specific typologies to be accurately detected.

Regarding Heritage Building Information Modeling (HBIM) from point cloud data (Scan-to-BIM), it is consolidated as an efficient and accurate process for the representation of heritage. But the difficulty lies in the fact that these historical constructions generally have complex geometry and shapes. The work of accurate digital modeling of a historic building remains a challenge, even more so when looking for a faithful representation. For this reason, it has been investigated to apply in a general way a flexible and productive methodology, not exceeding the LOD 200 level in the details of parametric elements of the Duomo di Molfetta, although complemented with the semantic data provided by the TLS&SFM techniques. The semantic and structured segmentation of the global point cloud has been used to use these portions as containers of faithful graphic information (geometry and materiality), even containing the same specific parameters for conservation work. The point cloud portion behaves like another parametric object, except that the surface is made up of a succession of 3D points. It will be the conditions of the environment at the time of scanning and the resolution of the equipment (scanner/camera) that will mark the quality of the graphic representation. Although it should not be forgotten that the most outstanding potential of the H-BIM methodology lies in the efficient and interdisciplinary management of cultural heritage documentation; and the project created has met these expectations.

Finally, the 3D model contained in the H-BIM project has been enriched by parametric elements obtained by various processes: the architectural-constructive typology of the historic building is represented globally as a simple parametric object; for certain objects identified with deterioration, they have been transformed into a polygonal mesh of the enveloping geometry; and, with regard to point clouds, the initial unstructured information provided by the TLS survey has been transformed into well-identified portions classified by typologies. And based on precise and organized semantic segmentation, it has been possible to develop the usual conservation processes, such as the identification of degradation, marking and labeling of deterioration (Decays), Risk Assessment and data interoperability with conservation specialists.

6. Conclusions

This research has justified the significant potential of the integration of TLS and photogrammetric scan data into the HBIM project to facilitate the conservation and restoration of artistic and architectural heritage, as demonstrated by the case study of the Duomo di Molfetta. Based on the results obtained, it can be concluded that the methodological process is very valid to be applied to buildings and historical assets, facilitating the complex process of identification and semantic segmentation in this field. In an HBIM project, generic typological elements of classical architecture may coexist, where there are recurrent classes that are supported by proportional rules (and therefore easy to parameterize), with others specific to a style characteristic of the time or geographical place and that contain complex geometries. The latter will be represented by the point clouds themselves or transformed into mesh surfaces for the faithful representation of the envelope. Any of the routes taken will never hinder the tasks of auscultation, control, maintenance and intervention.

The management of the usual deterioration in a historic building of this category is efficiently extended to all the agents of the multidisciplinary team. The 3D model incorporates not only faithful architectural details, but also supports up-to-date semantic data, optimizing the preservation of cultural heritage. The schemas with the appropriate properties for the analysis and conservation tasks are transmitted to the collaborators of the Teamwork project (Archicad), as well as to external specialists both in data tables (Excel Microsoft format) and in intrinsic values of the elements of the OpenBIM model (IFC format).

The methodological process described above facilitates the perfect integration of the most advanced current technologies, facilitating the most correct decision-making in the usual tasks of preservation and restoration. Bearing in mind the multidisciplinary participation in the conservation of CH, the HBIM project endorses the interoperability of data and, therefore, the interdisciplinary collaboration and participation of all specialist agents and other stakeholders. It is this way of preserving throughout the life cycle of the historic building that the protection of Italy's rich architectural and artistic heritage is guaranteed.

As future lines of research, we want to take advantage of the knowledge and skills acquired in this research to continue perfecting the methodological process in search of greater automation and perfection of rigid and rudimentary processes, but especially the best implementation of AI in the semantic segmentation of unstructured data.