Imaging Based Techniques Combined with Color Measurements for the Enhancement of Medieval Wall Paintings in the Framework of EHEM Project

1. Introduction

The research presented in this paper has been developed within the project EHEM, Enhancement of Heritage Experiences: The Middle Ages. Digital Layered Models of Architecture and Mural Paintings over Time, JPI Cultural Heritage - Horizon 2020 Conservation, Protection and Use. To develop the project a consortium between the Universitat de Barcelona, the University of Rome 3, the University of Tuscia and the Technological Center CYENS in Nicosia (Cyprus) was created.

The main aim of the project was addressed to face the complexity of the medieval monuments by investigating three pilot sites: Sant Quirze de Pedret (Spain), the Enkleistra of Agios Neophytos (Cyprus); and Santa Maria Antiqua in Rome with the specific focus for combining technical instruments and methodological approaches able to improve the knowledge of the monuments and, above all, the fruition quality by various audiences. The EHEM project aim to create “Digital Twin” of each selected pilot site that digitally represents of the real monument and is able to integrate information from different sources, to visualize the monument through the various transformations occurred across the part centuries and to perform simulations and analyses [1].

This process is particularly relevant in the medieval monuments because they often contain wall paintings very complex and articulated and fragmentary, such the case of the church of Santa Maria Antiqua in the Roman Forum, that is the specific object on which the present paper will focus the attention.

The church of Santa Maria Antiqua was discovered in 1900 on the slope of Palatine. The church was abandoned in the ninth century, probable after an important earthquake that interested the area of the Roman Forum. Santa Maria Antiqua church, which was established in the 6th century within the structures of the Domitian age, preserves mural paintings on the walls dating back to between the 6th and 11th centuries, mostly superimposed on each other in a palimpsest. Overall, these are around twenty decorative phases which find the fulcrum of early medieval pictorial evidence in the wall to the right of the apse (the so-called "palimpsest wall") through the superimposition of eight layers of wall paintings executed between the 4th and 8th centuries [2].

The indisputable quality of the paintings of Santa Maria Antiqua is today reflected in the state of conservation in which they came to us: fragmentary, incomplete and with a weakened pictorial film as well as lack of surface finishes. The state of conservation is determined by the buried permanence of the paintings until the moment of discovery and the subsequent changes undergone, in particular the whitening of the surface, when they came into contact with the air and the high relative humidity of the environment. Furthermore, during the restorations carried out between 1900 and 1910, substances such as wax and glue were applied to the surface which left traces on the pictorial surface, altering the perception of colors [3]. This is a substantial aspect when, as in our case, the painted surfaces must be digitally replicated to create their "digital twin" to be used as a dissemination tool.

In the EHEM project, the role of colour in wall paintings was decisive. This is why we set out to acquire the specific physical and chemical characteristics of the colours by combining them with the characteristics of the constituent materials and the manner of execution. From a methodological point of view, together with the autoptic investigation and the analytical-scientific characterization of the pigments performed in previous work [4] aimed at recomposing the palette used by the painters who worked in Santa Maria Antiqua, multispectral and colorimetric investigations were prepared to implement knowledge of the color characteristics of paintings as they appear today.

The multispectral reading of medieval wall paintings in a compromised state of conservation, such as the one present in Santa Maria Antiqua, allows the image to be "re-composed" and enhanced where finishes, highlights or brushstrokes are no more perceivable in visible light, or they have weakened. A similar case was that concerned the Giotto paintings where an investigation was explored but with different approach [5].

Here we propose, for the first time, the combined use of an innovative technique, named Hypercolorimetric Multispectral Imaging (HMI), coupled with colorimetry, for the mapping of the colors so allowing the reading of the images to be "returned" for their digital reproduction. Therefore, multispectral imaging investigations allow us to deepen our knowledge of the execution technique and the state of conservation of the paintings, but also to add precious information for the definition of color, especially where this is present only in traces. As demonstrated in the literature, imaging techniques have become today widely applied in cultural heritage because they can provide a complete knowledge of the surfaces in a fast and reliable way without the need for sampling micro-chips of materials for laboratory analysis, a fact that is in general highly desirable, or even mandatory in some cases [6,7,8,9,10,11,12]. In our approach the potentiality of multispectral imaging was exploited to obtain chromatic distribution of the colors accurately measured by point colorimetry applied on some selected areas of the wall paintings covering the most relevant historical phases of the church decoration. HMI is a multispectral imaging technique recently developed by the society Profilocolore (Rome, Italy) whose principle and potential application has been reported in some published papers [13,14,15,16]. This technique demonstrated its great utility in the investigation of paintings and other kinds of artworks [16,17,18,19,20].

The paper is organized as follows: after the present introduction, paragraph 2 describes the paintings chosen for the investigation and the methodologies used, then paragraph 3 reports the results in term of color similarity maps and chromatic coordinates. The subsequent paragraph, nr. 3, is discussion in which the results are commented and compared to previous results on pigment analysis; at last paragraph 4, conclusion, summarize the main findings of the study and the possible future developments of the research.

2. Materials and Methods

The methodology proposed in this paper and used to support the EHEM project, was based on the combination of an imaging technique and a spot measurement. The first allows to obtain mapping of the different colors visible in the different areas chosen for the application of the method, the second (i.e. spot color measurement) gives the precise data for each color.

So, by combining the maps and the chromatic coordinates we can obtain a complete palette of the paintings.

The used techniques are the hypercolorimetric multispectral imaging (HMI) and the colorimetry by a portable digital instrument as detailed in the following sub-paragraphs, after having briefly described the painted areas chosen for the measurements.

2.1. The Paintings Chosen to Apply the Color Measurement and Mapping Methodology

One of the crucial issues one faces when studying the wall paintings of Santa Maria Antiqua is the complex stratigraphy of the ten pictorial phases in the Presbyterian area. In the EHEM project, in order to understand the use of colors in the different phases, eight representative sample areas were selected (Figures S1–S8).

The numbering of the areas follows the order of acquisition of the images, which were selected by favoring the painted portions accessible without scaffolding but representative of the diachrony of the paintings and their stylistic characteristics. The images provide information on the use of colors between the 6th and 9th centuries by local workshops of painters and others influenced by Hellenistic modes or from Byzantium.

Following the chronological order, the first area is the image of St. Anne holding the Child Mary in her arms painted on the right wall of the presbytery and dated to the last quarter of the 6th century (area 2, Figure S2). This is followed by a nucleus of paintings datable to the 7th century that belong to different decorative phases and thus to different workshops. These are the paintings on the small wall to the left of the apse where a false marble area was painted in the first decade of the 7th century (area 4A lower part, Figure S4) and later, in the upper part, the figures of the Church Fathers pertaining to the year 663 AD (area 4B, upper part, Figure S4). On the pillars of the nave in the first half of the 7th century, various scenes were painted. For the color measurements the Solomon and the Maccabees scene was chosen. This scene was painted on the lower part of the northern side of the pillar positioned in the south-west area of the nave (area 6, Figure S6).

At the time of Pope John VII (705-707) the decoration of the church, specifically in the area of the chancel, was completely renewed and various scenes were added on the pillars of the nave. To compose the palette used by the painters, our attention went to three areas: the high choir (area 1, Figure S1); the veil painted on the left wall of the Presbytery (area 3, Figure S3) and the scene with the Annunciation (area 7, Figure S7).

A homogenous core is represented by the paintings in the Theodotus Chapel, created at the time of Pope Zechariah (741-752), which opens to the left of the chancel. Here as a reference area for colors’ investigation, the area depicting the so-called ’family portrait’ with the donors next to the figure of the Virgin was investigated (area 5, Figure S5).

The decorative phases of the church are completed by the pictorial apparatus created at the time of Pope Stephen II (752-757). From this phase we have taken into consideration a portion of the left wall of the eastern side with the theory of figures of saints, fathers of the western and eastern church (area 8, Figure S8).

2.2. Colorimetry

Color measurements were performed on selected points in order to cover the entire palette of the paintings. The color measurements points were carefully documented in detail and in the general photograph of each selected area (see Figures S1–S8 in the Supplementary file). An EOPTIS digital and portable colorimeter was used equipped with a conical head allowing to measure spot of 6mm in diameter. The instrument operates with a fixed geometry (45°/0°) and the selected standard observer a 10°, illuminant D65. Three measurements were performed in each selected point.

2.3. HMI Technique

HMI system and procedure has been explained in previous papers, so here a synthesis of the main characteristics and working methodology is summarized [13,18].

First of all, to obtain the calibrated multispectral images, the acquisition is done through a Nikon D800FR (Full Range) camera, a 36megapixel reflex camera modified under a Nikon/ Profilocolore® common project to achieve an extended 300-1000 nm range sensitivity, with the use of portable camera flashes also modified in full range. The accurate calibration of the system is done through a color checker made of 36 color patches from NCS (Natural Colour System® catalogue) standard catalogue and the use of white reference patches with 98% of spectral reflectance. The spectral reflectance of the color checker and white references was measured through a laboratory spectroradiometer (Instrument System Spectroradiometer CAS 140 CT) in the range 220-1050 nm with 0.7 nm accuracy in a dark room in Profilocolore® laboratory.

Image acquisition in the UV-NIR range is done using two bandpass filters called A (whose spectral transmittance include UV and VIS range) and B (including VIS and NIR band until 1000 nm) designed by Profilocolore®. The transmission spectra of the two filters are published in [19].

A great advantage of HMI is the short time needed for the acquisition – only two shots are needed – without the necessity of any power supply, thus providing a powerful instrument for in situ investigation, such the case of Santa Maria Antiqua.

The acquired raw images are then calibrated through the HMI software SpectraPick® that produces the seven monochromatic images in tiff format centered at nm: 350, 450, 550, 650, 750, 850 and 950 and the RGB output. The folder with the images produced after calibration can be processed through the software PickViewer®, included in the HMI system, that is used to apply different kinds of tools to the calibrated images in order to extract information useful for the analysis and knowledge of the artworks.

Specifically, PickViewer® was used to calculate the chromatic similarity maps of each selected color visible on the wall paintings. To do this, after having selected the color, that is the same measured by colorimeter, it was asked the software to find all pixels in the image having similar chromatic values.

An example of the procedure followed for obtaining the chromatic similarity maps is displayed in the Figure 1.

3. Results

The results of the HMI mapping and colorimetry were synthesised for each acquired area of the Santa Maria Antiqua wall paintings. In the Figure 2 the results obtained for the Area 1, with the scene of David and Goliath (high choir), are shown.

In the first examined area seven main colors have been identified. The point 1 in the Figure 2, is referred to the light green background whose average chromatic coordinates are: L*=59.6, a*=-7.40 and b*=12.2. From the map of the chromatic similarity, the green color is distributed in the background of the area (white pixels in the map) and not in other part of the painting.

Point 2, that is a red color, has been used in limited portion of the painting, well defined in the map of chromatic similarity. Point 3 is a dark green used in the background, superimposed to the light green selected in the point 1; for this reason the distribution of this color is also in the area where the light green has been mapped. Point 4 is a yellow color that shows a well-defined distribution in the painting elements being mapped in the decorative elements on the left, in the legs of the figure in the center, in the ground, in the frame (lower part) and in the flourish of the scarf.

Point 6 is taken in a white area of the dress of the figure in the center of the scene. White color is distributed in almost all the surface, apart from the yellow zones.

At last, the black color has been chosen in correspondence of the points 5 and 7. From the maps it can be seen that this color is distributed in all the surface, apart from the light yellow areas that characterise the two hills in the lower part of the background and the flourish.

The second examined wall painting was that representing St. Anne with Child Mary located on the right wall in respect to the central apse of the church (Figure 3).

Also in this case we obtained the color mapping associated to the values of the chromatic coordinates in the selected areas. It is interesting to note that some colors are limited to few areas of the painting such as the yellow color (point 4) that has been mapped only in the halo of the saint, or the light blue color that is present in the frame with the inscription, in the veil of St. Anne and in the garment of the Child Mary (point 6 and point 1).

The green color visible in the area of point 2 does not exhibit the coordinates of this color (a* should have a negative value). Instead, from the value of b* it can be derived a yellow hue. This leads to suppose that the green color was obtained by mixing pigments able to produce such hue.

Point 7, corresponding to a white area, has been mapped in the entire area, apart from the yellow, red and blue zones, even that referred to point 1 that is of a slightly bluish white. This led to suppose a different mixture for obtaining the white in the garment and that in the frame with the inscription. Considering the similarity between point 1 and point 6 colors, it may be derived that the difference with the white in point 7 is due to the presence (in points 1 and 6) of a blue pigment or a pigment having a bluish hue, such as for example vine black. This last black pigment, in fact, gives a clear bluish hue to the wall painting, if mixed with lime white, and is frequently used for this characteristic [21].

Point 5 corresponds to a dark red color used for the figures contours and, probably mixed with the white, to obtain the pink color of the Virgin mantle.

Area 3, representing aniconic decoration (Figure 4), has few basic colors: yellow (point 1), red (point 2), bluish grey (point 3), black (point 4) and white (point 5).

The chromatic similarity maps show a net distribution of yellow and red colors indicating the probable presence of pure pigments; on the other hand, the black and white colors (points 4 and 5) exhibit chromatic characteristics similar on the entire surface, apart from the yellow and red areas where the color are recognized completely different. This is probably due to the presence of a white pigment mixed to the black or the whitening of the surface in correspondence of the point 4. In fact, the wall surfaces of Santa Maria Antiqua are interested by whitening due to the high values of relative humidity in the church that caused recrystallisation of calcium carbonate [4]. This problem was partially solved with past and recent intervention on the structural asset, but the environmental conditions are not completely stabilized yet [22].

Area 4 is characterized by two different pictorial phases superimposed on each other: the false marble decoration, visible in the lower part of the area and the 663 AD phase, in the upper part of the area (Figure 5 and Figure 6 and Figure S4).

In this area, color measurement points from 1 to 8 are referred to the false marble decoration; point from 9 to 13 are taken from the layer date back to 663 AD (the pictorial phase realized during the Pontificate of Vitalian). This layer was very fragmentary, and it was not possible to reach the upper parts so that only five points were measured and mapped, but representative of the palette of the 663 AD phase.

In the false marble decoration, the various colors are generally well-defined in the map such as brown (point 1), yellow (point 2), red (point 4), black and different greys (points 3, 5, 7 and 8).

Point 6 is a thin white line (the sole white zone where it was possible to take a color measurement). Apart from the yellow zones, white color is present in all the other colors and in the grouting, according to the obtained mapping. In Medieval wall paintings white is usually obtained by using calcium carbonate from various sources that is also the binder of the pigments in the fresco technique, so it is not strange to find this white combined with the other colors [23].

The painting referred to the 663 phase, was sampled in five points: yellow-orange (point 9), black (point 10), red (point 11), yellow (point 12) and white (point 13). Red and yellow colors are well-defined whereas white is distributed on the entire surface, apart the red and yellow areas, being it probably mixed with the other pigments to obtain the light hues visible on the painting of the 663 AD phase.

The Area 5 has been chosen in the so-called Theodotus Chapel, positions in the left side of the church, in respect to the central apse. The results of the color measurements and mapping are shown in the Figure 7.

The main visible colors are blue (point 1), yellow (point 2), yellow-orange (point 3), white (point 4), brown (point 5), black (point 6), red (point 7) and green (point 8). The blue color is concentrated in the garment of the central figure (point 1).

Two kinds of yellow have been measured: a lighter hue in point 2 distributed in the pavement and in the garments of the first and third figures; and a darker yellow mapped in the footstool and in the upper part of the first figure’s garment (point 3).

The white color has been sampled in one of the pearls that decorates the footstool, where this color appears more intense (point 4) and in the mapping it is concentrated in the garment of the first figure and in the mantle of the fourth one. White is also mapped in the lines of the frame and in the garment of the central figure probably added to the blue to create highlights in the folds. Clearly, the white color has been mapped further in the lacunae and grouting visible in various zones of the panting.

Points 5 and 7 are associated to brown and red colors used for the garments, for the hair of the second figure (from the left), for the contours of the garment, for the frame dark lines, and also for the footstool base. It must be noted that the value of a* coordinate in point 5 is negative suggesting the possibility that the colorimeter spot collected not only the brown of the garment contour, but also part of the green background, due to the thin contour thickness. So, this measure cannot be considered corrected. On the other hand, the map, that is obtained by a much less reduced area of the image, is referred exclusively to the brown contour. This result further demonstrated the relevance of hypercolorimetric multispectral imaging in the in-depth investigation of colors’ distribution.

The point 6 is referred to a black line defining the frame of the painting. The black is also mapped in the background and in the garment of the central figure, probably because a black pigment has been added to the painting materials for obtaining a more saturated and dark color. This is a usual approach in the byzantine wall paintings [24].

Lastly, the green colour characterizes the background of the area as visible in the Figure 7 (mapping point 8). The chromatic coordinates identify a real green color (a* has a negative value) and the mapping clearly defines the distribution in the background of the scene.

The next examined area is number 6 located in one of the pillars on the right side of the church representing Solomon and the Maccabees (Figure 8).

In this area we sampled eight points corresponding to the main colors visible on the paintings: dark and light grey bluish (1 and 2 respectively), yellow (point 3), green (point 4), dark red (point 5), black (point 6), brown (point 7) and white (point 8).

The white and grey areas have color similarities, as found in the mapping of point 1, 2 and 8. Yellow, red and brown colors have defined areas and appeared used pure, without mixing of white or black. Moreover points 5 and 7 have very similar maps suggesting the use of the same pigments. Black color, sampled in the point 6, has been recognized also in the grey and light grey areas, including lacunae and grouting probably due to the addition of a black pigment.

Another chosen painting to apply our methodology for sampling colors, is area 7 attributed to the phase of Pope John VII, Figure 9.

This area represents the Annunciation of the Archangel Gabriel to the Virgin Mary. It is a detached area positioned on a mobile support, close to the entrance of the church.

Points 1 and 6 are referred to yellow colors that, from the mapping and also from the chromatic coordinates are found similar. Point 5 is another yellow that shows different coordinates, especially b* that is higher as value, indicating a more saturated color. Point 2 is a dark pink color applied over the yellow and in fact it is distributed also in the background on the right of the Arcangel. Point 3 is è greenish color whose mapping is limited to the line in correspondence of the Arcangel garment and to frame on the left. Some similarities have been found also in the background probably for the presence of common pigments or other materials (surface whitening due to calcium carbonate, as previously written). Point 4 s the dark pink color of the background that s well-highlighted in the right side of the area. Point 7 is taken on a white color of the Arcangel dress. It is present in the white parts of the painting but further in the grouting and lacunae where calcium carbonate may be hypothesized as mail constituent material.

Points 8, 9, 10 and 11 are referred to dark colors: brown, dark red and black.

The maps show similarities between points 8, 9 and 4 probably because the pink color was obtained with the same materials but with different combination with whites and black added to darken or to lighten the red pigment hue.

Point 10 is a completely different color, distributed only in the brown areas of the painting, probably associated to a brown earth.

The last examined area is that of the Pope Stephen II (Figure 10). The seven selected points represent the man color observed on the painting: white (point 1), yellow (point 2), black-grey of the dress (point 3), red (point 4), black of the frame (point 5), light yellow (point 6), and another red (point 7).

White color (point 1) is clearly well-distributed in the white parts of the panting, but also in the lacunae and in the black-grey zones, which are probably obtained by mixing black and white pigments. Point 2 is a yellow color distributed in the garments but also in the halo of the first saint from the right. The other yellow point (6) is found in the same areas of point 2 mapping but further in other parts of the dresses and in the pavement, where it was sampled for the analysis. From the values of the chromatic coordinates, the yellow referred to point 2 is more saturated in respect to that measured in point 6 where a mixture of yellow and red pigments seems probable. Point 4 and 7 are associated with red colors. The one sampled in the point 4 is well-distributed in the red areas of the garments, in the background and in the upper frame. The red corresponding to the point 7 has similar mapping of point 4, but it appears concentrated in some areas: the lines used to define the wrinkles on the lapel of the garment, the upper frame and the lower part of the background right side.

4. Discussion

In the church of Santa Maria Antiqua, a complex wall painting apparatus is today visible, after several restoration campaigns subsequently to the discovery and excavations that brought to light this incredible pictorial palimpsest.

The wall paintings were realized in different historical period and by various painters creating sometime overlapping of several layers, such as in the right wall of the central apse where the so-called Angelo Bello (The Beautiful Angel) can be admired; for a detailed description of the wall palimpsests see [25,26,27,28].

During the long restoration that interested the wall paintings of Santa Maria Antiqua, various diagnostics campaigns were performed always with spot analysis of pigments and binders that allowed to characterize most of the constituent materials [4,29,30,31].

But, until now, a color measurement campaign was never performed, for this reason the EHEM project addressed its focus on the aspect of colors evaluation and mapping.

The main colors observed on the wall paintings, in the different areas and historical phases, can be grouped in eight main hues: blue, grey, green, yellow, red, brown, black and white.

Blue and grey hues

Rarely a true-blue color has been detected, but rather a grey bluish appearance was found, such as in the St. Anne point 6 (Figure 3), area 3 point 3 (Figure 4) and Maccabees point 1 and 2 (Figure 8). The only true-blue color is that referred to point 1 in the Theodotus chapel painting (Figure 7). The analysis performed on various samples of blue or bluish appearance have been performed in the past and revealed two kinds of materials used for obtaining the blue appearance: Egyptian blue and wine black [4,29,30,31]. This last black pigment is characterized by a bluish hue, but the chromatic coordinates indicate that it is a carbon-based pigment not a real blue. Otherwise, the Egyptian blue is characterized by a* and b* coordinates typical of this pigment, in particular a pale greyish blue as reported by Bianchetti et al. [32].

Green hue

Green colors have been generally obtained by using green earth, specifically celadonite [4,30,33]. However, these are not always pure colors but are sometimes made by mixing different pigments to obtain variations in hue.

The study of the constituent materials has made it possible, in fact, to highlight the widespread presence of earth green pigment darkened with carbon black or lightened with lime white and the more limited use of a green hue obtained by mixing yellow ocher with carbon black (Figure 11). It is a specific color created by the mixture of two non-green pigments to obtain a pastel hue which marks the work of a specific workshop responsible for the paintings of the so-called phase of Angelo Bello on the palimpsest wall and on the area with Saint Anne and Child Mary painted on the west wall of the presbytery [31]. In the post-production of the photographic images of these paintings for the digital model we could also attribute the term "green" to the color obtained from the mixture of ocher and black, homogenizing it to the green hues created with green pigments.

Yellow, red and brown hues

These colors are obtained with iron oxide pigments containing different kinds of iron and manganese compounds [4,34]. Iron oxide pigments have been used for centuries, since ancient times to the present, thanks to their great availability from natural sources [34]. They are widely diffused on the wall paintings of Santa Maria Antiqua and were used in all examined areas for its compatibility with fresco techniques and for its stability over time. The chromatic coordinates of iron oxide pigments are highly variable depending on their source, on the materials present in the mineral and on the possible use in mixture with other pigments to make yellow, red and brown colors lighter or darker. The values of a* and b* obtained in our measurements for yellows, reds and browns are always lower in respect to those reported in the literature [34], probably due to the presence of surface layer of calcium carbonate white (whitening) formed because of re-crystallization phenomena.

Black and white hues

These two colors are present in all examined paintings and are used both for obtaining white and black areas but also mixed with other colors to create various hues, such as the case of green in the background of St. Anne area previously described.

Moreover, white is added to other colors to lighten them or black is added to darken.

White color is obtained with calcium carbonate [4]. In the wall paintings, generally this pigment was used in the form of the so-called Bianco San Giovanni, a stable material and also the same constituent of the plaster and binder for pigments in the fresco technique [35,36,37].

5. Conclusions

In this contribution we reported the results of the color measurements in some selected paintings of the medieval church of Santa Maria Antiqua in the Roman Forum, coupled with the multispectral similarity mapping of the same colors. An extensive color measurement campaign was never performed in Santa Maria Antiqua, so this paper contributes to the better knowledge of such extraordinary contest and to associate the observed colors with the pigments, characterized in previous diagnostic investigations.

The mapping through hypercolorimetric multispectral imaging demonstrated a highly useful tool for gathering the distribution of the colors. Moreover, the application of this imaging technique has allowed to acquire calibrated images that could be useful, in the future, to perform other processing taking the advantage of the PickVewer® software.

In the framework of the EHEM project, the chromatic and mapping data will be useful to try a reconstruction of the paintings in order to enhance their appearance for the visitors of the church. In fact, the fragmentary nature of the paintings may prevent a clear reading of the pictorial scenes represented in the various areas of the church and in the different historical phases.

The future goal of the project is to use our data to improve such readability.