Identity Document Presentation Attack Detection in Visible Light with Illumination-Controlled Scanner

1. Introduction

In the modern world, identity document recognition systems [1] are widespread, and the demand for automating their authenticity verification is gradually growing. For example, airports already operate special kiosks [2] for automated passport control, equipped with a camera for capturing a person’s face and a scanner for acquiring a document image.

Thanks to the use of a scanner, controlled acquisition conditions are created [3], which, in turn, lead to higher quality performance of document analysis systems. For reliable document authenticity verification, multispectral scanners are used [4,5]. Document analysis in the visible light, supplemented by analysis results in the IR and UV spectra [6,7], allows distinguishing an original document from a fake one with high accuracy. However, there is an approach where document authenticity verification could be performed only in the visible light.

A reliable sign of the absence of an attack on document presentation (when a fake is presented instead of the original document) during image analysis is the presence of OVDs (Optical Variable Devices) [8] (Figure 1a-c). They are characterized by a change in color properties when the angle of incidence and reflection of light changes. Therefore, to detect an OVD, a corresponding sequence of images is required. The variability of the angle of incidence and reflection of light when using a scanner can be achieved by controlling the illumination and using lamps located at different positions under the scanning surface.

Within this publication, we propose a method involving a scanner with controlled illumination (Intek PS4 [4]) to detect attacks on document presentation based on the presence of OVDs. The method is tested on the MIDV-Holo-Scan dataset collected in this work, consisting of scans of 200 physical passport samples used in the creation of the MIDV-Holo dataset [9]. The tested documents include 50 samples with OVDs (Figure 1a-b), accepted in this publication as originals, as well as samples simulating attacks on document presentation (3 classes of 50 samples each) (Figure 1d-f). The obtained dataset is open (https://zenodo.org/records/20758652) and is published in this work.

Section 2 reviews the existing literature related to input devices, scanners, attacks on document presentation, OVD extraction algorithms, quality assessment methods, and datasets. Section 3 describes the published dataset of document images from a controlled scanner. Section 4 is devoted to the description of the proposed method for detecting attacks on document presentation. Section 5 presents the results of testing the baseline and proposed methods, as well as conclusions about the results of the conducted research.

2. Literature Review

One of the common types of attacks on document presentation is the use of a printed fake made on a home or office printer based on a digital image of the original document [10,11]. Unlike a simple photocopy, such fakes can reproduce the color scheme, graphic elements, and personal data of a document with high quality, which significantly complicates their visual detection. At the same time, reproducing OVDs, which change their appearance depending on the angle of incidence and reflection of light, remains a difficult task for an attacker. Therefore, one effective way to detect such attacks is to analyze the presence of OVDs on the presented document.

Optical Variable Devices [8] are special marks whose appearance depends on the position of the light source and the observer. These elements allow a person to quickly verify the authenticity of a document. A common type of OVD is a hologram [9,12], characterized by high saturation and variability of appearance when the angle of incidence and reflection of light changes. Hologram detection [9,12,13,14] and verification [15] are usually performed in a video stream using rule-based [9,16] and trainable methods [17].

The OVD detection task is typically solved on a sequence of frames from a mobile phone camera [9,12,13,14], where the viewing angle of the analyzed object changes from frame to frame. An important requirement for methods of detecting OVDs is pixel-wise alignment of the document across different images. In mobile capturing [18], conditions are uncontrolled [19], so achieving pixel-wise alignment is difficult. Fixing the document relative to the capturing device helps avoid this problem. There are a few works on using special devices for OVD detection [20,21], where the angle of incidence and reflection of light is varied by turning on lamps located at different positions. However, such devices are not as wide-spread as scanners.

In tasks of document analysis and authenticity verification, compact multispectral scanners are often used, which feature LED illumination around the perimeter of the working surface. In such scanners, instead of a scanning carriage used in conventional office (flatbed) scanners [22], a camera is used, which makes such scanners similar to planetary scanners [18] in terms of image acquisition. Thanks to this, the speed of image acquisition increases, which is important when working with a large volume of documents.

A trade-off between the devices considered is a compact scanner with controlled illumination.

For the purposes of this research, physical document fakes created for the MIDV-Holo dataset [9] are scanned. The MIDV-Holo dataset [9] is publicly available and consists of images of artificial documents obtained using a mobile phone camera. The dataset includes documents of two formats (passport and ID card), 200 samples each, with 10 unique templates for each format. MIDV-Holo [9] presents (Figure 1) both documents with OVDs (of the hologram type) and documents simulating presentation attacks: 1) laminated copy of a document without an OVD, 2) laminated copy of a document with a hologram, 3) document with a hologram image.

In the baseline method for detecting OVDs [9], the TPR (True Positive Rate) and FPR (False Positive Rate) functions are used to evaluate detection quality:

$$TPR={\displaystyle \frac{TP}{TP+FN}},FPR={\displaystyle \frac{FP}{FP+TN}},$$

(1)

where TP is the number of correctly identified document samples with a presentation attack (samples without OVDs), TN is the number of correctly identified originals (samples with OVDs), FP is the number of originals falsely identified as a presentation attack, FN is the number of documents with a presentation attack falsely classified as originals.

In this work, scanned artificial passports from [9] are used, since there are no available datasets with pixel-wise aligned documents in images taken under different illumination modes. The quality assessment of the proposed OVD detection method is carried out in accordance with the reference publication [9].

3. MIDV-Holo-Scan Dataset

3.1. Controlled Scanner

For detecting OVDs in this work, a PS4 document scanner manufactured by Intek [4] is used (Figure 2a). The light source during scanning consists of six LED lamps, schematically presented in Figure 2b, whose relative brightness, as well as on/off state, can be controlled (Figure 2c-e).

3.2. Dataset Structure

To assess the quality of the proposed method, the MIDV-Holo-Scan dataset was created, containing scans of physical documents used in the creation of the MIDV-Holo dataset [9]. The set of scanned documents contained 200 passports, presented in 4 types (one type with OVD and 3 types without OVD simulating a presentation attack). Thus, the published dataset contains scans of 50 negative (with OVD) and 150 positive (without OVD) samples.

Let us call a scanning mode the set of active illumination lamps and their relative power values. This work considers modes in which exactly one lamp is active. To form the MIDV-Holo-Scan dataset, each document was scanned in all such illumination modes with variable (from 0 to 1 in steps of $0.1$) relative power of the active lamp. Examples of images contained in the published dataset are shown in Figure 4. In addition to document scans, calibration images are added to the dataset, the details of which will be considered below. The obtained dataset is available at the link https://zenodo.org/records/20758652.

4. Proposed Method

4.1. Algorithm for Detecting OVDs

The scanner’s standard software allows obtaining a document image in which OVDs are partially visible (Figure 2e). However, for reliable detection of OVDs, a single document image is insufficient (see Section 2), and a series of document images with a varying angle of incidence and reflection of light is required.

The proposed algorithm consists of four main steps:

1.

Acquiring a series of raw document images under different illumination modes;

2.

Normalization of the raw images in the series;

3.

Extracting features indicative of the presence of OVDs;

4.

Classification of the presented document into original/attack by the presence of an OVD.

Demonstration of the algorithm steps is carried out using documents from the published MIDV-Holo-Scan dataset (Figure 4), with holograms serving as the OVDs. The input images of the algorithm are normalized and have pixel brightness values in the range $[0,1]$.

4.2. Acquiring a Series of Raw Document Images

The proposed algorithm analyzes images obtained by scanning in modes, characterized in Figure 3a. Here, the mode is encoded by the binary representation of the active lamp number. The operating relative power values are chosen so as to minimize areas with saturated brightness values.

Figure 4 shows examples of document images obtained using the scanner in different illumination modes: the standard scanner mode, conditionally designated as 111111, when all lamps are on, and those considered in Figure 3a from MIDV-Holo-Scan. OVDs in these images are quite noticeable, but the areas of the document opposite the light source remain dark with hardly distinguishable details. This is why normalization of raw images will be considered below by brightness level (flat-field correction).

Figure 4. Examples of images by scanned document type, obtained in different illumination modes and by the scanner’s standard software: (a) document with hologram; (b) document without hologram; (c) document with hologram image; (d) photocopy of document with hologram.

Figure 4. Examples of images by scanned document type, obtained in different illumination modes and by the scanner’s standard software: (a) document with hologram; (b) document without hologram; (c) document with hologram image; (d) photocopy of document with hologram.

4.3. Normalization of the Series of Raw Document Images

Let us introduce the following notations. Let us call an image a set of pixels that are implemented as vectors in m-dimensional space $\mathbf{c}=(c^1,\dots ,c^m)$ depending on the Color Coordinate System (CCS) used. Then the image is representable by a matrix $C=\left\{\mathbf{c}\right\}$, in which the color vectors $\mathbf{c}$ are organized in a certain order. Hereinafter, pixel-wise operations will be denoted using color vectors $\mathbf{c}$, and reference to the entire image will be made through the same pixel notation c.

Images $C_k$ obtained from the scanner in mode $k=\overline{1,K}$, where K is the number of modes, are represented in the RGB CCS. Therefore, a pixel of such an image in mode k will be denoted as ${\mathbf{c}}_k=(c_k^R,c_k^G,c_k^B)=(c_k^1,c_k^2,c_k^3)$.

Normalization of the series of raw document images is performed in two stages: dark current correction and brightness normalization using calibration images. The preparation of calibration images is performed once and includes scanning a calibration object and processing the raw calibration images. The latter consists of 1) dark current correction, 2) clipping of low values, 3) smoothing, and 4) forming the calibration images. Let us consider these operations in detail.

4.3.1. Dark Current Correction

At low illumination levels, dark current arises in the sensor, which leads to the false appearance of non-zero values. To estimate the dark current $\mathbf{d}$, a set of images $D_k$ taken with the illumination completely turned off is used. Averaging over space and over the series is performed:

$$\mathbf{d}=\underset{k}{\mathcal{M}}\left[\underset{(x,y)}{\mathcal{M}}\left[D_k\right]\right],$$

(2)

where $\mathcal{M}$ is the averaging operator. Dark current correction for image C is defined as

$${\mathbf{c}}_{dark}=\mathrm{clip}(\mathbf{c}-\mathbf{d},0,max(\mathbf{c}-\mathbf{d})),$$

(3)

where the $\mathrm{clip}$ function means:

$$\mathrm{clip}(v,a,b)=\left\{\begin{array}{c}v,a\le v\le b,\hfill \\ a,v<a,\hfill \\ b,v>b.\hfill \end{array}\right.$$

(4)

4.3.2. Preparation of Calibration Data

Calibration images (Figure 5) are used for brightness normalization (Figure 6). To prepare them, a calibration object is scanned in the modes considered in Section 4.2. A sheet of laminated white paper is used as the calibration object, emulating the surface of most documents, whose pages are either laminated or made of plastic.

The dark current correction operation is applied to the series of raw calibration images. Then, low values are clipped according to the calib_threshold:

$${\mathbf{u}}_{clip}=\mathrm{clip}(\mathbf{u},\mathtt{calib}\text{\_}\mathtt{threshold},max\left(\mathbf{u}\right)).$$

(5)

The next step is smoothing of the calibration images:

1.

Downsampling by a factor of scale using bilinear interpolation;

2.

Application of morphological operations Opening and Closing with a square kernel of kernel_size, followed by averaging the images obtained as a result of these operations;

3.

Smoothing with a Gaussian kernel of size $2\mathtt{sigma}+1$;

4.

Upscaling to the original size (by a factor of scale) using bicubic interpolation.

Finally, the calibration images $U_k$ are brought to the same average brightness level:

1.

The average over the area $R_k$ (Figure 3b) of the calibration image k is calculated:

$$M_k=\mathcal{M}\left[R_k\right];$$

(6)

2.

The smoothed calibration image $U_k^{blur}$ is normalized for each mode k pixel-wise:

$${\mathbf{u}}_k={\mathbf{u}}_k^{blur}/N_k,N_k=max(M_k,\mathtt{target}\text{\_}\mathtt{brightness});$$

(7)

3.

The calibration images $U_k$ are converted to single-channel $U_k=\left\{u_k\right\}$ by averaging across channels:

$$u_k=\mathcal{M}\left[{\mathbf{u}}_k\right].$$

(8)

4.3.3. Raw Images Brightness Normalization

The first stage of normalization is dark current correction. The second stage uses the prepared calibration images to equalize the brightness of images $C_k$ for each mode (Figure 6):

$${\mathbf{c}}_k^{calib}={\displaystyle \frac{{\mathbf{c}}_k}{u_k}}.$$

(9)

4.4. Extracting Features of OVDs

To detect an OVD, the change in color between normalized images of the same series is calculated. The estimate of color change in a pixel is the standard deviation of the projections of color vectors onto the plane of average brightness (Figure 7), calculated over the set of normalized document images.

The projection of color vectors ${\mathbf{c}}_k$ onto the plane of average brightness for each mode (Figure 7) is calculated by the formula:

$${\mathbf{c}}_k^{\prime }={\mathbf{c}}_k{\displaystyle \frac{{\mathcal{M}}_W[\parallel {\mathbf{c}}_k{\parallel }_1]}{\parallel {\mathbf{c}}_k{\parallel }_1}},W=\left\{U_k^{\mathtt{norm}\text{\_}\mathtt{param}}\right\},k=\overline{1,K},$$

(10)

where ${\parallel ·\parallel }_1$ is the ${\ell }_1$ norm, and ${\mathcal{M}}_W$ is the weighted average operator with weights W, exponentiation with parameter norm_param of calibration images $U_k$ is pixel-wise.

The calculation of the standard deviation (Figure 8) of the color vectors projected onto the plane of average brightness is performed using the formula:

$$s={\left({\mathcal{M}}_W\left[{\left({\mathbf{c}}_k^{\prime }-{\mathcal{M}}_W\left[{\mathbf{c}}_k\right]\right)}^2\right]\right)}^{1/2},k=\overline{1,K}.$$

(11)

From Figure 8, it can be seen that areas with OVDs have a higher standard deviation than areas without OVDs.

4.5. Making a Decision on the Presence of an OVD

Let us construct a binary mask of OVDs H (Figure 9), in which pixels have a value of 1 if the original pixel of the deviation map is above ovd_threshold, and 0 otherwise.

Let us introduce a rectangular area specified by the parameter ovd_roi in the format (left, top, width, height). We zero out the OVDs lying outside this area of the image H (Figure 10).

Let us calculate the relative area of the found OVDs after zeroing and compare it with the specified threshold rel_area_threshold. If the relative area exceeds the set threshold, then we consider that the OVD is present on the given document. Thus, the document is classified as genuine (if OVD is present) or it is classified as a presentation attack (if OVD is absent).

5. Testing of OVD Detection Methods

5.1. Baseline Method

As a baseline method, the hologram extraction algorithm published in [9] is used. This method was originally developed for analyzing frames obtained from a mobile phone camera. Despite this, the method [9] is applicable for working on images from MIDV-Holo-Scan, since the MIDV-Holo-Scan and MIDV-Holo [9] datasets contain images of the same documents on which the method [9] was tested.

The baseline method was run on series of raw document images (Figure 4). To adapt this method to the characteristics of the images in question, the ovd_roi of the proposed method is used, as well as a tuned threshold ovd_threshold=0.0035, according to which the input image is classified. Examples of masks of detected OVDs for various document types are shown in Figure 11.

5.2. Testing Results

Comparison of the quality of the baseline and proposed methods is performed on the collected MIDV-Holo-Scan dataset, a description of which is presented in Section 3. To tune the parameters, a training data split, consisting of 25% of the MIDV-Holo-Scan dataset (10 documents with OVDs, 30 documents without OVDs) was used. The resulting parameter values of the proposed method are shown in Table 1. The parameters ovd_threshold and rel_area_threshold were tuned on the training set during the maximization of the Accuracy function:

$$Accuracy={\displaystyle \frac{TP+TN}{TP+TN+FP+FN}}$$

(12)

with fixed values of the remaining parameters. The values of the remaining parameters were chosen based on the structure of the tested documents and the specifics of the images obtained from the scanner.

The performance quality of the baseline [9] and proposed methods on the entire MIDV-Holo-Scan dataset is shown in Table 2. The table shows that the proposed method significantly outperforms the baseline method, which proves the effectiveness of the developed scheme for detecting document presentation attacks. The main problems of the baseline method are the unstable response to document types such as documents with a hologram image and a printed photograph of a document with a hologram, as well as unstable extraction of holograms on originals.

6. Discussion

The proposed method detects presentation attacks from MIDV-Holo [9] with high accuracy. However, the number of attack types considered in this dataset is quite limited. Recently, the authors [17] published the MIDV-DynAttack dataset, consisting of documents with various distortions (e.g., a holographic sticker) that could be falsely accepted as OVDs. Thus, testing the algorithm on new types of attacks is required.

Furthermore, the proposed algorithm does not guarantee complete extraction of the OVD when constructing the mask. When comparing Figure 1.c and Figure 10.a, one can notice that the OVD is not fully extracted. This limits the application of this method in tasks such as verification of OVDs, in particular holograms. Extraction of the OVD in full is another possible direction for future research.

7. Conclusions

In this work, we proposed a method for detecting document presentation attacks based on the presence of OVDs using a scanner with controlled illumination in the visible light. For reliable detection of OVDs, a series of document images obtained under different illumination modes is analyzed.

During the development of the method, the MIDV-Holo-Scan dataset was collected, consisting of 200 physical documents scanned in various illumination modes, corresponding to the documents from the previously published MIDV-Holo dataset [9]. A distinctive feature of the collected dataset is the pixel-wise alignment of the document across different illumination modes. This dataset is published with this work and can be useful not only in tasks related to documents and authenticity verification but also in tasks of brightness normalization and image integration.

The quality assessment of the proposed method is carried out on the published dataset. The performance quality of the proposed method is TPR=100% and FPR=0%, which significantly exceeds the quality of the baseline method [9].

The high performance quality of the proposed method in the considered configuration of the authenticity verification system highlights the relevance of developing new, cheaper devices, such as controlled scanners without additional spectra. Scanning devices with controlled illumination, combined with the proposed method for detecting document presentation attacks, become advantageous and useful for application in different systems.