Roughness Estimation and Image Rendering for Glossy Object Surface

1. Introduction

Most objects with a strong gloss on their surfaces are metals or dielectrics. The light reflection properties of these two types of materials are quite different. Metals are homogeneous objects consisting of specular reflection, while the surface layer of dielectrics is inhomogeneous, and their light reflection is a linear combination of diffuse reflection and specular reflection, known as the dichromatic reflection model [1]. Dielectric objects include many of the everyday items that are familiar to us, such as plastic objects, ceramics, and painted objects [2].

Lacquerware is dielectric, which refers to objects covered with lacquer. Lacquerware includes tableware, containers, furniture, daily necessities, and various small and large objects that people carry around. East Asian countries such as Japan, China, and Korea have a tradition of lacquer crafts that spans thousands of years [3]. The best-known lacquer is a urushiol-based lacquer common in East Asia obtained from the dried sap of Toxicodendron vernicifluum [4].

Lacquerware is basically handmade. Every lacquerware object made with the traditional technique, which has been in use for more than 900 years, is produced using natural wood and urushi lacquer. However, the use of traditional lacquerware has decreased recently because of the increase in cheaper plastic-based lacquerware-like objects. Plastic lacquerware-like objects are so well-made that it is difficult to judge whether an object is plastic or real lacquerware through a casual glance. Unlike natural lacquerware handmade, most synthetic object is mass-produced through molding by pouring plastic into a mold.

In this paper, we focus on the appearance of glossy inhomogeneous dielectric materials. The overall appearance of three-dimensional (3D) objects results from a combination of the chromatic factor of the surface spectral reflectance and geometric factors such as the surface shape and texture. It is however not possible to identify the objects based only on their spectral reflectance and 3D surface shapes, but also the appearance of the objects is affected by the surface microstructure.

The surface roughness is a measure of the microscopic surface structure. It is often quantified by the height deviation along the vertical direction relative to that in the surface shape of an object with an ideal surface. A variety of methods to measure the surface roughness have been proposed for industrial surface inspection [5]. The International Organization for Standardization (ISO) specifies the well-known standard roughness parameter, Ra, in ISO 4287:1997. However, Ra does not necessarily match the perceived appearance of the surface roughness on an object. Instead, the perceived appearance of the surface roughness is correlated with the deviation of the surface normal vectors. Ohtsuki et al. [6] analyzed the surface roughness of human skin, and Oren and Nayar [7] proposed a reflection model for rough surfaces such as concrete and sand in which the surface normals of the surfaces are described using 1D Gaussian distributions.

On the other hand, in the field of computer graphics and image rendering, the concept of roughness has already been used to create realistic appearance of objects. Almost all surface scattering models and illumination models contain parameters that describe surface roughness. For example, the Beckmann function [8] has often been used to render the realistic appearance of objects [9]-[14]. However, such a roughness is merely a roughness parameter in a mathematical model and does not necessarily represent the actual physical roughness.

In this paper, we study the relationship between the roughness parameter in image rendering and the actual surface roughness—i.e., the physical roughness. We then apply this relationship based on flat surfaces to image rendering and roughness estimation of complex curved surfaces.　In order to analyze the surface roughness of glossy objects, we use lacquer plates which are handmade and allow to control the surface glossiness, as well as several plastics and an actual lacquer product from everyday life. The physical surface roughness of a glossy object surface is defined as the standard deviation of the surface normal

In the following, in Section 3, we describe a method for estimating surface roughness based on physical measurements. We use a laser scanning system to measure the microscopic surface height of the target object. The surface normal vector is then calculated at every pixel point using the heigh information.

In Section 4, a method is developed for estimating roughness parameter based on camera images. We first describe mathematical models for representing specular reflection. The specular lobe generated by light reflection has a significant dependence on the surface roughness, which is approximated using an analytical function with a roughness parameter. We estimate the parameter from the observed high-dynamic-range (HDR) image of a flat surface on the target object.

Section 5 describes a relationship between measurement-based roughness and the image-based roughness parameter. We create a linear regression model between them. This model is shown to be useful later.

Section 6 presents practical applications to glossy objects. We demonstrate appearance rendering using the estimated roughness parameter, and also estimate the physical roughness values for curved glossy object surfaces.

2. Materials Used in the Study

Figure 1 shows a set of object materials used in this study.　Since lacquer products are handcrafted unlike plastics, it is possible to vary the surface roughness over a wider range, so we used various lacquer products with differing gloss levels. The degree of gloss varies depending on the surface finishing process. Figure 1(a) shows the five test samples named as follows:

A: Black lacquer, Roiro polished finish,

B: Matte black lacquer painted finish,

C: Glossy painted lacquer finish,

D: 30% glossy black lacquer painted finish, and

E: 50% glossy black lacquer painted finish.

Roiro is a traditional technique where lacquer is applied with a brush, allowed to dry, and then rubbed with charcoal called Suruga charcoal, followed by a process of dozuri, awazuri, and buffing. The surface gloss levels decrease in the order A > C > E > D > B. To illustrate the glossiness of the object surface, Figure 1(b) presents a specular reflection of a ceiling fluorescent light on test sample C.

Figure 1(c) shows two black plastic test boards: acrylic on the left and polyvinyl chloride (PVC) on the right. Figure 1(d) presents three lacquerware samples selected from everyday items: a lacquer tray (left), a lacquer towel holder (middle), and a lacquer box (right). Figure 1(e) shows a red Japanese sake cup made of real lacquer with a curved surface.

3. Measurement-Based Surface Roughness Estimation

3.1. Definition and Calculation for Surface Roughness

The surface roughness of a glossy object surface is defined as the standard deviation of the surface normal. The computational procedure is given below [15]. Assume that the height deviation distribution of the surface is obtained at grid points on the XY plane as shown in Figure 2. The surface normal vector at each grid point can be estimated from the height distribution, as described later.

Let N be the total number of surface normal data points. The surface normal vector $n_i$at the i-th point (i=1, 2, …, N) and the averaged vector $n_0$ are described using 3D column vectors:

$$n_i=\left[\begin{array}{c}{x}_i\\ y_i\\ z_i\end{array}\right](i=1,2,\mathrm{...},N),,$$

(1)

$$n_0=\left[\begin{array}{c}{x}_0\\ y_0\\ z_0\end{array}\right]={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N{n}_i},,$$

(2)

where

$$‖n_i‖=‖n_0‖=1,$$

(3)

We define the autocorrelation matrix T and the scalar J (see [15]) as

$$T={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N{n}_i{n_i}^{\mathrm{t}},},$$

(4)

$$J=1-{n_0}^{\mathrm{t}}T{n}_0,$$

(5)

where the superscript t denotes matrix transposition. We then have

$$\begin{array}{l}J={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N\left(1-{n_0}^t{n}_i{n_i}^t{n}_0\right)=}{\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N\left(1-{\left({n_0}^t{n}_i\right)}^2\right)}\end{array},$$

(6)

Giving the constraints in Eq. (3) and the fact that ${n_0}^t{n}_i$is the inner product corresponding to the cosine $\cos {\theta }_i$ of the angle ${\theta }_i$ between vectors$n_0$and$n_i$, as illustrated in Figure 3. Eq. (6) can thus be rewritten as

$$J={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N\left(1-{\left(\cos {\theta }_i\right)}^2\right)=}{\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N{\left(\sin {\theta }_i\right)}^2},$$

(7)

where $\sin {\theta }_i$ is the length of the perpendicular line drawn from the tip of the vector $n_i$to $n_0$ (see Figure 3). That is, J represents the mean of the squared lengths of the perpendicular lines drawn from each $n_i$to the average vector $n_0$. Therefore, the square root of J,$\sqrt{J}(\triangleq R)$, is the standard deviation of the surface normal vectors–i.e., the surface roughness R defined above.

3.2. Surface Roughness Estimation Based on Physical Measurement

A laser scanning system, as shown in Figure 4, was used to obtain the precise surface height information of the target objects. The system consisted of an XY-stage and a sensor called a laser displacement meter. The sensor used was mainly the Keyence Model LT8010, and occasionally Model LT8110. The surface of an object is scanned with high accuracy at a resolution of 0.1 μm. The advantage of this measurement system is that unlike a camera system, there is no lens distortion owing to the direct measurement of the surface.

Measuring a large area on the surface of the target object is time-consuming and physically challenging, so we measure the height of a small area. As a typical example, the surface height of a tiny flat rectangular area of 0.5 mm x 0.5 mm on the target surface was measured at a 2 μm pitch, resulting in a height profile of 251 × 251 grid points, from which a subset of 201 x 101 points was used as height information for analysis. Because the target surface was not perfectly flat, the base surface was determined via smoothing using a moving average and the height deviation was calculated as the difference between the measured and base heights.

Figure 5 shows the measured height deviation distributions of the four different samples of real lacquer plates and opaque plastic boards, where panels (a) and (b) show the measurement results for the real lacquer-C and -D plates and acrylic and PVC plastic boards, respectively. The z-axis scale unit is μm.

The MATLAB function “pcnormals” was used to estimate the surface normal vectors from the height data. In this function, the six neighboring points to each point are used for local plane fitting to determine the normal vector at the point [16]. The 3D distributions of the normal vectors suggest that even when object surfaces appear similarly glossy, the microscopic features of their roughness can be significantly different. In fact, Figure 6 shows the 3D distributions of the surface normals for the lacquer and plastics, where panel (a) compares the surface normals between the lacquer-C and -D plates, and panel (b) compares the ones between the acrylic and PVC boards.　Overall, the surface normals of lacquer are more widely distributed than those of plastic, but when examined individually, it can be seen that lacquer C and acrylic have normal directions concentrated more vertically than the others.

Figure 7 shows images shaded using the surface normals obtained at the grid points over the respective object surfaces. The illumination is assumed to be incident at 45°. A comparison of the two sets of images in Figure 7 (a) and (b) shows that the surface of the real lacquer-D plate look rougher than those of the plastic objects.

The values of the surface roughness R were calculated from the standard deviation of the surface normals according to the previous subsection’s procedure as R=0.0554, 0.1062, 0.0310, and 0.0740 for lacquer-C and -D plates, and acrylic and PVC boards, respectively.

4. Image-Based Roughness Parameter Estimation

4.1. Mathematical Models for Specular Reflection

The surface roughness parameters of a glossy object are estimated using images captured by a camera. The specular function of a dielectric material is a mathematical function used to model its specular reflection. This function excludes color (spectral) information and primarily represents the geometric characteristics of the specular surface, which depends on the surface orientation, illumination, and viewing direction. Therefore, it can well represent the appearance of a glossy object using parameters related to its roughness and sharpness.

Consider a reflection geometry shown in Figure8, in which n is the surface normal vector, l the incident light vector, and v the viewing vector. Let $R_l$be l mirrored about n, and q be the vector bisector of l and v. Specular reflection occurs only within a limited range of viewing angles. This reflection component is typically strongest along the direction of reflection$R_l$and decreases rapidly as the angle $\theta$between $R_l$and v increases. This rapid falloff is often approximated as

$$f(\theta )=\beta {\cos }^m\theta ,$$

(8)

where $\beta$ is a constant representing the specular peak intensity and m is a parameter representing the surface roughness. This intensity distribution is known as the Phong distribution [9]. If the highlight has a pointed peak, a Gaussian distribution can be used to model the sharp falloff as follows:

$$f(\theta )=\beta \exp \left(-m{\theta }^2\right),$$

(9)

However, simple specular functions such as the Phong distribution cannot describe the surface specularity of rough surfaces adequately due to the presence of unknown constant parameters. In addition, there is always some surface roughness even when the surface appears smooth. The shape of the specular reflection lobe generated by dichromatic reflection depends significantly on the surface roughness. Rough specular surfaces are often modeled as assemblies of small planar patches, known microfacets [17].

The Beckmann distribution is a physics-based microfacet distribution model [8]. The specular lobe can be approximated using the Beckmann distribution function with a surface roughness parameter m as

$$f(\phi )=\left({\displaystyle \frac{1}{m^2{\cos }^4\phi }}\right)\exp \left\{-{\left(\tan \phi /m\right)}^2\right\},$$

(10)

where$\phi$ is the angle between q and n. Unlike the empirical model in Eqs. (8) and (9), this function yields the absolute magnitude of the reflectance without the need for arbitrary constants. The distribution function called GGX is also a distribution function that models microfacets, similar to the Beckmann function [18]. This function is written as

$$f(\phi )={\displaystyle \frac{m^2}{{\cos }^4\phi {\left(m^2+{\tan }^2\phi \right)}^2}}.$$

(11)

Our experimental results suggested that the Beckmann function performed slightly better than GGX, so we adopted the Beckmann function in this paper.

Figure 8. Reflection geometry.

Figure 8. Reflection geometry.

4.2. Roughness Parameter Estimation Based on Camera Images

To estimate the Beckmann roughness parameter from camera images, we consider a simple measurement setup. Figure 9 illustrates the setup for capturing a glossy flat object surface. The light source and camera are positioned above the surface, and the surface reflection forms a 1D intensity distribution curve. In this case, the angles ϕ and θ in Figure 8 are both equal to the viewing angle. In the actual system we used, the distance between the object and light source (camera) was 1160 mm, and the light source was a small LED. A digital single-lens reflex (DSLR) camera (Sony alpha 7C) with a 14-bit depth per color channel was used. The camera images were captured in the lossless SONY-ARW raw image format. The captured images of the object surface include strong glosses or highlights. Many images were captured by changing the shutter speed and then linearly combined to obtain an image without saturation and with the highest dynamic range to create a HDR image of the target surface.

Figure 10 shows the HDR images obtained from the same objects as the previous section, where panels (a) and (b) are the HDR images for the lacquer-C and -D plates and acrylic and PVC plastic boards, respectively. The image intensity is a relative value, normalized using a white reference standard with a matte surface photographed under the same conditions. Figure 11 shows the corresponding mesh representations of the luminance intensity distributions for (a) and (b). The mesh size is 512 × 512, and one pixel corresponds to approximately 0.0246°. The 2D intensity distribution is isotropic in the (x, y) plane and symmetric about each axis. Therefore, the Beckmann function in Eq. (10) can be fitted to the 1D profile of the intensity distribution.

We employed the least-squares fitting method. Let $I(\phi )$ and $f(\phi )$ be the 1D image intensity and the Beckman function, respectively, at the viewing angle ϕ. The roughness parameter m is treated as a variable included in the function in the form as $m=1/i(i=1,2,\mathrm{....})$. We search for m that minimizes the squared error

$$E={{\displaystyle \sum _{\phi }\left(I(\phi )-f(\phi )\right)}}^2$$

(12)

Figure 12 draws the error variation as a function of i for C plate, with the minimum error E=0.001997 occurring at i=349, corresponding to m=0.002865. Figure 13 shows the fitting results for the four glossy objects as functions of the viewing angle ϕ, where the black and red curves represent the intensity curve of the HDR image and the fitted Beckmann function, respectively. Thus, the roughness parameters m were estimated as m=0.00287, 0.011236, 0.002288, and 0.002222 for lacquer-C and -D plates, and acrylic and PVC boards, respectively. It can be seen that the HDR intensity curves observed by the camera closely match the Beckmann distribution with a single parameter, particularly for the lacquer-C plate, and the acrylic and PVC boards.

5. Relationship Between Measurement- and Image-Based Roughness

For all the glossy objects with flat surface in Figure 1, we estimated the surface roughness R by the physical measurement using a laser scanning system, and also estimated the parameter m to mathematically model the surface roughness based on HDR images taken by a camera. Figure 14 plots the estimated R and m values for ten objects in a two-dimensional coordinate system (R, m), where the symbols denote the following: A - E: black lacquer plates in Figure 1(a), a: black acrylic board, b: black PVC board, c: lacquer tray, d: lacquer box, and e: lacquer tower holder.

The correlation coefficient r between each R and m element pair was calculated as r=corr(R, m) = 0.9654 and also the coefficient of determination of the regression equation (13) was $r^2=0.9320$ (see [19]). These results suggest a strong correlation between the measurement-based roughness values and the image-based Beckmann roughness parameters, and the latter is well explained by the former by Eq. (13).

$$m=c_0+c_{1}R$$

(13)

The coefficients $c_0$and $c_1$ were determined by least-squares fitting to the ten estimate pairs as $c_0$=-0.0023 and $c_1$=0.0831. Note that$c_0$ represents a small bias term.

6. Practical Applications to Glossy Objects

6.1. Appearance Reproduction Using the Estimated Roughness Parameter

The Cook–Torrance model [10] was employed to reproduce the realistic appearance of glossy objects under various conditions. The spectral reflection model for the color signal, incorporating the light source and reflectance, is given by

$$C(n,v,l,\lambda )=\left(n·l\right)S_d(\lambda )E(\lambda )+c{F}_0{\displaystyle \frac{f(\phi )G\left(n,v,l\right)}{\left(n·v\right)\left(n·l\right)}}{S}_p(\lambda )E(\lambda ),$$

(14)

where the first and second terms in the right hand side represent the diffuse and specular reflection components, respectively, c is a constant, $E(\lambda )$ the spectral power distribution of the light source, $S_d(\lambda )$the spectral reflectance for the diffuse reflection component, $S_p(\lambda )$ the spectral reflectance for the specular reflection component, $F_0$ the Fresnel reflectance at normal incidence, $f(\phi )$ the Beckmann distribution function in Eq. (10), and G the geometric attenuation factor describing self-shadowing due to the microfacets (see [10]), the symbol$·$the inner product of two vectors.

We used the physical spectral renderer Mitsuba [20] to predict the reflection based on an underlying Monte Carlo simulation. The Cook–Torrance model was implemented in the renderer by Guarnera [14]. The reflectance and illuminance spectral functions were represented in 5 nm intervals within the wavelength range of 400–700 nm. A perspective camera model was used to set the field of view such that the rendered image fitted the acquired image. The location and orientation of the camera and lighting were adjusted to match the actual camera images. An output image with the size of 512 by 512 pixels was spectrally rendered. The 3D shape data of the objects acquired using a 3D scanner were input as OBJ files. The spectral reflectance $S_d(\lambda )$ and illuminant distribution $E(\lambda )$ from the target object and the actual light source were measured using a spectral colorimeter and spectral radiometer, respectively. The specular reflectance $S_p(\lambda )$ was set to 1. The attenuation factor G could be regarded to 1 (no shadowing) in the usual observation condition for a smoothed surface. The color signal C obtained at each pixel was converted to the tristimulus values XYZ and then to sRGB.

We first verify the reliability of the estimated roughness parameters in the rendered images. Figure 15(a) shows the rendered image of the lacquer towel holder using Mitsuba based on the setup shown in Figure 9. The estimate roughness parameter m=0.01859 was used. The light source was assumed to be a point light source. Figure 15(b) shows the real image for comparison. We calculated the intensity profile around the specular peak in the rendered image and compared it with the measured intensity and the fitted Beckmann function. Figure 16 compares three curves of the measured intensity distribution obtained from the original HDR image, the fitted Beckmann distribution, and the rendered image intensity distribution. The Beckmann distribution was estimated by the least squares fitting to the measured intensity distribution, and it accurately represents the rendered image’s intensity distribution.

Let us consider a curved object surface. If the surface is gently curved, it is possible to measure the surface heights in small, approximately planar regions to estimate the surface roughness. However, because the specular profile of a glossy object reflects the light reflection at the entire object, the intensity distribution of a curved surface cannot be directly obtained from the captured images, unlike in flat surfaces, to estimate the surface roughness parameter in the Beckmann function.

Look at the lacquer sake cup in Figure 1(e). The bottom of the cup is flat locally. Therefore, according to the measurement and computation procedure in Section 3, the surface roughness R was estimated as R=0.2560. The roughness parameter m in Beckmann function was subsequently estimated as m=0.0193 using the regression model in Eq. (13). Figure 17(a) shows the rendered image of the lacquer sake cup using the estimated roughness parameter in the Cook–Torrance mode, assuming the camera is placed 500 mm from the object and tilted 15 degrees from the normal, and the LED light regarded as a point light source is placed 1300 mm away and tilted 15 degrees in the opposite direction. Figure 17(b) shows the real image for comparison.

6.2. Estimation of Physical Roughness for Glossy Object with Curved Surface

The measurement-based surface roughness estimation method using a laser scanning system is primarily applicable to relatively flat and thin surfaces. On the other hand, if 3D shape data for a curved object is available, it is possible to vary the roughness parameter m of the Beckmann function, create many glossy surfaces by rendering, and predict the parameter that gives a surface appearance close to that of the glossy surface captured by a camera. Specifically, we can find a parameter that makes the intensity distribution around the specular peak closer to the actual image intensity. The physical surface roughness R can then be estimated from the regression equation.

Examples of glossy curved surfaces are demonstrated in Figure 18. Figure 18 (a) is an image taken with the front surface of the lacquer towel holder by tilting the object, where gloss appears in the grooves on the concave surface as indicated by the yellow arrow. Figure 18 (b) is an image taken with the back surface in the same condition, where gloss appears on the convex surface. To precisely render the specular reflections, we used the rough dielectric model in Mitsuba, where the Beckmann function was used to model the surface roughness.

Figure 19 (a)-(b) compare between the intensity distributions around the specular peaks obtained from the directly captured HDR images and those of the fitted rendered images, where (a) and (b) correspond to the front and back surfaces. The intensity distributions are taken orthogonally to the highlight lines. The camera and light were supposed 300 mm away from the object and 1160 mm away, respectively. The fitting results suggested that m=0.030 for (a) and m=0.025 for (b). The physical surface roughness was finally estimated as R=0.3887 for (a) and R=0.3285 for (b) from Eq. (13)

7. Conclusions

In this paper, we have studied the relationship between the surface roughness of glossy object surface on dielectric objects, such as painted objects and plastics—specifically, the physical surface roughness and the roughness parameter used in image rendering. It should be noted that even a mirror-like surface on a dielectric object exhibit microscopic irregularities. The physical surface roughness refers to a measure of the microscopic surface structure of a real object’s surface. The image-based surface roughness parameter is a modeling factor used to produce the realistic appearance of objects in computer graphics and image rendering. To analyze the surface roughness of glossy dielectric objects, we used handcrafted lacquer plates with controlled gloss levels, as well as several plastics and an actual lacquer product from everyday life.

First, we defined the physical surface roughness of a glossy object surface as the standard deviation of the surface normal, and described the computational procedure from the surface height information. A laser scanning system was used to obtain the precise surface height information of the target objects. By scanning tiny flat rectangular areas on the surface, we were able to measure the surface heights precisely on a dense grid of points.

Next, a method was developed for estimating the surface roughness parameter based on images taken of the surface with a camera. We adopted the Beckmann distribution function with a roughness parameter which was used to model the microfacet distribution describing the specular surface with a set of microfacets. We created a simple setup for capturing a glossy flat object surface. The roughness parameter was estimated by fitting the Beckmann distribution function to the image intensity distribution in the observed HDR image of a target flat surface in the least squares method.

Based on the above results, we demonstrated the relationship between measurement-based surface roughness and image-based surface roughness, and proposed a linear regression model to describe the relationship. Furthermore, we presented practical applications to glossy objects with curved surface using the relationship. The measurement-based method using a laser scanning system is primarily suitable for relatively thin and flat object surfaces. If the surface is gently curved, the surface heights can be measured in small, approximately planar areas, to estimate the surface roughness. However, 3D shape data are requited for general curved surfaces. Under this condition, we proposed a method to identify the roughness parameter that produces a rendered intensity distribution most closely matching the actual image intensity distribution, and then estimate the physical surface roughness from a regression model equation.