How Well Mobile Phone App DecibelX measures Noise and the Usability Study

This study aims to assess using a smartphone app (DecibelX), as a noise measuring alternative to the more costly traditional use of measuring noise levels with a Sound Level Meter (SLM). The study compares the accuracy of the app to readings taken with an SLM and dosimeter. It also evaluates the app’s performance for pure tone and narrow band noise. The usability study identifies strengths and weaknesses related to usability of the app.


Introduction
One of the most prevalent workplace hazards today is noise exposure and the subsequent life-impacting hearing loss that can result. According to the CDC, Center for Disease Control, more than 11% of the working population experiences hearing loss, with approximately one in four of these hearing losses being work-related [1]. Because noise exposure poses such a significant health risk to the working population, it is important to accurately monitor and control occupational noise exposure. Noise exposure can be controlled through engineering controls (example: reduce noise level with sound barrier), administrative controls (example: limit total exposure time spent by workers in high noise areas), and PPE/Personal Protective Equipment (i.e., wear hearing protection). The process of monitoring and controlling noise exposure begins with accurately measuring the noise levels to which workers are exposed.
The most common means to measure occupational noise exposure is an SLM, Sound Level Meter, or dosimeter [2]. An SLM measures an instantaneous amount of noise whereas a dosimeter measures the "daily dose" by finding Equivalent Sound Levels (Leq). Leq is a constant noise level which transmits the same acoustical energy as varying noise levels over a defined period of time. SLM and dosimeter devices can cost thousands of dollars and may not be practical for small companies or individuals to purchase and use. To overcome this challenge, workers and some safety professionals have begun using personal smartphones to measure noise exposures in the workplace. The ability of workers to measure their own noise exposures appears advantageous [3]. However, the accuracy of these measurements may depend on the individual app the worker chooses as well as the make and model of the user's smartphone. An additional challenge is the fact that noise measurement apps and smartphones are constantly evolving/changing. Whenever a smartphone app is used in place of a sound level meter or dosimeter, there are questions that need to be answered.
1. How accurate is the device/application compared to that of the sound level meter or dosimeter? 2. If there is a difference in accuracy, is the difference of any practical significance? 3. Does the device/application accurately measure sound levels at different frequencies.
The answers to these three questions will enable us to better assess the effectiveness of measuring noise levels using a smartphone app.
In this noise study, experimental data was collected in a laboratory setting. Known sound levels (pure tone and narrow band noise levels, documented by an SLM) were measured with the DecibelX smartphone app (one case using the internal mic, another case using an external mic) and with a dosimeter. The sound levels measured with the smartphone app were compared to SLM-measured values and compared to the Leq readings of the dosimeter. To better understand the usability challenges of a wide range of users using a smartphone app to measure noise levels, a usability study was also conducted.
The results of this project provide insights into the accuracy of measuring noise levels with a smartphone app, and a provide a better understanding of the usability challenges faced by users in the workplace relying on this type of technology to self-measure their individual noise exposures.

Calibration and Data Collection for Noise Study
Before data was collected the sound level meter, dosimeter, phone application, and external microphone were calibrated. The sound level meter was calibrated using a device emitting a constant tone of 110 dB (decibels) @ 1000 Hz (Hertz). In response to the constant tone, the sound level meter read 110 dB indicating acceptable calibration. See figure 1.  The smartphone app, DecibelX, was calibrated using the built-in calibration feature of the app. For the smartphone with an internal mic, the app calibration feature was adjusted until the decibel readout of the app matched the reading of the SLM. For the smartphone with an external mic, the external microphone was calibrated by adjusting the gain (+/-) until the app readout (in dB) matched the SLM readout.
After calibration, the team began collecting data. To eliminate outside noise, data was collected inside of a sound-proof booth in one of Auburn University's audiometry labs. These sound proof booths virtually eliminate sounds from outside sources. The devices included in the data collection were the SLM, smartphone with external mic, smart phone with internal mic, and the dosimeter. All four devices were positioned at an equal distance and consistent position/orientation from the speaker emitting the test sound levels. See figure 3.
Data was collected using two types of noise: pure tone and narrow-band. Each noise type was then tested at 1000Hz, 2000Hz, and 4000Hz frequencies. Two decibel levels were tested for each noise type at each of the three frequency levels, for a total of 12 test conditions. These decibel levels ranged from 85-96 dB for the narrow-band noise conditions, and 95-100 dB for pure tone conditions.

Data Collection for Usability Test
The most important part of assessing usability is collecting user feedback and assessing that feedback. The usability testing for the DecibelX app was conducted under the assumption that workplace noise measurements are often taken by individuals with minimal knowledge of noise science and minimal knowledge of noise measurement equipment. To conduct user testing a procedure was created that included brief training on calibration, selecting of correct weight scales, pausing and unpausing data collection, resetting the data, pulling up a summary report and reading a value (instructed to read either dose or Leq), how to export data, and microphone positioning/orientation. This training was followed by a "test" where the users performed a list of instructed actions/steps. These steps included calibrating the application (+/-a specified dB correction magnitude), selecting the correct decibel scale, selecting either dose or maximum/average, starting the measurement, and stopping the measurement. The users were timed on how long it took them to complete the test. Once users were finished, they completed a subjective survey. This survey included questions pertaining to the app's usefulness (one question), ease of use (two questions), ease of learning (two questions), and user satisfaction (two questions). The survey questions also encompassed particulars of using the app. These detailed particulars rated by usability subjects included: 4. The application makes it easier to measure noise 5. The application is easy and simple to use. 6. The application requires the fewest steps possible to accomplish the required task. 7. I learned to use the application quickly. 8. I can easily remember how to use the application. 9. I am satisfied with the application. 10. I would recommend this application to a co-worker. The users were asked to rate each one of these particulars on a 5-point rating scale. 1 -Strongly Disagree, 2 -Disagree, 3 -Neutral, 4 -Agree, and 5 -Strongly Agree. In addition to these ratings, users were asked for any opinions of and recommendations for the DecibelX application.
This section may be divided by subheadings. It should provide a concise and precise description of the experimental results, their interpretation as well as the experimental conclusions that can be drawn.

DecibelX Noise Study
Sound level readings were recorded for each equipment type (smartphone with internal mic, smartphone with an external mic, and dosimeter) for two sound types (pure tone, and narrow band noise) at three frequencies (1000 Hz, 2000 Hz, 4000 Hz) and at two different sound levels (95 dBA and 100 dBA) for each frequency. In total, there were 12 combinations for sound type, frequency, and sound level.
The detailed data collected for the pure tone noise is provided in Figure 4. The Actual is the value measured with the SLM at the pure tone frequencies of 1000 Hz, 2000 Hz, or 4,000 Hz. The values measured for the cases of with microphone, without microphone, and dosimeter are provided as well.   Figure 6 (below) provides a graphical summary of the data in Figure 4 (for the differences between actual and measured pure tones in parenthesis). The following observations were made for measuring pure tone noise with the smartphone app, without an external mic.
• The average measurement, using all six test conditions for pure tone noise, was 6.53 dBA greater than the Actual values. • For two of the six test conditions, (2000 Hz, 95 dBA) and (4000 Hz, 100 dBA), the measured value was less than the Actual value. See Figure 4. • It should be noted that for one of the individual test conditions (4000 Hz, 100 dBA), the measured value was slightly less (1 dBA less) than the Actual value. The only test condition where the measurement was substantially less than the Actual value was (2000 Hz, 95 dBA). In summary, five of the six measured values were approximately equal to or greater than the actual SLM sound level. • This data supports a conclusion that a smartphone without a microphone using the DecibelX app will overstate the sound level, by 6.53 dBA on average.

Pure Tone Noise: Smartphone with External Mic
The following observations were made for measuring pure tone noise with the smartphone app, with an external mic.
• The average measurement, using all six test conditions for pure tone noise, was 6.15 dBA greater than the Actual value. • Only one test condition (2000 Hz, 95 dBA) recorded a measurement less than the Actual value. Note that this value was only 1.9 dBA less than the actual 95 dBA value. For the remaining five test conditions, the measured values were greater than Actual values.

Pure Tone Noise: Dosimeter
The following observations were made for measuring pure tone noise using a dosimeter.
• Only one test condition (2000 Hz, 100 dBA) resulted in a dosimeter measurement greater than the Actual value. It would be expected for dosimeter measured values to be less than Actual values, since the dosimeter is averaging the sound levels over time. This was the case for five of the six test conditions. • On average, for all six test conditions, the dosimeter measurement was 1.1 dBA less than the Actual value. Only one test condition (2000 Hz, 100 dBA) had a dosimeter reading greater than the actual value. If this value were removed from the dataset, it is interesting to note that the average dosimeter measurement for the remaining five test conditions is 3.52 dBA less than the Actual values. The detailed data collected for the narrow band noise level is provided in Figure 7. The Actual is the value measured at the pure tone frequency (1000 Hz, 2000 Hz, or 4,000 Hz) with the SLM. The values measured for the cases of the smartphone app with microphone, without microphone, and also measured by dosimeter are included in Figure 7. Also, Figure 7 presents differences between the narrow band noise measurements taken with the smartphone app and the Actual values (in parenthesis). Also, note the difference between the dosimeter measured values and the Actual values.    Figure 7 (for the differences between measured and actual narrow band tones in parenthesis).

Narrow Band Noise: Smartphone without External Mic
The following observations were made for measuring narrow band noise using the smartphone app, without an external mic.
• The average measurement, across the six test conditions, was 3.28 dBA greater than the Actual (SLM-measured) value. • This data supports a conclusion that a smartphone without an external microphone using the DecibelX app will overstate the sound level, by 3.28 dBA on average, when measuring narrow band noise.

Narrow Band Noise: Smartphone with External Mic
The following observations were made for measuring narrow band noise using the smartphone app, with an external mic.
• For all six test conditions the measurement exceeded the Actual value.
• The average measurement, across the six test conditions, was 2.75 dBA greater than the Actual (SLM-measured) value. • This data supports a conclusion that a smartphone with an external microphone using the DecibelX app will overstate the sound level, by 2.75 dBA on average, when measuring narrow band noise. Narrow Band Noise: Dosimeter The following observations were made for measuring narrow band noise using a dosimeter. • The dosimeter reading was less than the Actual value for all six of the test conditions. • The average dosimeter reading, across all six test conditions, was 3.58 dBA less than the Actual value. • This data supports that a dosimeter will understate the sound level, by 3.58 dBA on average, when measuring narrow band noise.

Statistical Analysis:
The ANOVA (Analysis of Variance) calculations for the data collected are summarized in Figure 10.

Figure 10. Analysis of Variance
Note that the above ANOVA differentiates between the phone with and without the external mic, between tone and noise, and between frequencies exactly, but that the loudness (computed as the difference as displayed by the SLM) was given qualitatively as "high" and "low" loudness levels for every factor combination. A confidence level of 0.95 (α = 0.05) was chosen for the analysis.
According to the ANOVA, only the distinction between tone and the N-b (narrow band) noise was significant in performance, with difference in phones being completely negligible after the internal mic only phone correctly calibrated. However, error during runtime of the experiment (most likely from not exactly replicating the loudness levels between treatments) could be present, and it is possible that results may indicate that both loudness and frequency do have a significant effect. Future studies should aim to control factor levels more closely and perform additional replications to gain greater confidence in the results.

DecibelX Usability Test
The usability test was administered to ten subjects. The results are presented in Figure 11 below.

Figure 11. Usability Test Results Summary
Viewing Figure 11, the following observations can be made about the usability study. • The highest average response across the ten subjects (4.5 out of a possible 1-5) was for Question 5, I easily remember how to use it, and Question 7, I would recommend it to a coworker or employee. • The lowest average response across the ten subjects (4.5 out of a possible 1-5) was for Question 2, The app is easy/simple to use. • The average response for all 7 Questions exceeded 4.0 (i.e., fell between 4.0, Agree, and 5.0, Strongly Agree). • Interestingly, observing Figure 11, it can be seen that two of the ten subjects had relatively low ratings across all seven questions. Subject 5 had a 2.9 average rating across all five questions, and subject 10 had a 2.7 average rating across all five questions.
• Eight of the ten subjects had relatively high ratings, rating all questions (with one exception -Subject 6 rating one item a 3 out of 5) either a 4, Agree, or 5, Strongly Agree. Specific user feedback is outlined in Figure 12, user Comments Summary. The following observations can be made about user comments. • The most frequent user comment (four occurrences) was Buttons too small/too easy to get wrong input. • The next most frequent user comments (each with two occurrences) were Desirable to have a tutorial available and Desirable to color/label buttons. • There were two additional user comments (each with one occurrence): Decibel slider easier to accurately change and Increase font size of summary report.

Noise Study
The results from the data analysis show a clear distinction between the decibel levels read by the phone with an external microphone and the phone without and external microphone when compared to the actual decibel level. The application performed better with the external microphone to measure narrow band noise rather than not using an external microphone with a difference of 0.53 dBA between the two conditions. For pure tones, the application performed better without the external microphone than with it, resulting in a difference of 0.38 dBA between the two conditions. As far as frequency, the application performed better in the 2000Hz range for both narrow band noises and pure tones. When comparing the performance of narrow band vs. Pure tone and external microphone vs. No external microphone, the best results happen with a combination of measuring narrow band noises with an external microphone in the 2000Hz range. For all test conditions, the application performed better when measuring lower decibel levels.
To conclude, the results appear to show that the use of the decibelX application in lieu of a sound level meter to perform noise tests is not recommended. However, when a sound level meter is not available, the application could be useful for a quick check to see if the noise level was in a specific range. For example, the noise levels across all conditions are higher than the actual value read by the sound level meter, so a safety professional could use the application to indicate acceptable sound levels as long as the necessary value is below that of which the decibelX application indicates. If you are to use the decibelX application to measure noise, it is best to do so in the 2000Hz range at approximate decibel levels lower than 90 dBA for narrow band and 95 dBA for pure tones and while using an external microphone that is correctly calibrated.

Usability Study
The user ratings and comments were quite informative. Generally, it appears that the ten subjects had a positive impression of the DecibelX app. Evidence of this is the average answer to all seven usability questions falling between 4, Agree, and 5, Strongly Agree. Across the board, users provided favorable ratings.
The user comments indicated improvements which could be made to the app. Four users commented, Buttons were too small/too easy to get wrong input. This type of comment would appear to apply to the smaller control buttons such as the calibration slider and the scale selection slider. Upon inspecting the app user interface, these controls (both slider controls) are small in size, and obscured/covered by the finger as the finger slides/moves the control (Figure 13). In addition to not being able to see the control itself, of more relevance might be that the user can't see the level/status of the control as the finger moves it.

Figure 13. Finger Obscures Slider Control
Another relevant usability design aspect of complaints of buttons or controls being "too small" can be understood by Fitt's Law (Figure 14). Fundamental implications of Fitts's Law (for app interface design) are that larger objects are easier to accurately select with the finger than are smaller objects. Additionally, due to the nature of the function of Fitts's Law, small increases in size of small objects make a big difference (i.e., make it much easier to select the object). While small increases in size of a larger object simply does not make much of a difference. It would appear that user complaints of buttons and controls being "too small" are implications of Fitts's Law not being properly taken into account in the design of the decibelX app interface (Figure 15).
The buttons most complained about were the calibration, export, and reset buttons along the middle-left of the interface, as well as the weight scale slider along the middle-right. Most observed error was from subjects attempting to press one of the above functions, missing, and being surprised and confused at receiving an unrelated function. An increase in the size of these keys should dramatically reduce the amount of errors during operation.