Virtual Reality and Digital Human Modeling for the Physical Ergonomic Analysis in Product Development in Industry: a Systematic Review

The efficacy of the product development process is measured by the ability to launch a project with product and production process specifications that could guarantee that the manufacturing can produce it with the least impact. If a problem is detected late, they bring consequences beyond the high cost of the solution, if related to physical ergonomics, which will influence the well-being of operators, productivity, and quality. Virtual Reality (VR) and Digital Human Modeling (DHM) are ones of the enabling technologies of Industry 4.0 and has already been applied on a large scale in industries such as automotive, construction, and aeronautics. However, even though the huge applications, these technologies are not yet applied by these industries for the analysis of physical ergonomics during product development phases. This study aims to characterize the state of the art and technology about the application of Virtual Reality and Digital Human Modeling for the physical ergonomics analysis in the during product development phases in the industry through a systematic review of the literature and patents. In patent documents recovery, we used Derwent Innovation database. The research is based on searching the selected terms in the title, summary, and claims of the documents through a search strategy containing IPC code and keywords. In articles recovery, we searched ScienceDirect, Springer, and IEEExplore databases for scientific publications. The search resulted in 311 patents documents and 16 articles in the scientific database. This study analyzed the patents to map out the technological progress in this area, where we found in the charts and data an increasing number of publications per year and a spread application with a considerable number of new technologies presented in these recent patents. The literature review indicated that Virtual Reality technology complements the Digital Human Modeling during physical ergonomics analysis for manufacturing process already designed. The majority of research on the use of VR and DHM Preprints (www.preprints.org) | NOT PEER-REVIEWED | Posted: 22 November 2021 © 2021 by the author(s). Distributed under a Creative Commons CC BY license. technologies for physical ergonomics analysis focus on the automotive industry and the ergonomic assessment of workstations and current processes. Further research is needed to investigate how Virtual Reality and Digital Human Modeling might assist in the understanding of physical ergonomics in certain tasks throughout the product development process, such as the simulation of worker posture or effort when assembling parts.


INTRODUCTION
The product development process is composed of several phases, one of which is the definition of the manufacturing strategy. It determines its efficiency by starting a project with product and manufacturing process specifications that assure it can manufacture it with minimal impact [1].
The late discovery of physical ergonomic issues after the release phase will have a substantial influence on the cost of the solution in this process. This is because typical production tools are almost finished at this stage of development, and modifying them is more complex and costly. In contrast, everything in the prior phases was performed in a virtual model [2].
In terms of ergonomics, finding flaws in the virtual manufacturing model early on decreases the time and cost of solution while also protecting workers from being exposed to undue risk of harm and improving workplace well-being [3]. Therefore, physical ergonomics is an essential principle in the manufacturing process since it analyzes the connection between physical aspects of humans and workplaces, including human anthropometry, physiology, anatomy, and biomechanics, among other factors [4]. As a result, everything in the workplace (tools, devices, items to be handled, etc.) must be designed in such a manner that the worker executes activities with better motions, using less energy, and decreasing and mitigating the risk of injury [4].
Virtual Reality (VR) technology can improve the analysis of manufacturing processes.
VR is defined as "a computer-generated digital environment that can be experienced and interacted with as if it were real" [5]. In turn, Digital Human Modeling (DHM) provides a quick, virtual representation of humans, likewise employing a computer to simulate a real-world situation but without interactivity [6].
Virtual simulation related to RV and DHM technologies employ computer modeling and simulation technology, which has been widely utilized for more than a half-century in education, health, entertainment, culture, sports, engineering, the military forces, and other fields. However, it has only lately become a valuable tool in the manufacturing industry [7].
Despite their numerous applications, to date, no investigation of scientific and technological knowledge bases has been undertaken on this subject. DA SILVA et al. (2020) reviewed the application of the VR to physical ergonomic analysis in the industry. ZHU at al.
(2019) reviewed the applications and research trends of digital human models in the manufacturing industry. Although the similar approach these studies did cover the joint applications of RV and DHM in the industry to support physical ergonomic analysis, hence the need for this scientific study.
Thus, this study aims to characterize the state of the art and technology about the application of Virtual Reality and Digital Human Modeling for the physical ergonomics analysis in the during product development phases in the industry through a systematic review of the literature and patents.
The following is how this document is structured. Section 2 describes materials and methods adopted. Section 3 details and examines the Results. Finally, Section 4 provides our Conclusions and recommendations for further research.

MATERIALS AND METHODS
This systematic review followed the PRISMA guidelines [8] and the method described in [9] was used, which encompasses seven steps: Planning, Scoping, Searching, Assessing, Synthetizing, Analyzing, and Writing.

PLANNING
During the Planning step, the knowledge bases that will be explored are defined. The search for document patents was undertaken in the Derwent Innovation Index database, while the search for research articles was conducted in the scientific databases ScienceDirect, Springer, IEEEXplore, and Google Scholar. Derwent Innovation Index® (Derwent World Patents Index -DWPI). DWPI is a database containing patent applications and grants, covers over 14.3 million basic inventions from almost 60 worldwide patent-issuing authorities. We carried out the technological prospecting on the Derwent Innovation Index® (Derwent World Patents Index -DWPI) database, Thomson Innovation©, with a license for use by University Center SENAI/CIMATEC. The database must be evaluated according to some essential criteria for a search focusing on patents. Paid access tools, such as Derwent Innovations Index, present as a differential the rate in retrieving the information sought, being able to analyze a large amount of data in a simplified and efficient way, allows the use of statistical tools, and promotes the generation of information through graphics, maps, among others, allowing their analysis. The efficiency of this base can be directly related to the high cost of obtaining its license.
Regarding articles, ScienceDirect, Springer, and IEEEXplore were chosen because they are reliable and multi-disciplinary scientific databases of international scope with comprehensive coverage of citation indexing, allowing the best data from scientific publications.

SCOPING
Defining the scope results in appropriately formulated research questions. A brainstorming session was held with an interdisciplinary focus group comprised of five experts on product development in the automobile industry and virtual reality, which selected two pertinent research questions to this systematic review address, namely:

SEARCHING
The Searching step involves exploring the database specified in the Planning step using a specific string based on the questions stated in the Scope step.
The search strategy was developed by a specialist in virtual reality-based market of automotive business. This researcher identified candidate search terms by looking at words in the titles, abstracts, and keywords sections of two known relevant publications. The prospective search phrases were then peer-reviewed by five additional members of our team with expertise using virtual reality for product design in the automobile sector. For the ultimate selection of relevant terms, we used DWPI's "Smart Search" resource, where one simply input text and the system expand those key terms and synonyms (https://www.derwentinnovation.com). The following final search string was used to search patents fillings in the Derwent Innovation Index database: (("Virtual Environment" OR "Virtual Reality") AND ("Human Factors" OR Ergonomics) AND ("Product Design" OR "Product Development") AND ("Digital Human Modeling") AND ("Industry")). A similar search strategy was used for article retrieval, with minor adjustments to fit the search engine requirements of each scientific database.
The search was carried out in November 2021, and the preliminary screening yielded 524 patents and 209 articles.

ASSESSING
The Assessing step employs inclusion and exclusion criteria filters to narrow down the number of documents discovered during the Searching step that are relevant to the research questions. We reduced the number of documents retrieved during the Searching step by combining the following exclusion criteria: E1: Exclude the documents not written in English language; E2: Exclude the documents published more than 6 years ago; E3: Exclude publications not related to the "industry", "industries" or "industrial" areas.

RESULTS AND DISCUSSION
The patents and papers retrieved are examined in the sections that follow.

PATENT MAPPING
The findings of the patent analysis are presented below, taking into account key information such as Publication Trend, Origin Location, International Patent Classification (IPC), and others.
The trend chart (Fig. 2) demonstrates that yearly patent publishing has grown considerably between 2000 and 2020. As of the date of this research, the 2021 result had not yet completely matured data, hence it was not considered in this analysis.   The International Patent Classification (IPC) is a method for determining a standard classification for registered patents, allowing for the search and access of technical information available in documents related to the same topic [10]. This classification facilitates the search for patent filings and allows access to the technical information included within them. Fig. 4 illustrates the distribution of patents based on the IPC classification. The most prevalent classification is G06F (computing and computation or counting), followed by H04L (Transmission of Digital Information). Industrial uses include health care, automotive, and aeronautics.
Patent applications for ergonomic features were projected to expand since they are frequent in mass production and manual production industries such as automotive that use technology to enhance process conditions. G06F, on the other side, is a concentration of technology connected to virtual simulations such as VR and DHM.
The word cloud (Fig. 5) provides a little less organized idea of the information available in the patents content. The main goal of this graphic is to offer an insight of what is being sought more frequently in terms of technology magnitude of the words.   Table 1 shows the primary characteristics observed in the sixteen articles selected by the search strategy.

Industry
The majority of articles are on the automotive industry. It may be attributed to the fact that virtual technologies have been prevalent in this segment for a long time, being employed in a variety of sectors and applications such as production, training, and maintenance, among others [4]. According to the graph in Fig. 7, the analysis of the identified articles revealed that the automotive industry accounts for 38% of the examined studies.

Type of Application
As seen in the Application Type column of Table I, the applications of VR and DHM technologies for physical ergonomic analysis focus on workplace assessment and process redesign. In the former, the purpose is to understand the ergonomic conditions of current processes and operations; in the latter, the technologies are used to develop new process concept aiming to improve the workers well-being.
For the most studies a preliminary on-the-spot inspections are performed for workplace assessments, allowing simulated data collection, observation of personnel during task execution, and description of the operational conditions to be reproduced. Then different methods, hardware and software should be used, according to the column in Table I.

Methods
The studies have used a range of methods. Some approaches, such as RULA, EAWS, NIOSH, and OCRA, have a special use for the industry for which the case study is designed, whilst others, such as Motion Analysis System (MAS), Person posture recognition (PPR), and Ergo-Uas and others have a more specific applicability.
The Virtual Assembly Process Assessment for Large Parts (VAPA-LP) approach is offered as an example of a specific application. It is a proof-of-concept prototype of a virtual environment for analyzing large-part assembly operations, namely aircraft wing riveting [11].
The ergonomic changes between the assembly of smaller mechanical components, the size of the piece, the task's complexity, and the accuracy of movement explain the method's uniqueness.
The hybrid method enabled a more engaging Virtual Reality since assessments are supported by 3D printed components and devices. Touching a physical product during virtual analysis offers a better understanding of how challenging is for workers to handle the components during assembly.
Haptic technologies have also been employed, allowing individuals to interact with computers through wearable devices by providing tactile feedback. A haptic device, such as a phantom, glove, model, or manipulator, is packed with several sensors that capture information such as movement direction and speed. These variables are properly processed, allowing the user to get input, such as vibration at specified regions with suitable frequency and amplitude [12].
The Ergonomic Assessment Worksheet (EAWS), which validates ergonomic conditions, is the general principle used by the majority of the studies examined. It is used to enable virtual analyses and is based on a holistic verification of the work process, taking into account all tasks completed in a single working day [14]. The EAWS technique provides a broad evaluation, combining various biomechanical loads that affect workers and therefore providing a baseline risk assessment [14]. This is a popular method in the European automotive industry. The articles retrieved indicate that this tool is used as a method of analysis in certain studies, but also with connections to other approaches.

The Ergo-Uas method was developed by the automobile manufacturer Fiat Chrysler with the goal of designing new workplaces. It is a combination of EAWS and Universal
Analyzing System (UAS). Its application aims to merge ergonomic aspects into process design, allowing the creation of workplaces, mostly manuals, with a human-centered approach [3].
The Human-Centered Design (HCD) method focuses on increasing human well-being, user satisfaction, accessibility, and sustainability to improve the efficacy and efficiency of processes [4]. The current production equipment designs, and processes are ergonomically evaluated in digital configurations in order to propose redesign activities centered on man.
Other tools often utilized in industry, according to the studies, are the Rapid Upper Limb Assessment (RULA) and Rapid Entire Body Assessment (REBA) procedures. Because the upper limb and the operator's complete body must be monitored in all tasks related to the activities, these are recognized of wide application in the evaluation of the assembly process.
As a result, the shoulders, wrists, elbows, neck, torso, and legs are the focus of attention [11].
Work Musculoskeletal Disorders (WRMSD) is also mentioned in studies, and it is distinguished by its focus on the most critical occupational challenges, which result in human suffering and high economic costs for organizations, among other concerns for employees and society [2] [4].
Even though just one of the examined studies employed 3D simulation tools as its primary method, this approach supports the majority of the concepts that guides recent research. 3D geometry, assembly, human factors, assembly timings, position distribution, assembly position creation, and 3D design are the foundations of this approach [15]. Person Posture Recognition (PPR) is a method for assessing postural ergonomic issues in the industry.
The primary goal of the tool is to determine ergonomic posture from a photograph using a convolutional neural network. A standard camera is used in combination with other resources as a posture capturing device for ergonomic PPR building [16].
Ovako's Work Posture Analysis System (OWAS) to identify and evaluate work postures [15] and the ISO 11226 / ISO 11228 guidelines to set global standards for poses and bodily motions [11]. The NIOSH index elevation (LI), a standard evaluation technique for handling loads, and OCRA, a set of tools that allow for various levels of risk assessment, are both part of the ISO 1128 standard and are both recognized as references by literature [16][24].
The Motion Analysis System (MAS) is a software architecture developed to evaluate human-machine interaction in the manufacturing process. It is intended to adapt to typical workplace setups, and its objective is to assess human labor and supply production management with a broad and comprehensive productivity report. A human markerless MOCAP hardware device was built to collect data from the worker's body while doing a task, and ad hoc software was developed to perform dynamic analysis to achieve the desired result [17].
Washington State Ergonomics Checklist is North America ergonomics analysis methodology created by Washington State Department of Labor and Industries to identify the presence of "Caution Zone" conditions were considered targets for corrective actions. It is widely applied to industries supporting civil construction [22].
Postural Ergonomic Risk Assessments (PERA) is a screening method used to determine simple statements about a present risk from postural stress. This method uses the concept of RULA as a reference [23].

Hardware
In terms of resources, an integrative VR system often requires more than one hardware device to function properly. The hardware can vary greatly depending on the application, ranging from a standard computer to certain display devices, motion capture equipment, and interactive gadgets (wearable devices, camera, head-mounted display-HMD etc.).
According to [3][14] some applications of virtual reality using wearable devices missing the tactile feedback during ergonomic analysis part manipulation. The mobile devices to be defined for the analysis need to take into account the workers' mobility and ability to perform operations normally [03].
This seems to be a design problem, especially when employing them for workplace design. Probably the easiest way to achieve the greatest outcomes is to blend all virtual technologies sequentially or concurrently. This will allow one to explore all of the benefits and create a virtual workplace that satisfies all of company requirements.
The studies indicate certain typical hardware such as Kinect Sensor, wearable devices, and projectors, but there are examples of 3D printing applications as a tool to support virtual analysis, in order to boost the accuracy of the results. Fig. 9 is an example of hardware used in virtual workplace analyses.

Digital Human Models Software
Digital Human Modeling (DHM), as the name implies, is a tool for human representation in a virtual environment that employs several software [6]. We observed in the reviewed studies that some software is for everyday application, while others, in addition to having particular applications, provide complementing features. SAFEWORK, DELMIA, and CATIA are examples of these, which simulate and evaluate diverse body postures and combined, create virtual mannequins that reproduce worker movements [4]. The simulations performed with DELMIA software enable the development of a database for fatigue indices, which will accelerate future analysis and facilitate information exchange.
The analyzed software is used to evaluate the ergonomic conditions of the manufacturing process, emphasizing decision support on changing the workplace layout; developing assistive assembly devices; replacing tools; special operator training; and changing routines and assembly methods.

Virtual Reality Software
According to the reviewed studies, the virtual reality software used can influence the immersion sensation provided and must be chosen according to the quality of the simulations that are expected.
In this review, the VR software described is different for each of the applications. When applied to industries with unique features, all of the methods discussed in the studies may show to be quite limiting. It is the scenario of the aircraft sector, which has large assembling parts, and the civil construction industry, which has a high turnover of operations.
The methods can be optimized in the critical analysis performed prior to application, which must consider a detailed scope of what is intended to be achieved, as well as the constraints of the available tools.
Methods using hybrid techniques, for example, given as an essential tool (where VR analysis is assisted by 3D printed components and haptic devices). However, the technology used must be carefully defined in order to not limit the worker's mobility, as the operators under observation must move naturally in order to do their tasks.
Other constraints identified relate to ergonomic analyses of the application of force with the finger when conducting tiny component assembly, as is frequent in the automobile sector.
The methods and equipment discussed are effective for posture analysis and measuring the angle of the upper and lower limbs. They are unable to quantify the forces exerted with the fingers, though. This sort of constraint might be an issue in applications in the automobile sector, as well as others with similar operations.
Finally, it was found that the majority of VR and DHM methods of physical ergonomic analysis are oriented to pre-designed production processes with all resources (devices, facilities, equipment, etc.) completely installed. There has been no study on the examination of ergonomic conditions during the product development phases. However, research pointed out that preventative analyses of ergonomic difficulties in the product design phases are critical since design revisions in this phase are simpler and less expensive [1] [11]. Doing preventative analyses of ergonomic hazards of parts while they are still in development can greatly reduce the effect of necessary adjustments. Aside, of course, from the unquantifiable consequences of worker injuries.

CONCLUSION
We found that the number of patent publications has grown significantly, rising from 8 yearly patents in 2000 to 90 in 2020. Applications were observed across several areas, such as computing, information processing, virtualization, management, blockchain, augmented reality, and three-dimensional. This range of technologies highlights current advancements and indicates the industry's commitment to developing innovative solutions.
The Virtual Reality technology and Digital Human Modeling during physical ergonomics assessment with a variety of software and hardware, that support particular applications for various industries.
Most studies on the use of VR and DHM for physical ergonomic assessments have focused on the automotive industry. Similar technologies are used as a tool for ergonomic evaluation in other industries with multiple processes and features, such as aircraft, welding, and civil construction.
The applications of VR and DHM as technologies for physical ergonomic analysis are intended to evaluate and improve workplace ergonomic conditions for known industrial processes, while also taking into consideration simulations that aim to optimize cost and productivity without compromising safety.
Further research is suggested on the use of VR and DHM to support the assessment of physical ergonomics of more specific tasks, such as the simulation of worker posture or effort when assembling parts, throughout the product development cycle.