An Auditable Downstream Safety-Intelligence Framework for Manufacturing Workplaces: Reliability-Aware PPE Evidence, Regulation-Grounded Retrieval, and Role-Specific Decision Support

1. Introduction

In high-risk industrial environments, including manufacturing and construction, considerable research has focused on the automated recognition of worker behaviors, accident situations such as falls and fires, personal protective equipment (PPE) use, and other unsafe conditions using CCTV and computer vision [1,2]. Deep-learning-based object detection, pose estimation, tracking, and scene analysis have extended industrial safety surveillance toward continuous and quantitative visual monitoring, and recent studies have identified computer vision and artificial intelligence (AI) as key technologies for hazard detection, PPE monitoring, and proactive safety management [2,3,4]. However, the practical use of AI for industrial safety does not end with accurately detecting or classifying hazardous situations. Information generated by different perception sources must also be associated with the same safety event and transformed into safety information that managers and workers can review and use while preserving the reliability and uncertainty of the underlying evidence. Accordingly, an important next challenge in vision-based safety monitoring is not only to improve perception performance, but also to determine how the resulting multimodal safety evidence can be transformed into reliable, traceable, and operationally usable safety intelligence.

Industrial safety monitoring has increasingly moved beyond single-frame object recognition toward contextual and temporal understanding. Approaches have been proposed that use Vision-Language Models (VLMs) to jointly interpret global-scene and region-level safety information [5,6], exploit temporal evidence to reduce frame-level false alarms [7], and associate worker identity with PPE status to construct continuous safety information [8]. Although these developments provide richer safety evidence, high accuracy or F1-score values, or contextual reasoning capability alone, do not ensure that the event identity, source provenance, and uncertainty of heterogeneous perception outputs remain consistent throughout downstream processing. Rather than collapsing heterogeneous perception outputs into a single final label, an event-level representation is therefore needed to structurally preserve accident-related, PPE-related, semantic, and routing evidence under a common event identity.

This reliability issue is particularly important in PPE monitoring. In real industrial scenes, PPE detection can become unstable because of occlusion, scale variation, worker overlap, movement, and posture changes [9]. Recent studies have therefore sought to address the limitations of single-frame detection by associating worker identity with PPE status or incorporating temporal evidence [7,8,10]. In safety-critical monitoring, however, an observation that “PPE was not detected in a particular frame” should not necessarily be treated as equivalent to the operational claim that “the worker was not wearing PPE.” In particular, under conditions of reduced visual reliability, such as fall-like postures or partial occlusion, evidence reliability must be managed so that transient non-detection is not escalated into a more definitive claim [11]. PPE evidence should therefore be interpreted within a worker-centered temporal context, while the certainty and component specificity supported by the source evidence should be preserved in downstream information.

Industrial safety monitoring also cannot be considered independently of its regulatory context. Responsibility for industrial accidents has been addressed not only in terms of immediate workplace safety measures but also from the perspectives of organizational safety governance and management responsibility. For example, the United Kingdom’s Corporate Manslaughter and Corporate Homicide Act 2007 treats the manner in which an organization’s activities are managed or organized, together with the involvement of senior management, as important elements of organizational liability in cases involving serious fatalities [12]. In Korea, the implementation of the Serious Accidents Punishment Act (SAPA) has likewise strengthened corporate responsibility and the safety-management duties of responsible managing officers [13,14]. In light of these developments, AI-based safety monitoring should support not only hazard detection but also the review of regulatory information that may be relevant to an observed event. Retrieval-Augmented Generation (RAG) has emerged as an approach for grounding generated information by retrieving external knowledge sources [15], and studies have begun to connect regulatory knowledge with RAG in construction safety [16]. However, retrieving regulatory information must be distinguished from automatically determining the violation or applicability of a specific provision, legal responsibility, or penalties, and practical deployment requires attention to source traceability and human oversight [17,18]. Safety-critical decision support should therefore link observed safety evidence to potentially relevant candidate regulatory references in a reproducible and provenance-preserving manner, while leaving final legal judgment to human review.

Explainability alone is also insufficient when safety information is ultimately delivered to users. Trustworthy AI encompasses not only explainability but also robustness, transparency, accountability, traceability, and auditability [19,20]. Human-centered decision support likewise emphasizes maintaining human oversight by providing information appropriate to users’ roles and operational purposes rather than replacing human judgment [21]. In industrial safety, a manager may need to review event evidence together with regulatory context, whereas a worker may require more immediate hazard and protective-action information for the same source event. Role-specific output should therefore be viewed not simply as producing longer or shorter versions of the same report, but as an information-boundary problem concerning which parts of the same evidence should be communicated, and to what extent, to each audience. When natural-language generation is used, it should also be possible to verify that the generated output does not strengthen claims beyond the source evidence, that retrieved references are preserved, and that information from different events or audiences is not mixed. The importance of such source-to-output provenance and auditability has also been emphasized in other safety-critical decision-support domains [22].

Taken together, prior research has advanced vision-based hazard recognition, PPE monitoring, temporal and contextual safety analysis, regulatory knowledge retrieval, and trustworthy decision support as important but largely distinct research directions [2,9,15,19]. However, placing these capabilities within a single application is different from explicitly managing the evidence lifecycle from perception to final decision-support output. In particular, comparatively less attention has been given to jointly structuring heterogeneous frame-level evidence under a common event identity, preserving PPE-related uncertainty throughout downstream processing, separating the provenance of visual evidence from that of regulatory references, applying different audience-specific information boundaries to the same event, and independently verifying this overall transformation. Accordingly, rather than proposing a new standalone perception architecture or a general-purpose legal reasoning model, this study uses previously trained and evaluated perception outputs as frozen multimodal safety evidence and proposes an auditable downstream safety-intelligence framework that transforms this evidence into operational safety decision support.

The main contributions of this study are fourfold. First, we construct a structured event representation that associates accident-related, PPE-related, semantic, and routing evidence with the same event identity, thereby avoiding the forced integration of heterogeneous evidence into a single classification label. Second, we establish a reliability-aware PPE evidence policy that distinguishes Candidate, Confirmed, and accident-related uncertainty based on worker-centered temporal PPE observations and model-derived accident conditions, and we apply semantic grounding so that downstream language does not exceed the component specificity or certainty supported by the source evidence. Third, we perform deterministic event-to-query routing from the same structured event evidence and independently retrieve event-specific Direct regulatory candidates and Governance context from a frozen regulatory corpus, without using the retrieved information for automated legal determination. Fourth, we transform the same source event into manager-oriented and worker-oriented safety information under distinct audience-specific information boundaries, and independently verify PPE semantic consistency, regulatory-reference preservation, audience/event integrity, and worker-facing legal-information isolation through end-to-end validation separated from report generation. Through these contributions, this study extends vision-based industrial safety monitoring from hazard recognition toward evidence-grounded, reliability-aware, regulation-grounded, role-specific, and auditable safety decision support.

2. Related Work

2.1. Vision-Based Industrial Safety Monitoring and Multimodal Safety Evidence

Computer-vision-based industrial safety monitoring has evolved toward the automated identification of workers, equipment, unsafe behaviors, and hazardous conditions from images and videos. Research initially focused on object detection, tracking, and action recognition, but has since expanded to PPE monitoring, fall-risk recognition, and worker–equipment interaction [23,24,25]. In manufacturing environments, multi-camera-based factory safety monitoring [26] and CCTV-based industrial accident detection [27] have also been proposed, along with approaches that address multiple safety tasks within a unified computer-vision monitoring framework [28]. These developments indicate that industrial CCTV can serve not only as a recording medium but also as a continuous source of safety-related visual evidence.

Recent research has increasingly moved beyond individual object detection to incorporate spatial, temporal, and semantic context. Representative examples include PPE interpretation using worker spatial context and depth information [29], spatiotemporal risk analysis based on multi-camera tracking and trajectory prediction [30], mitigation of frame-level false alarms through temporal windows [7], and the association of worker re-identification with PPE status [8]. Multimodal VLM-based approaches have further enabled context-aware safety interpretation [31,32], while semantic reasoning that combines global-scene and region-level evidence has extended safety evidence into natural-language information [6].

Through these developments, the analytical focus of industrial safety monitoring has expanded from individual objects and frames toward worker-, event-, and context-centered evidence. However, prior work has primarily emphasized detection or classification performance, temporal stability, or the quality of semantic outputs. How heterogeneous perception outputs should preserve a common event identity, reliability level, and provenance throughout downstream safety-management processes therefore remains a distinct evidence-management problem.

2.2. Reliability-Aware PPE Monitoring and Uncertainty Handling

PPE monitoring has evolved from the detection of individual items such as helmets and safety vests toward worker–PPE association and proper-use assessment [33,34]. Non-compliance identification based on geometric relationships between workers and hardhats [35], together with systematic reviews of PPE monitoring [9], has highlighted the effects of posture, occlusion, camera conditions, and worker identification on practical PPE monitoring. Recent studies have also addressed challenging conditions such as small-scale targets and occlusion [36,37]. Approaches including pose-guided body-region anchoring [38] and worker-specific PPE assignment under occlusion [39] explicitly account for the relationship between a PPE object and the corresponding worker.

Temporal continuity is also important in video environments because movement, viewpoint changes, and temporary occlusion can cause the observed PPE state of the same worker to vary across frames. Previous studies have addressed this problem through PPE status tracking based on worker re-identification [8,40], helmet-state stabilization using preceding-frame information [41], and the mitigation of short-term detection dropout through temporal consistency [42]. Temporal stabilization, however, should be distinguished from preserving evidence uncertainty. In particular, immediately escalating transient PPE non-detection caused by fall-like postures or severe occlusion into persistent non-wearing or non-compliance can produce a claim stronger than the underlying source evidence supports.

Research has also sought to extend PPE monitoring into semantic information. For example, semantic PPE comparison using pose-based worker-region extraction and image captioning has been proposed [43]. Such capabilities can transform PPE evidence into human-readable information; however, when component identity or visual reliability is uncertain, the specificity and certainty of generated claims should not exceed what the source evidence supports. Accordingly, downstream safety decision support should preserve not only PPE detection results but also the reliability and uncertainty of worker-linked temporal evidence.

2.3. Regulation-Grounded Retrieval for Industrial Safety Decision Support

Industrial safety management requires the consideration of observed hazards together with relevant regulations, safety requirements, and management duties. Research has therefore explored the transformation of regulatory text into machine-readable representations for automated compliance checking, including semantic and ontology-based rule representation [44], NLP- and machine-learning-based compliance approaches [45], and relation extraction and knowledge-graph representation of safety regulations [46].

Even before the emergence of RAG-based approaches, NLP had been applied to automatically retrieve and associate safety requirements from project documents [47]. The development of RAG has enabled regulatory knowledge to be connected more directly with safety decision support. Approaches have included grounding generated outputs through evidence retrieved from external corpora [15], multi-hop retrieval over OSHA regulations for construction safety [16], and onsite safety assistants that connect observed hazards with safety regulations [48]. Regulatory information has consequently evolved from an independent document-search resource toward a knowledge-grounded resource that can be associated with visual or structured safety evidence.

Regulatory retrieval, however, should be distinguished from legal determination. In Legal RAG, source traceability and provision-level citation have been discussed as important requirements [18]. More recent work on legal-domain language models has also explored the use of RAG to support more trustworthy outputs [49]. Retrieval configuration has additionally been reported to affect factual consistency [50]. Furthermore, practical applications of automated regulatory compliance continue to emphasize the importance of human oversight rather than fully autonomous determination [17]. Industrial safety decision support should therefore provide relevant regulatory articles as traceable candidate context without automatically determining the legal applicability of a provision, the existence of a violation, responsibility, or applicable penalties.

2.4. Auditable and Role-Specific AI Safety Decision Support

In safety-critical AI, human oversight, transparency, accountability, and traceability are treated as important system-level requirements alongside model accuracy and explainability [20]. Human-Centered AI research likewise emphasizes connecting data, models, and decision logic to human oversight rather than replacing human decision-makers [21]. Explanation alone has also been shown not to automatically guarantee appropriate user understanding or trust [51]. In human–AI collaboration, over-reliance on automated recommendations and automation bias have additionally been discussed as distinct human-oversight concerns [52].

The purpose and level of information required from the same AI evidence can differ across stakeholders. Research on stakeholder-tailored explanations has shown that user groups may require different levels of detail and supporting evidence [53]. In occupational safety, managers may require risk and management-related evidence, whereas frontline workers may benefit more from immediate feedback and actionable guidance [54]. Role-specific decision support should therefore consider audience-specific operational purposes and information boundaries rather than merely providing detailed and abbreviated versions of the same report.

When natural-language generation is involved, consistency between source evidence and generated claims is also important. LLM-based generation can produce information that conflicts with or is unsupported by the source [55], and in safety-critical contexts, temporary uncertainty should not be escalated into a definitive safety claim or transformed into a stronger legal interpretation than the retrieved regulation supports. Traceability approaches have therefore been proposed to record data, process, and artifact provenance [56,57], while RAG audit trails have been used to connect retrieved evidence with final recommendations [22]. These studies indicate that, independently of the linguistic quality of a final output, source–event correspondence, reference preservation, and evidence-to-output lineage should also be subject to verification.

2.5. Research Gap and Positioning of the Present Study

Taken together, the literature reviewed in Section 2.1, Section 2.2, Section 2.3 and Section 2.4 shows that industrial safety AI has advanced in worker-centered temporal PPE monitoring [8,41], VLM-based contextual safety reasoning [6], RAG systems connecting visual evidence with regulatory knowledge [58,59,60], stakeholder-specific information design [53], and auditable RAG-based decision support [22]. The research gap addressed in this study therefore does not arise from the individual absence of these capabilities or merely from whether VLM and RAG components can be combined.

The central issue addressed in this study is the evidence lifecycle from perception to final decision-support output. More specifically, the problem involves structuring heterogeneous visual evidence under a common event identity, preserving accident-related ambiguity and PPE reliability in downstream language, separating the provenance of visual evidence from that of candidate regulatory references, applying distinct information boundaries for managers and workers to the same source event, and independently verifying these transformations. Prior studies have made important advances in each of these areas; the present study focuses on jointly managing event identity, evidence reliability, regulatory provenance, audience-specific information boundaries, and final-output integrity within a single frozen downstream analytical process.

Table 1. Compact comparison of representative studies related to the positioning of the present study.

Study	Primary focus	Evidence / regulatory emphasis	Role-specific / provenance emphasis
Cheng et al. [8]	Worker Re-ID and PPE monitoring	Worker identity and temporal PPE history	Role-specific policy and full downstream audit not primary
Zhang et al. [41]	Helmet state tracking	Temporal stabilization under occlusion and missed detection	Downstream uncertainty/provenance not primary
SafeVision [6]	Context-aware VLM safety reasoning	Scene- and region-level semantic grounding; regulatory retrieval not primary	Human-readable/VQA output; explicit manager–worker boundary and full downstream audit not primary
Guo et al. [58]	Visual-LLM and RAG for safety compliance	Context-aware visual reasoning with safety-rule retrieval and compliance analysis	Retrieval/compliance evaluation; explicit manager–worker boundary and full evidence-lifecycle audit not primary
Katsampiris et al. [59]	MLLM and RAG for manufacturing risk assessment	Structured multimodal inference with ISO/OSHA grounding	Citation verification and structured reporting; explicit manager–worker separation not primary
Orooje et al. [60]	Trustworthy VLM hazard detection with RAG	Visual grounding, hallucination mitigation, and regulation-aware verification	Traceable compliance support; cross-audience integrity not primary
Kim et al. [53]	Stakeholder-tailored XAI	Stakeholder-specific explanation rather than industrial perception/regulatory retrieval	Explicit stakeholder-specific information design
Alu and Oluwadare [22]	Auditable RAG-based decision support	Source-grounded domain-knowledge retrieval	Explicit source verification and audit trail
Present study	Downstream industrial safety-intelligence framework	Worker-centered PPE reliability and independent Direct/Governance regulatory retrieval without automated legal determination	Independent manager/worker information boundaries with frozen evidence/reference provenance and separated end-to-end verification

Table 1. Compact comparison of representative studies related to the positioning of the present study.

Study	Primary focus	Evidence / regulatory emphasis	Role-specific / provenance emphasis
Cheng et al. [8]	Worker Re-ID and PPE monitoring	Worker identity and temporal PPE history	Role-specific policy and full downstream audit not primary
Zhang et al. [41]	Helmet state tracking	Temporal stabilization under occlusion and missed detection	Downstream uncertainty/provenance not primary
SafeVision [6]	Context-aware VLM safety reasoning	Scene- and region-level semantic grounding; regulatory retrieval not primary	Human-readable/VQA output; explicit manager–worker boundary and full downstream audit not primary
Guo et al. [58]	Visual-LLM and RAG for safety compliance	Context-aware visual reasoning with safety-rule retrieval and compliance analysis	Retrieval/compliance evaluation; explicit manager–worker boundary and full evidence-lifecycle audit not primary
Katsampiris et al. [59]	MLLM and RAG for manufacturing risk assessment	Structured multimodal inference with ISO/OSHA grounding	Citation verification and structured reporting; explicit manager–worker separation not primary
Orooje et al. [60]	Trustworthy VLM hazard detection with RAG	Visual grounding, hallucination mitigation, and regulation-aware verification	Traceable compliance support; cross-audience integrity not primary
Kim et al. [53]	Stakeholder-tailored XAI	Stakeholder-specific explanation rather than industrial perception/regulatory retrieval	Explicit stakeholder-specific information design
Alu and Oluwadare [22]	Auditable RAG-based decision support	Source-grounded domain-knowledge retrieval	Explicit source verification and audit trail
Present study	Downstream industrial safety-intelligence framework	Worker-centered PPE reliability and independent Direct/Governance regulatory retrieval without automated legal determination	Independent manager/worker information boundaries with frozen evidence/reference provenance and separated end-to-end verification

Consistent with this positioning, the present study does not propose a new detector, VLM, or general-purpose RAG architecture. Instead, previously trained and evaluated multimodal perception outputs are used as frozen safety evidence and transformed into event-level structured evidence. Reliability-aware PPE processing and regulation-grounded retrieval are then performed independently from the same event evidence. While maintaining the same event identity, manager- and worker-oriented information boundaries are subsequently applied, and source–event correspondence, PPE semantic consistency, regulatory-reference preservation, and audience/information integrity are evaluated through validation separated from generation.

Accordingly, the research question is not whether perception accuracy can be further maximized or automated legal reasoning can be strengthened, but whether already generated multimodal safety evidence can be transformed into regulation-grounded, role-specific, and auditable safety information without compromising its reliability, uncertainty, or provenance. The validation in this study is therefore limited to evidence preservation, retrieval reproducibility, and information integrity; it does not evaluate the final legal applicability of retrieved regulations or actual manager/worker comprehension, decision quality, or usability. In addition, because the study uses a fixed internal benchmark with historical provenance and a Korean industrial-safety corpus, the results should not be interpreted as demonstrating independent external generalization.

3. Materials and Methods

3.1. Overall Study Framework

This study proposes an auditable downstream safety-intelligence framework for transforming multimodal safety evidence generated from manufacturing-workplace video into information that can support practical safety-management decision making. Rather than developing a new perception architecture, the outputs of a previously trained and evaluated perception backbone were fixed as frozen multimodal safety evidence and connected to structured event evidence, reliability-aware PPE grounding, regulation-grounded retrieval, and role-specific safety decision support. As illustrated in Figure 1, the overall framework consists of frozen upstream perception, downstream safety-intelligence processing, role-specific decision support, and end-to-end audit and validation.

Frame-level multimodal safety evidence generated by the frozen upstream perception component was transformed into structured event evidence at the video/event level. This representation associates event and risk states with accident-related, PPE-related, semantic, and routing evidence under a common event identity. Two downstream pathways were then applied independently to the same structured event evidence. The first pathway preserves the reliability and uncertainty of worker-centered temporal PPE evidence and performs semantic grounding to constrain downstream PPE-related language. The second pathway converts event evidence into deterministic regulatory queries and retrieves relevant references from a frozen regulatory corpus. Regulation-grounded outputs are separated into event-specific Direct regulatory candidates and Governance context and are not used to automatically determine the applicability or violation of a specific regulation or to assign responsibility.

Role-specific decision support presents the same event evidence differently according to audience purpose. Manager-oriented outputs provide structured event evidence together with regulation-grounded references, whereas worker-oriented outputs focus on accident-related and PPE-related evidence and immediate safety actions. Separate information boundaries were applied so that manager-oriented legal information, including specific law names, article-level information, and penalty-related details, was not directly exposed in worker-facing outputs.

Finally, end-to-end audit and validation were used to verify PPE semantic consistency, regulatory-reference preservation, audience integrity, and worker-facing legal-information isolation. The same frozen upstream evidence, regulatory corpus, and retrieval configuration were used throughout all downstream analyses. The perception models, decision thresholds, regulatory corpus, and retrieval rules were not adjusted post hoc on the basis of downstream evaluation results.

3.2. Dataset and Frozen Multimodal Safety Evidence

3.2.1. Fixed Internal Evaluation Benchmark

The video data used in this study were based on the AI-Hub dataset Data for Smart Manufacturing Facility Safety Monitoring [61]. The source dataset contains RGB and thermal CCTV videos together with object annotations and comprises 20,000 videos and approximately 400,000 extracted images.

For downstream evaluation, a fixed internal benchmark was constructed from Normal, Fall-down, Fall-off, Fire RGB, and Fire Thermal scenarios. The benchmark contained 150 videos and 3,000 frames, with 20 evaluation frames selected from each video. For the final three-class evaluation, Fall-down and Fall-off were combined into the Fall Accident class, whereas RGB and thermal fire scenarios were combined into the Fire Accident class. The resulting benchmark therefore contained 50 videos each for Normal, Fall Accident, and Fire Accident (Table 2).

Because this benchmark had been used in historical analyses, it was not treated as a pristine blind test set or an independent external validation set. However, once the final benchmark was fixed, no samples were added or removed and the class distribution was not altered on the basis of downstream results. The same 150-video benchmark was used for safety-event recognition, structured-event construction, and regulation-grounded downstream evaluation, whereas PPE reliability was evaluated separately on the 100-video human subset described in Section 3.4.1.

3.2.2. Frozen Upstream Perception Configuration

The frozen upstream perception component consisted of YOLO11s-based safety detection [62], YOLO11s-Pose-based worker posture estimation, and Florence-2-base-based semantic verification [63]. Because the objective of this study was not to further optimize the upstream perception architecture, the final model checkpoints and inference configurations were fixed before downstream analysis.

For safety detection, YOLO11s was retrained after removing direct training overlap with the fixed benchmark, while posture information was generated using pretrained YOLO11s-Pose. Florence-2-base was likewise retrained using training/internal-validation data that had been audited to exclude direct sample overlap with the benchmark. The fixed benchmark was not used for Florence-2 training or internal model selection. These corrections were intended to remove direct training exposure from the final perception models and do not redefine the historically used benchmark as an independent external validation set.

The effect of provenance correction was evaluated in Section 4.1 by comparing three configurations: the Historical Original legacy pair, an intermediate configuration combining clean YOLO with the legacy VLM, and the Full-clean configuration combining clean YOLO with overlap-audited clean Florence-2. The Historical and intermediate configurations were used only as references for provenance analysis. All downstream analyses from Section 3.3 onward used only frozen multimodal safety evidence generated by the Full-clean configuration.

Detailed operating parameters for safety detection, pose estimation, semantic verification, and accident-related evidence generation are provided in Supplementary Table S3. After the final configuration was fixed, model weights, decision thresholds, keyframe intervals, and accident-related decision criteria were not readjusted on the basis of the benchmark or downstream results. PPE-specific worker association and Candidate/Confirmed evidence generation were treated as a separate downstream reliability policy and are defined in Section 3.4.

3.2.3. Evaluation Metrics and Statistical Analysis

Safety-event recognition was evaluated at both the frame and video levels using the same fixed benchmark and evaluation units for the Historical Original, intermediate, and Full-clean configurations. The primary metrics were Accuracy and Macro-F1, while class-level prediction behavior and residual errors were examined through confusion-based analysis.

Uncertainty in paired performance differences was evaluated using a video-cluster bootstrap that accounted for clustering among frames extracted from the same video. Videos were resampled with replacement, and all evaluation frames belonging to each resampled video were included as a cluster. This procedure was repeated 10,000 times. The 2.5th and 97.5th percentiles of the resulting Accuracy and Macro-F1 differences were used to construct 95% confidence intervals. The same procedure was applied to video-level differences, with videos used as the resampling unit. A fixed random seed was used for all bootstrap analyses.

A two-sided exact McNemar test was applied to paired classification changes. Because frame-level observations within the same video are not independent, the frame-level McNemar result was used only as descriptive supporting analysis, whereas primary frame-level inference was based on the video-cluster bootstrap. For the video-level McNemar test, each video was treated as one paired observation. These comparisons were not interpreted as evidence of equivalence or non-inferiority.

For PPE reliability, Accuracy, Precision, Recall, Specificity, F1-score, and Balanced Accuracy were calculated on ground-truth normal-scene human frames. Ninety-five percent confidence intervals for the primary metrics were estimated using 10,000 cluster-bootstrap resamples with the video as the resampling unit. Regulation-grounded retrieval regression and role-specific output integrity were evaluated using exact event/reference counts and satisfaction of a predefined deterministic validation contract.

3.3. Structured Event Evidence Construction

The frozen frame-level multimodal safety evidence generated in Section 3.2 includes worker/object location, posture, accident-related evidence, PPE observations, and semantic-verification information. To use this information consistently as an event-level input for downstream PPE processing, regulation-grounded retrieval, and role-specific safety-information generation, the frame-level evidence was transformed into structured event evidence at the video/event level, as shown in Panel B of Figure 1. This stage does not introduce a new classifier; rather, it structures and aggregates frozen upstream evidence according to a common event identity.

Each event record preserves the final video-level recognition class, report-oriented event type, fall- and fire-related evidence, PPE Candidate/Confirmed evidence and accident-related uncertainty, supporting semantic evidence, and regulatory-routing information under the same event identity. PPE-related fields represent event-level aggregation of the operational evidence generated by the worker-centered temporal policy defined in Section 3.4; the corresponding decision rules are specified separately in that section.

The video-level recognition class was distinguished from the report-oriented event type. The recognition class represents the Normal, Fall Accident, or Fire Accident output generated by the frozen upstream perception component, whereas the report-oriented event type is an operational representation used for downstream safety-information generation. Consequently, a separate PPE-related state can be assigned even when the recognition class is Normal if PPE-related evidence is present. The report-oriented event type and operational risk level were determined using a deterministic priority rule that prioritizes Fire Accident and Fall Accident, followed sequentially by PPE Confirmed and Candidate evidence (Table 3).

This priority rule is used only to select one representative event type and one operational risk level; it does not remove evidence that coexists within the same event. In particular, PPE non-compliance is a report-oriented operational state assigned on the basis of Confirmed PPE evidence. It does not imply the absence of unknown_during_accident evidence or establish a legal or workplace compliance violation. For example, when a Fall Accident coexists with PPE Candidate evidence or accident-related uncertainty, the representative event state is set to Fall Accident while the associated PPE evidence remains preserved in the structured event record.

Structured event evidence was also used as the common input to the deterministic event-to-query routing described in Section 3.5.2. Regulatory query categories generated from accident and PPE evidence were associated with and transmitted under the same event identity, while the detailed multi-query activation rules are defined in Section 3.5.2. Each video in the fixed benchmark was mapped to one event, producing a total of 150 structured event records. After the representation was frozen, event classification, operational risk level, PPE evidence, and routing state were not modified post hoc on the basis of downstream results.

3.4. PPE Reliability and Semantic Grounding

3.4.1. PPE Ground Truth and Observation

To evaluate the reliability of the PPE evidence used in the structured event representation defined in Section 3.3, this study distinguished among ground-truth PPE states, frame-level observations generated by the Full-clean perception configuration, and operational PPE evidence generated by the temporal policy described in Section 3.4.2. PPE evaluation was conducted on the 100-video human subset of the fixed internal benchmark, comprising 2,000 frames: 50 Normal videos (1,000 frames), 25 Fall-down videos (500 frames), and 25 Fall-off videos (500 frames).

For each frame, helmet and safety-vest status were independently labeled as either worn or not worn. A frame was defined as PPE-violation positive when at least one component was labeled not worn. Of the 2,000 frames, 1,680 (84.0%) were PPE-positive, with the detailed distribution shown in Table 4.

PPE ground truth was strongly clustered at the video level: all evaluation frames were PPE-positive in 84 videos, whereas all evaluation frames were PPE-negative in the remaining 16 videos. Accordingly, frame-level uncertainty was evaluated using the video-cluster bootstrap described in Section 3.2.3.

Frame-level PPE observations were generated using the Full-clean upstream configuration, and worker evidence was associated with helmet and safety-vest evidence at the component level. For evaluation alignment in frames containing multiple workers, a fixed primary-worker selection rule was applied. Detailed scoring and threshold rules for worker selection and worker–PPE association are provided in Supplementary Table S1. Ground-truth normal/accident scene-status annotations were not used as inputs to PPE inference or the accident-aware policy; the accident gate was determined solely from fall-related evidence generated by the frozen perception configuration. The primary frame-level PPE evaluation was performed on 1,552 ground-truth normal-scene frames, while accident-scene frames were retained separately for accident-aware uncertainty and condition-specific analyses.

3.4.2. Operational PPE Evidence Tiers

To avoid immediately interpreting single-frame PPE non-detection as non-compliance, observations from the same worker track were accumulated temporally. Candidate and Confirmed evidence were then generated after preserving uncertainty under model-derived accident conditions. These are not outputs of separate classifiers; rather, they are operational evidence generated by applying different reliability criteria to the same worker-centered temporal evidence.

Frame-level PPE observations were associated with individual workers through the worker–PPE association procedure defined in Section 3.4.1. Temporal evidence was constructed from the most recent available observations within the same worker track, up to a maximum of 10 observations, rather than from 10 consecutive video frames

Let $c$ denote the PPE component, $r$ the worker track, and $j$ an observation index within that track.

Let $X_{c,r,j}$indicate whether PPE component $\mathrm{c}$ is validly detected for worker track r at observation $j$, and let $W_{r,k}$ denote the set of the most recent observations available at the $\mathrm{k}-\mathrm{t}\mathrm{h}$ observation of worker track r. Present and missing counts were defined as follows:

$$P_{c,r}\left(k\right)=\sum _{j\in W_{r,k}}{X}_{c,r,j},M_{c,r}^{\left(k\right)}=\left|W_{r,k}\right|-P_{c,r}^{\left(k\right)},\left|W_{r,k}\right|\le 10$$

(1)

Here, $X_{c,r,j}$ takes a value of 1 when PPE component c is validly detected at the j-th observation of worker track r, and 0 otherwise. $W_{r,k}$ is the index set of the most recent observations actually accumulated in the worker track at observation k, up to a maximum of 10 observations. $P_{c,r}\left(k\right)$ and $M_{c,r}^{\left(k\right)}$ represent the present and missing counts within this temporal window, respectively. When fewer than 10 observations were available during the early stage of a track, only the observations actually accumulated up to that point were used.

The Raw temporal state was categorized as worn, missing_candidate, not_detected_candidate, or uncertain based on the present and missing evidence. The state was defined as worn when the present count was at least 2. When the present count was less than 2 and the missing count was at least 5, the state was defined as missing_candidate, indicating persistent non-detection. When the present count was less than 2, the missing count was less than 5, and PPE was absent in the current observation, the state was retained as the preliminary not_detected_candidate state. All other cases were categorized as uncertain.

Because PPE observation reliability can temporarily deteriorate under fall-related posture changes or occlusion, a model-derived accident gate was applied. Ground-truth accident labels were not used to activate this gate. Instead, broad-fall, Fall-down, and Fall-off evidence generated by the frozen perception configuration were combined as follows:

$$G_r^{\left(k\right)}=B_{\mathrm{f}\mathrm{a}\mathrm{l}\mathrm{l},r}^{\left(k\right)}\vee F_{\mathrm{d}\mathrm{o}\mathrm{w}\mathrm{n},r}^{\left(k\right)}\vee F_{\mathrm{o}\mathrm{f}\mathrm{f},r}^{\left(k\right)}$$

(2)

Here, $G_r^{\left(k\right)}$ is a binary model-derived accident gate for worker track r at observation k. $B_{\mathrm{f}\mathrm{a}\mathrm{l}\mathrm{l},r}^{\left(k\right)}$, $F_{\mathrm{d}\mathrm{o}\mathrm{w}\mathrm{n},r}^{\left(k\right)}$, and $F_{\mathrm{o}\mathrm{f}\mathrm{f},r}^{\left(k\right)}$ are the binary indicators of broad-fall, Fall-down, and Fall-off evidence, respectively, generated by the frozen perception configuration. OR denotes the logical OR operation. When the accident gate was active, the worn state was preserved, while all other Raw temporal states were converted to unknown_during_accident to retain accident-related uncertainty. When the gate was inactive, the Raw temporal state was left unchanged.

Candidate evidence was activated only when the component state after accident-aware handling was missing_candidate and the missing count was at least 5. Accordingly, not_detected_candidate remained a preliminary non-detection state before the persistent-missing threshold was satisfied and did not immediately generate Candidate evidence. Candidate evidence was treated as review-oriented evidence of persistent non-detection and was not interpreted as confirmed non-compliance.

Confirmed evidence was generated using stricter reliability criteria. It was activated only when, after accident-aware handling, the component state was not worn, the PPE-specific upright criterion was satisfied, the missing count was at least 8, and the present count was 0. The PPE-specific upright criterion was defined as either a valid torso angle of at least 55° or a worker height-to-width ratio of at least 1.25. This criterion was treated as a PPE reliability condition distinct from the upstream fall-related posture criteria listed in Supplementary Table S3.

Candidate and Confirmed evidence were calculated separately for helmet and safety vest. If the corresponding flag was activated for at least one component, it was reflected in the frame-level PPE evidence. unknown_during_accident, Candidate, and Confirmed do not constitute a mutually exclusive multi-class classification. unknown_during_accident is a component state representing accident-related uncertainty, whereas Candidate and Confirmed are operational flags derived from the same temporal evidence under different reliability requirements.

Accordingly, Candidate and Confirmed may coexist when their respective criteria are simultaneously satisfied. In addition, because the Confirmed criterion requires the state to be different from worn and therefore includes unknown_during_accident, Confirmed evidence may also coexist with accident-related uncertainty when both the temporal-persistence and upright criteria are satisfied. Such coexistence does not mean that one form of evidence replaces or invalidates another; each is preserved independently within the structured event record. The corresponding evidence is subsequently used for report-oriented event representation and regulatory routing according to the downstream rules defined in Section 3.3 and Section 3.5.2.

The overall worker-centered operational PPE evidence procedure is illustrated in Figure 2.

The core frozen criteria for PPE evidence generation are summarized in Table 5. Detailed spatial and scoring rules for worker–PPE association are provided in Supplementary Table S1, while upstream accident-related parameters are provided in Supplementary Table S3.

The criteria in Table 5 were retained as the frozen configuration used for the final Full-clean evaluation and were not readjusted on the basis of downstream PPE evaluation or report-generation results.

PPE-related Florence-2 semantic verification was selectively activated by either Confirmed evidence or a predefined long-sustained PPE condition. The long-sustained condition was defined as an upright worker with an inactive accident gate, a missing count of at least 8, and a present count of 0. Semantic-verification results did not modify temporal counts, the accident gate, the Raw state, or Candidate/Confirmed criteria. Florence-2 semantic information was therefore used only as auxiliary semantic evidence for connecting generated PPE evidence to downstream language, rather than as post-hoc decision evidence for redefining operational PPE evidence.

3.4.3. PPE Semantic Grounding

Candidate, Confirmed, and accident-related uncertainty were used as grounding evidence to constrain the component specificity and certainty of downstream PPE-related language. PPE semantic grounding was not used as a classifier that retrospectively modified PPE recognition; instead, it functioned as an output constraint for maintaining semantic consistency between frozen operational evidence and natural-language output.

A specific component, such as a helmet or safety vest, was mentioned only when structured evidence for that component was available. When component-specific evidence was unavailable, component-neutral PPE expressions were used. Candidate evidence was maintained as review-oriented evidence of persistent non-detection and was not escalated into confirmed non-compliance. Likewise, PPE terminology returned by regulatory retrieval was not used as evidence of a visual PPE fact. Actual wearing status and component identity were grounded exclusively in structured visual evidence.

unknown_during_accident alone was not used as sufficient evidence to activate a PPE non-compliance claim or a component-specific non-wearing claim. However, when separate Confirmed evidence satisfying the temporal and posture criteria coexisted within the same event, the corresponding Confirmed flag could be used in the report-oriented state defined in Section 3.3 and in the PPE-related routing defined in Section 3.5.2, while accident-related uncertainty remained independently preserved as supporting evidence.

This grounding policy was applied consistently to the role-specific safety information described in Section 3.6. Unsupported component-specific terminology, evidence certainty, and the consistency of PPE interpretation between notifications and linked reports were evaluated through the semantic-integrity validation described in Section 3.7.2.

3.5. Regulation-Grounded Retrieval

To connect the structured event evidence defined in Section 3.3 with industrial-safety regulatory information, an independent regulation-grounded retrieval pathway was constructed. This pathway does not use the semantic-grounding outputs described in Section 3.4 or downstream natural-language expressions as retrieval inputs. Instead, it uses only the accident-related evidence and PPE Candidate/Confirmed evidence preserved in the structured event record. Accordingly, PPE reliability/semantic grounding and regulation-grounded retrieval are performed independently from the same structured event evidence.

Retrieval-Augmented Generation (RAG) is an approach that connects evidence retrieved from external knowledge sources with language generation to provide grounding and provenance [15]. In this study, retrieval outputs were not used to automatically determine the violation or legal applicability of a specific regulation, assign responsibility, or determine applicable penalties. Rather, candidate regulatory context potentially relevant to the observed safety evidence was provided to manager-oriented decision support. The overall pathway consists of a frozen regulatory corpus, deterministic event-to-query routing, Direct regulatory candidates, and Governance context.

3.5.1. Regulatory Corpus Construction and Freezing

The regulatory corpus was constructed from the official Korean-language texts of industrial-safety-related statutes and regulations provided by the Korean National Law Information Center [64]. The corpus included six regulatory instruments relevant to manufacturing-workplace safety management, and the original Korean texts were used for retrieval. To preserve temporal provenance, the statutory versions in force at the time the corpus was constructed were used (Table 6).

The retrieval unit was defined at the individual-article level. For each article, regulatory-instrument identity, article number, title, body text, and source/effective-date provenance were preserved. Fall-, fire-, PPE-, and routine-related metadata required for event routing were also maintained within the same corpus, together with the duty–penalty linkage used by the Governance branch.

A predefined runtime scope filter was applied to the 1,251 article-level records in the frozen source corpus. This filter excluded 373 records, resulting in a final retrieval pool of 878 records. The final pool contained no duplicate law–article groups, future-effective marker records, or records with empty article titles (Table 7). The scope filter was applied as a predefined exclusion rule rather than as a post-hoc relevance adjustment based on retrieval results. After the corpus was frozen, article membership, query metadata, penalty linkage, and scope rules were not modified. Detailed scope-filter and retrieval configurations are provided in Supplementary Table S2.

3.5.2. Event-to-Query Routing

Structured event evidence was converted into regulatory query categories through predefined deterministic routing rather than through queries freely generated by a language model. Only video-level accident recognition and PPE Candidate/Confirmed evidence were used for routing (Table 8).

Routing was implemented as a multi-query structure rather than as a mutually exclusive single-class classification. Consequently, accident-related and PPE-related queries could be activated simultaneously. PPE Candidate/Confirmed evidence coexisting in the structured event record was therefore retained for regulatory routing regardless of the representative event type defined in Section 3.3.

By contrast, unknown_during_accident alone did not activate a PPE compliance query. PPE-related routing was performed only when Candidate or Confirmed evidence was present. Event-level trigger information was retained solely as provenance metadata and was not used for a learned query-generation mechanism or for determining legal applicability.

3.5.3. Direct Regulatory Candidates

The Direct branch retrieves technical and operational regulatory context directly related to the routed safety issue and is separated from the Governance branch, which provides broader management-duty and responsibility context. Candidates were ranked within the frozen 878-record pool using BM25 lexical relevance together with frozen query-type and rule-based signals. Only articles whose combined title and body Direct relevance scores satisfied the minimum criterion in Equation (3) were permitted as Direct candidates.

$$\mathrm{D}\mathrm{i}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{C}\mathrm{a}\mathrm{n}\mathrm{d}\mathrm{i}\mathrm{d}\mathrm{a}\mathrm{t}\mathrm{e}\left(\mathrm{a},\mathrm{e}\right)=\left\{\begin{array}{cc}1,& S_{\mathrm{t}\mathrm{i}\mathrm{t}\mathrm{l}\mathrm{e}}\left(a,e\right)+S_{\mathrm{b}\mathrm{o}\mathrm{d}\mathrm{y}}\left(a,e\right)\ge 1.0\\ 0,& \mathrm{o}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{w}\mathrm{i}\mathrm{s}\mathrm{e}\end{array}\right.$$

(3)

Here, a denotes a candidate regulatory article and e denotes a structured safety event. $S_{\mathrm{t}\mathrm{i}\mathrm{t}\mathrm{l}\mathrm{e}}\left(a,e\right)$ and $S_{\mathrm{b}\mathrm{o}\mathrm{d}\mathrm{y}}\left(a,e\right)$ represent the title and body Direct relevance scores, respectively, for article a with respect to event e. Detailed scoring rules, BM25 parameters, and frozen retrieval settings are provided in Supplementary Table S2.

Direct retrieval did not forcibly include articles that failed to satisfy the threshold merely to fill a fixed candidate count. For multi-query events, technical references selected for each routed issue were retained together within the same event. No Direct references were generated for events in which only Routine monitoring was activated. Direct outputs were passed downstream as candidate regulatory context related to the structured safety evidence rather than as determinations of legal applicability, violation, or compliance.

3.5.4. Governance Context

The Governance context was implemented as a branch independent of the Direct candidates and was designed to provide higher-level safety-management duty, responsibility, and penalty-linked regulatory context rather than event-specific technical measures. Although the two branches use the same frozen corpus, their retrieval purposes and provenance are kept separate.

Governance references were generated through two paths: a ranking-based path and a guaranteed worker-duty path. The ranking-based path selects governance-oriented articles according to event-to-query relevance. The guaranteed worker-duty path independently preserves relevant duty articles and linked penalty provenance to reduce the possibility that worker-related duty context is omitted by the general ranking process. These two paths do not represent different levels of legal certainty; rather, they indicate different reference-inclusion provenance.

Penalty linkage likewise does not indicate that a penalty is actually applicable to a particular event. Instead, it is used as a mechanism for tracing regulatory provenance associated with duty/responsibility articles. The Governance reference set was frozen at the retrieval stage, after which reranking, refiltering, truncation, or reference substitution that could alter membership or ordering was not permitted during report generation. Ranking-based, guaranteed worker-duty, and penalty-linked provenance were likewise preserved without modification.

No Direct or Governance references were generated for Routine monitoring events. Governance outputs were also limited to grounded regulatory context and were not treated as determinations of legal applicability, violation, compliance, or penalty applicability. The final Direct and Governance evidence was subsequently passed to the manager-oriented decision-support process described in Section 3.6.

3.6. Role-Specific Safety Decision Support

A role-specific safety decision-support layer was constructed to transform frozen structured event evidence into information that could be used in practical safety management. Manager- and worker-oriented outputs share the same source event but apply different audience policies. Managers are provided with structured event evidence together with Direct/Governance regulatory context, whereas workers receive accident/PPE evidence and immediate safety actions, with detailed legal, governance, and penalty-related information excluded. Accordingly, PPE grounding and regulation-grounded retrieval are not combined as a serial process; instead, independent evidence generated from the same structured event evidence is selectively used according to audience purpose.

3.6.1. Audience-Specific Safety Reports

Manager-oriented reports were designed to include an event summary, operational risk information, accident/PPE evidence, and the Direct/Governance references described in Section 3.5. GPT-5.6 Terra (gpt-5.6-terra) was used for natural-language realization, and the prompt/template configuration was fixed before final report generation. Detailed generation settings are provided in Supplementary Table S4.

The language model was used only to transform frozen structured evidence and regulatory context into audience-specific natural language. It was not allowed to newly determine or modify event classification, operational risk, PPE state, or regulatory-reference selection. Prompts were constructed deterministically from a predefined template, structured event evidence, and frozen regulatory references. Even when a generated output did not satisfy the validation conditions, the underlying evidence and references were left unchanged, and only the natural-language realization was retried, for a maximum of five attempts.

In manager-oriented outputs, accident evidence, PPE Candidate/Confirmed evidence, and accident-related uncertainty coexisting within the same event were retained as supporting evidence independently of the representative report state. Direct/Governance reference membership and ordering were also preserved during generation, with no reranking, refiltering, truncation, or reference substitution applied.

Worker-oriented reports were generated according to an independent audience policy rather than as abbreviated versions of manager-oriented reports. For accident-related events, the worker-oriented reports focused on immediate protective actions, while PPE-related information followed the same reliability and semantic-grounding policy described in Section 3.4. Specific helmet or safety-vest information was included only when directly supported by the corresponding structured evidence. Candidate evidence or accident-related uncertainty alone was not used to generate a stronger non-compliance claim. When separate Confirmed evidence coexisted with uncertainty evidence, the respective reliability states of both Confirmed and uncertainty evidence were preserved.

Worker-facing outputs did not include specific law names, article-level information, legal responsibility, or penalty-related details from Direct/Governance references. Regulatory terminology was also not used to infer a visual PPE state. The same information boundary was applied to detailed worker reports and notifications to prevent indirect legal-information exposure through linked reports. Role-specific generation was therefore limited to expressing frozen evidence according to different audience purposes and was not used to modify upstream safety, PPE, or legal decisions.

3.6.2. Audience-Specific Notification Routing

Notification emission was determined deterministically from the frozen event/query state. Manager notifications were generated for reportable accident or PPE-related review events, excluding Routine monitoring events. Worker notifications were generated when a PPE-related query was activated. Thus, when accident and PPE evidence coexisted, a worker notification was still generated if the PPE query was active, whereas no separate alert notification was generated for Routine monitoring events.

Each notification was linked only to a detailed report corresponding to the same source event and the same audience. Manager notifications were linked to manager-oriented reports, and worker notifications were linked to worker-oriented reports; cross-audience linkage was not permitted. Notification routing was limited to the final delivery layer and did not modify event classification, PPE interpretation, risk assessment, or regulatory-reference selection. Source-event correspondence, audience consistency, and notification–report integrity were evaluated through the independent validation procedure described in Section 3.7.2.

3.7. End-to-End Validation and Audit Protocol

An end-to-end validation protocol separated from report generation was applied to evaluate the provenance and integrity of the structured event evidence, PPE grounding, regulatory references, and role-specific outputs generated in Section 3.3, Section 3.4, Section 3.5 and Section 3.6. Validation was not used as a feedback stage for modifying the model, thresholds, retrieval configuration, or report policy on the basis of downstream results. Instead, it was defined as a procedure for verifying the reproducibility, evidence preservation, and audience/information integrity of the frozen pipeline.

3.7.1. Regulatory Reference Preservation and Controlled Retrieval Regression

To verify that Direct and Governance references were transmitted unchanged from retrieval to manager-oriented outputs, reference identity and ordering were compared. For the Governance branch, preservation of ranking-based, guaranteed worker-duty, and penalty-linked provenance was evaluated together with whether any downstream reranking, refiltering, truncation, or reference substitution occurred.

To separate the reproducibility of the frozen Legal-RAG process from the effects of upstream evidence correction, a controlled regression analysis was performed by inputting Historical and final Full-clean structured event evidence into the same regulatory corpus, runtime scope, routing rules, and retrieval configuration. Historical outputs were not treated as a legal gold standard; rather, they served as regression references for comparing the effects of evidence transitions within the same frozen retrieval process.

Events were divided into three transition groups. In the Strict invariant group, downstream-relevant evidence remained unchanged, and exact Direct/Governance retrieval was therefore required. In the Query-invariant evidence-change group, some structured evidence could change while the query scope was expected to remain unchanged; retrieval outcomes were then examined under the unchanged query scope. In the Query-changed group, a change in query scope caused by corrected evidence was defined as expected behavior, together with the corresponding retrieval change. This design was used to determine whether observed retrieval differences reflected changes in structured evidence and routing scope rather than corpus modification or post-hoc retuning.

3.7.2. Audience and End-to-End Integrity Verification

For role-specific outputs, audience/event consistency, PPE semantic integrity, and worker-facing legal-information isolation were evaluated. Notifications and linked reports were checked to verify that they preserved the same source event and the correct audience. Cross-audience linkage, a missing target report, or an event mismatch was defined as an integrity failure.

PPE semantic integrity was evaluated according to whether downstream language remained within the component specificity and certainty supported by the structured PPE evidence. Specific helmet or safety-vest claims were permitted only when corresponding supporting source evidence was available. The reliability states of Candidate, Confirmed, and accident-related uncertainty evidence were also checked for preservation throughout downstream processing. A semantic-integrity failure was defined as a case in which Candidate evidence or accident-related uncertainty alone was escalated into a stronger PPE claim, or regulatory terminology was used as evidence for a visual PPE fact. For PPE-primary events, consistency in component identity and certainty between the notification and its linked worker-oriented report was also evaluated.

Worker-facing legal-information isolation was evaluated by checking whether specific law names, article-level information, legal responsibility, or penalty-related details were exposed in either notifications or linked detailed reports. Indirect exposure of manager-oriented regulatory information through a linked report was likewise defined as a failure.

Final integrity was evaluated using a frozen end-to-end verifier that operated separately from the report-generation process. Under a predefined validation contract, the verifier evaluated source event–report correspondence, Direct/Governance reference preservation, manager/worker audience consistency, notification–report linkage, PPE semantic grounding, and worker-facing legal-information isolation on the fixed benchmark. Validation findings were not used as post-processing to automatically modify the outputs, nor were they used to readjust the perception model, PPE policy, Legal-RAG configuration, or report content.

Provenance for major analytical artifacts and execution configurations was preserved using hash fingerprints and a manifest, as described in Supplementary Table S4. Publication-oriented tables and visualizations were limited to a read-only presentation stage and did not recalculate or semantically modify the frozen analytical outputs. Representative publication images were used only when both the predefined source event and target frame matched exactly; cross-event frame substitution was not permitted. Final integrity results are reported in Section 4.4. OpenAI GPT Image 2 was used only to assist in the visual drafting of the schematic presented in Figure 7. The final figure was reviewed and edited by the authors, and the tool was not used to generate, alter, or interpret source-event images, research data, quantitative results, or analytical outputs.

4. Results and Discussion

This section presents the results of the final safety-intelligence pipeline based on the frozen evaluation framework defined in Section 3. Safety-event recognition performance and uncertainty associated with provenance correction are evaluated first, followed by PPE reliability and operational evidence, regulation-grounded retrieval, role-specific safety outputs, and end-to-end integrity. All results were obtained using the benchmark, model configuration, and validation protocol fixed in advance. No model retraining, threshold adjustment, or post-hoc modification of downstream rules was performed on the basis of the results reported in this section.

4.1. Safety-Event Recognition and Provenance-Correction Effects

On the fixed internal benchmark, the Full-clean configuration achieved a frame-level Accuracy of 98.23% and Macro-F1 of 97.47%, together with a video-level Accuracy and Macro-F1 of 98.67%, correctly classifying 148 of 150 videos. The Historical Original configuration achieved frame-level Accuracy/Macro-F1 of 97.87%/96.98%, while the intermediate configuration achieved 97.83%/96.93%; both reference configurations achieved a video-level Accuracy of 98.00% (Figure 3). The Historical Original configuration was treated as a historical reference preceding provenance correction rather than as an independent performance baseline.

Compared with the intermediate configuration, the Full-clean configuration increased frame-level Accuracy by 0.40 percentage points and Macro-F1 by 0.54 percentage points. All 12 frames misclassified by the intermediate configuration were corrected without introducing any new frame-level errors. The corrected cases comprised six Normal frames, five Fall-down frames, and one Fall-off frame. The two-sided exact McNemar test yielded p = 0.000488; however, because frame-level observations were not independent within videos, this result was interpreted only as descriptive supporting evidence. In the video-cluster bootstrap analysis, the 95% confidence intervals for the frame-level Accuracy and Macro-F1 differences were +0.13 to +0.77 and +0.17 to +1.07 percentage points, respectively, neither of which included zero (Figure 4).

At the video level, one Normal video that had previously been misclassified as Fall Accident was corrected, and no new errors were introduced, reducing the total number of video-level errors from three to two. However, the paired McNemar test yielded p = 1.0, and the bootstrap confidence interval also included zero. The results therefore support a frame-level improvement with the Full-clean configuration but do not establish a statistically significant improvement, equivalence, or non-inferiority at the video level. The two remaining errors consisted of one Fall-off-to-Normal false negative and one Normal-to-Fall Accident false positive, both occurring at the fall-related decision boundary.

4.2. PPE Reliability and Operational Evidence Performance

PPE reliability was evaluated on the primary evaluation stratum defined in Section 3.4.1, consisting of 1,552 ground-truth normal-scene frames. For combined raw PPE-violation detection, TP = 1,296, TN = 238, FP = 16, and FN = 2. The corresponding performance was 98.84% Accuracy, 98.78% Precision, 99.85% Recall, 93.70% Specificity, 99.31% F1-score, and 96.77% Balanced Accuracy. The video-cluster bootstrap 95% confidence intervals were 97.66–99.67% for Accuracy, 97.32–99.72% for Precision, 99.50–100.00% for Recall, and 98.54–99.81% for F1-score.

At the component level, the helmet and safety-vest F1-scores on ground-truth normal-scene frames were 95.70% and 97.94%, respectively. The lowest condition-specific PPE performance was observed for helmet detection under the Fall-off condition, with a Precision of 65.65% and an F1-score of 77.83%. For the 1,372 frames in which the model-derived accident gate was inactive, the combined PPE F1-score was 99.79%. By contrast, among the 180 frames in which the gate was active, the F1-score decreased to 95.27%, with a Precision of 92.25% and a Specificity of 76.60%. These results indicate that PPE observation reliability can decrease when model-derived fall evidence is present even within the ground-truth normal-scene stratum, supporting the policy of preserving accident-related uncertainty separately from temporal Candidate/Confirmed evidence.

For the operational evidence tiers, Candidate achieved a Precision of 99.64%, Recall of 64.10%, and F1-score of 78.01%, whereas Confirmed achieved a Precision of 100.00%, Recall of 42.06%, and F1-score of 59.22% (Figure 5). Candidate and Confirmed are not separately trained classifiers; rather, they are operational tiers generated by applying progressively stricter reliability criteria to the same worker-track evidence. Accordingly, the decrease in Recall across the stricter tiers was interpreted not as classifier degradation, but as the effect of conservative filtering intended to distinguish transient non-detection from persistent evidence satisfying stricter reliability criteria.

At the event level, PPE-related routing was activated for 90 of the 100 human-related videos, which comprised 84 ground-truth-positive and 16 ground-truth-negative videos. The routing results were TP = 84, TN = 10, FP = 6, and FN = 0, corresponding to an Accuracy of 94.00%, Precision of 93.33%, Recall of 100.00%, and F1-score of 96.55%. Neither the number of routing activations nor the video-level routing decisions changed between the Historical and Full-clean evidence. These results were interpreted as downstream routing performance for selecting events in which PPE-related regulatory candidates should be retrieved, rather than as a legal or compliance determination of PPE violations.

4.3. Regulation-Grounded Retrieval and Controlled Regression

Using the final Full-clean structured evidence, the report-oriented event types consisted of 50 Fall Accident events, 50 Fire Accident events, 43 PPE non-compliance events, 1 PPE non-detection candidate event, and 6 No reportable accident events. Deterministic routing activated 50 Fall-related, 50 Fire-related, 90 PPE-related, and 6 Routine monitoring queries. Because the query categories were not mutually exclusive, the 90 PPE-related events included multi-query events in which accident and PPE evidence coexisted.

Retrieval coverage and provenance results are summarized in Table 9. Direct and Governance references were generated for 144 of the 150 non-routine events, whereas no references were generated for the 6 Routine monitoring events. The Direct branch produced 1,108 references corresponding to 23 unique articles, while the Governance branch produced 576 references corresponding to 6 unique articles. The Governance set comprised 288 ranked references and 288 guaranteed worker-duty references.

In the controlled regression analysis, all 117 Strict invariant events reproduced the exact retrieval results (117/117). Among the 32 evidence-change events for which the query scope remained unchanged, routing was preserved in all 32 cases and no retrieval changes occurred. In the single event for which the query scope changed, the expected retrieval change was observed. Specifically, the Historical evidence activated both Fall and PPE queries, whereas the Full-clean evidence activated only the PPE query; consequently, the number of Direct candidates decreased from 8 to 7, with only the fall-specific candidate removed. These results indicate that the frozen retrieval process reproduced the same retrieval outputs when the query scope remained unchanged and produced a corresponding retrieval change only when the upstream correction altered the routing scope.

Downstream processing preserved all 1,108/1,108 Direct references and all 576/576 Governance references, together with their ranked/guaranteed provenance, and no reference loss attributable to report generation was observed. These results, however, do not validate the legal applicability of the retrieved articles or the accuracy of violation/compliance determinations. The controlled regression and reference-preservation analyses were used to evaluate the reproducibility and provenance preservation of the frozen Legal-RAG process, and the Direct/Governance outputs were interpreted as candidate regulatory context rather than as automated legal determinations.

4.4. Role-Specific Safety Outputs and End-to-End Integrity

Manager- and worker-oriented reports were generated for all 150 events. Manager-oriented natural-language generation satisfied the validation conditions on the first attempt for 135 of 150 events (90.0%). The remaining 15 events satisfied the validation conditions on Attempts 2–5, with 9, 2, 2, and 2 events succeeding at each respective attempt. No template fallback was required. During retries, the structured evidence and regulatory references remained unchanged, and only the natural-language realization was regenerated.

For audience-specific routing, all 144 expected manager notifications and all 90 expected worker notifications were generated. Across the resulting 234 notification–report links, both the source event and intended audience were consistent in all cases. No worker-facing legal-information leakage was observed, and PPE semantic consistency was satisfied in all 44 evaluated cases. The overall generation, routing, and integrity results are summarized in Table 10.

The independent frozen verifier evaluated source event–report correspondence, regulatory-reference preservation, audience consistency, notification linkage, PPE semantic grounding, and worker-facing legal-information isolation according to the predefined validation contract. All 82 verifier findings received a PASS outcome. This indicates that the final outputs satisfied the predefined provenance and information-boundary conditions; however, it does not establish overall natural-language quality or actual operational effectiveness. Worker comprehension, manager decision quality, report usability, and the legal applicability of the retrieved regulatory references were outside the scope of this validation.

4.5. Representative Event-Level Safety Evidence

To complement the preceding quantitative evaluations, representative event-level visual evidence for Fall, Fire, PPE, and Routine cases is presented in Figure 6. Each image was used only when both the designated source event and target frame matched the corresponding frozen analytical output; cross-event frame substitution was not permitted. Figure 6 therefore preserves the same event-level provenance as the structured event evidence evaluated in Section 4.1, Section 4.2, Section 4.3 and Section 4.4, rather than serving as a set of arbitrarily selected illustrative examples.

The Fall and Fire examples respectively illustrate fall-related posture/body geometry and fire-related visual evidence; however, the representative frame itself does not independently determine the overall video-level decision. Likewise, the PPE examples do not interpret single-frame evidence as confirmed non-compliance. Operational PPE evidence was instead differentiated into Candidate and Confirmed evidence on the basis of worker-track temporal history and the accident-aware reliability policy. Specific helmet- or safety-vest-related language was used only when directly supported by the corresponding structured evidence.

The Routine example represents a negative case in which neither a reportable accident nor a PPE-related review condition was activated. Accordingly, Direct/Governance references and alerts were not forcibly generated. Figure 6 should therefore be interpreted as a qualitative complement demonstrating how event identity, visual provenance, and uncertainty boundaries were preserved from frozen perception through downstream decision support, rather than as a separate performance evaluation.

To further illustrate how a representative source event is transformed within the proposed downstream framework, Figure 7 presents an end-to-end evidence trace from frozen multimodal safety evidence to role-specific outputs and independent audit validation.

Figure 7. Representative end-to-end evidence trace from frozen multimodal safety evidence to role-specific decision support and independent audit and validation.

Figure 7. Representative end-to-end evidence trace from frozen multimodal safety evidence to role-specific decision support and independent audit and validation.

As shown in Figure 7, the same source event is preserved under a common event identity, processed through independent PPE-reliability and regulation-grounded pathways, and subsequently transformed into manager-oriented and worker-oriented outputs under distinct information boundaries.

4.6. Discussion, Practical Implications, and Limitations

The primary significance of this study lies in extending manufacturing-workplace video analysis beyond accident classification or PPE detection by introducing a downstream framework that transforms frozen perception evidence into structured and auditable safety information. Centered on structured event evidence, the framework performs PPE reliability/semantic grounding and regulation-grounded retrieval as independent downstream processes and applies different audience-specific information boundaries to the same source event. In this way, evidence reliability, provenance, and uncertainty preservation are treated as core design principles rather than relying solely on predictive performance.

The perception and PPE results support this reliability-centered design. The Full-clean configuration showed an improvement at the frame level, whereas no statistically significant improvement was established at the video level. VLM verification should therefore be interpreted as an auxiliary semantic source for supporting ambiguous evidence rather than as an independently superior classifier. For PPE, stricter reliability requirements from Candidate to Confirmed maintained or increased Precision while reducing Recall. This pattern was interpreted not as classifier degradation, but as a consequence of conservative evidence filtering designed to avoid escalating transient non-detection into stronger non-compliance claims.

Regulation-grounded retrieval was likewise designed around reproducible evidence linkage rather than legal decision making. Retrieval was reproduced when the query scope remained unchanged, whereas corresponding changes in the candidate set occurred only when upstream correction altered the routing scope. These results do not establish the legal applicability of the retrieved articles or the presence of a violation. Direct and Governance outputs should therefore be interpreted as candidate regulatory context for manager review rather than as automated legal determinations.

For role-specific decision support, managers were provided with event evidence and regulatory context, whereas workers received immediate action and PPE-related information, with detailed legal and penalty-related information kept separate. Reference preservation, audience/event consistency, PPE semantic grounding, and legal-information isolation were additionally evaluated using a verifier independent of the generation process. From a practical perspective, this structure may support a human-review-oriented decision-support model in which AI provides review-oriented information while preserving source provenance and uncertainty, rather than replacing final safety or legal judgment.

This study has four principal limitations. First, the evaluation was based on a fixed internal benchmark comprising 150 videos and 3,000 frames with a history of prior use; it should therefore not be considered a pristine blind test or an independent external validation. Second, the safety-event taxonomy was limited to Normal, Fall Accident, and Fire Accident, while PPE evaluation was limited to helmets and safety vests. Validation across additional hazard types and PPE components is therefore required. Third, the regulatory evaluation was conducted using a frozen Korean industrial-safety corpus and predefined retrieval rules; no expert-derived legal gold standard or cross-jurisdictional applicability assessment was performed. Fourth, the integrity validation assessed evidence preservation and information boundaries rather than worker comprehension, manager decision quality, response time, alert fatigue, trust, or actual usability. Future work should therefore evaluate generalizability and operational utility using independent external datasets, an expanded hazard/PPE taxonomy, safety/legal expert assessment, and human-centered prospective field validation.

5. Conclusions

This study proposed an auditable downstream safety-intelligence framework for transforming multimodal safety evidence generated from manufacturing-workplace video into information that can be reviewed and used in practical safety management. Frozen multimodal perception outputs were transformed into event-level structured evidence, after which reliability-aware PPE processing and regulation-grounded retrieval were performed independently from the same event evidence and connected to manager- and worker-oriented outputs. PPE semantic consistency, regulatory-reference preservation, audience integrity, and worker-facing legal-information isolation were additionally evaluated through validation separated from the generation process.

On the fixed internal benchmark, the Full-clean configuration achieved a frame-level Accuracy of 98.23%, a Macro-F1 of 97.47%, and a video-level Accuracy and Macro-F1 of 98.67%. It corrected all 12 frame-level errors made by the intermediate configuration without introducing any new frame-level errors. However, the video-level improvement was not interpreted as statistically significant. On the ground-truth normal-scene PPE stratum, raw PPE-violation detection achieved an F1-score of 99.31%, while Candidate and Confirmed evidence exhibited higher Precision and lower Recall under progressively stricter reliability criteria. In the regulation-grounded controlled regression, all 117 Strict invariant cases reproduced the same retrieval results, all 32 Query-invariant cases preserved query scope without unexpected retrieval changes, and the single Query-changed event produced the expected retrieval change. All 1,108 Direct references and 576 Governance references were preserved throughout downstream processing. No audience/event mismatch was observed across the 234 notification–report links, and all 44 PPE semantic checks and all 82 findings from the independent verifier satisfied the predefined validation contract.

Overall, the proposed framework extends vision-based industrial safety monitoring from hazard recognition toward evidence-grounded, reliability-aware, regulation-grounded, role-specific, and auditable decision support. Its central design principle is to preserve uncertainty and provenance throughout downstream processing rather than forcing accident- and PPE-related evidence into a single label, while positioning AI as a human-review support layer rather than as an autonomous safety or legal decision-maker. The findings are nevertheless based on a fixed internal benchmark with historical provenance and a Korean regulatory corpus and do not establish legal applicability, independent external generalization, or actual worker/manager usability. Future work should therefore include independent external validation, expanded hazard and PPE evaluation, safety/legal expert assessment, and human-centered prospective field validation.