Training Community-Academic Teams to Use Low-Cost Air Monitors: A Mixed Method Evaluation of the RISE Communities Program

1. Introduction

The significant negative impacts that air pollution has on human and environmental health have been extensively described and include cardiopulmonary, neurologic, skin, and endocrine disease [1,2,3,4,5]. Globally, exposure to air pollution, including fine particulate matter (PM2.5), is responsible for more than 7.5 million deaths, including nearly 70,000 deaths per year in the United States (US) [6]. Recently, the US Environmental Protection Agency (US EPA) adopted a lower annual average air quality standard for PM2.5 (9 mg/m3); compliance with this limit and other National Ambient Air Quality Standards

(NAAQS) is determined through a network of regulatory monitoring stations. A recent analysis of NAAQS monitoring locations estimates that more than 40% of urban locations, representing approximately 20 million residents, will exceed the new standard but be undetected due to the sparsely distributed locations of the current monitoring sites [7]. Notably, inadequate monitoring disproportionately impacts people of color (PoC) and lower-income individuals [8,9].

In the past decade, low-cost air monitors have been deployed by community residents and researchers to address the limitations of regulatory monitoring networks. These

monitors are increasingly accurate, reliable, and economical, and their deployment in impacted communities may help residents identify specific sources of poor air quality and motivate policy changes based on improved awareness [10,11,12,13,14,15]. While low-cost sensor networks are being established in some communities, widespread application of this technology remains limited due to challenges in sensor selection, maintenance, calibration, data collection and analysis, interpretation, and other technical barriers faced by users [13,16,17,18]. Community-based research projects across the US have shown much promise in integrating the technical science of air quality research with the advocacy and policy needs of the communities involved in these projects [19]. Current projects have evaluated the interactions between researchers and policy makers as well as taught young people the basics of air quality monitoring [20,21]. Often, participants in these projects indicate the need for additional technical training and time to establish trust between academic and community partners. There are few formal training programs that develop these technical and teamwork skills for community engaged research partnerships. Furthermore, evaluation of the effectiveness of a formal training program has not been discussed in the literature before.

Our Research Innovations using Sensor Technology to Improve Community Health (RISE Communities) training program is motivated by this need and serves as one of the first programs to offer formal training in air quality monitor installation, data extrapolation, data visualization, team building, and best practices for community engagement. The program was intentionally designed for teams of community members and academic partners and includes formal evaluation of participant feedback throughout the training program to guide our recommendations for future, similar training programs. Our program goals include: 1) foster community-academic partnerships through research education, training, and team development activities, 2) provide technical training in the application of low-cost sensors for indoor, outdoor, and personal air monitoring in communities, and 3) establish a community of practice to address air quality in communities nationwide. The objective of this report is to describe the RISE Communities training program and evaluate its effectiveness to increase confidence and skills in community engagement, team function, and air sensor data collection and analysis in the program participants.

2. Materials and Methods

Participants: Participants applied to join the RISE Communities program, and eligible applications included a research team comprised of at least one member from an academic institution and one member associated with a community organization or living in a community impacted by poor air quality. Applicants were recruited through advertisements and dissemination of program information on the RISE Communities website, professional networks including the National Institute of Environmental Health Sciences (NIEHS) Partnerships for Environmental Public Health (PEPH) newsletter, and colleagues and collaborators at institutions and community organizations throughout the US. RISE Communities faculty members also distributed information about the program to their networks. The RISE Communities program directors presented an overview of the program at the annual PEPH meeting held at NIEHS, the Association of Clinical and Translational Science meeting held in Las Vegas, NV, and the Air Sensors International Conference held in Riverside, CA, all held in 2024. Application requirements included a description of the project community and rationale for deploying low-cost air monitors. Applications were received online through the RISE Communities website and reviewed and scored by the program directors and program manager based on the applicant’s qualifications, needs, and alignment with RISE Communities program goals.

In-Person Training: In-person training sessions were held in Cincinnati, OH during the summers of 2023 (Cohort 1) and 2024 (Cohort 2). Each session consisted of a two-and-a-half-day program led by faculty, staff, professionals in the training topics, and sensor developers who are national experts in team science, community-engaged research, environmental health, epidemiology, and low-cost air monitors. Topics included principles of air pollution and health, choosing the right air sensor for your project, team development, principles of community engaged research, engaging with communities using air sensors, sensor operation, data visualization, and responsible conduct of research. Local community partners provided a tour of a neighborhood in Cincinnati, OH that is impacted by poor air quality and where a network of low cost (PurpleAir) sensors was established to learn how they implement community learning and research in their work. Figure 1 provides an example schedule for the in-person training components. All participants were also provided with ten PurpleAir sensors for deployment in their project communities. Participants completed walk through demonstrations on setting up low-cost sensors, downloading data recorded by the sensors, and visualizing an analyzed version of this data. Dedicated time was provided for teams to describe their communities, projects, and goals for the use of low-cost air monitors and receive group feedback and suggestions from all cohort participants and faculty.

Figure 1. Example Schedule of RISE Communities In-Person Training Program.

Webinars and Online Resources: After the in-person intensive education, monthly webinars were offered on follow-up topics that were suggested by trainees or that aligned with the status of their air sampling project implementation. All community-academic teams and faculty were invited to suggest topics and/or speakers for the webinar series based on interest and challenges faced by the participating teams. Webinar speakers included experts in data visualization, community-engaged research, public policy, and other topics. In addition, one webinar per quarter was reserved for teams to provide updates on their projects, receive peer feedback, and ask faculty and other attendees for suggestions or guidance. Figure 2 provides an example webinar schedule from one cohort. Webinars were recorded and posted to the program website. An online library of resources such as specific analysis methods and codes discussed during the training, a sensor calibration guide, and templates and examples of quality assurance project plans were also developed on the program website, as well as a discussion board to discuss common issues faced by teams and to share news stories.

Figure 2. Example Webinar Schedule of RISE Communities Webinars.

Figure 2. Example Webinar Schedule of RISE Communities Webinars.

Survey Data: Demographic information for the entirety of cohort 1 and cohort 2 were self-reported and collected through REDCap surveys when participants began the program.

We evaluated the effectiveness and outcomes of the RISE Communities training program using data collected in a pre-survey, post-survey, a survey collected one year after training was completed, webinar evaluation surveys, and focus group feedback collected from participants at the end of their training. Participants completed a pre-survey before attending the in-person training program in Cincinnati and a post-survey shortly after the training program concluded. Trainees were asked to rate their confidence on a 4-point scale correlating to the levels ‘Extremely confident (4)’, ‘Pretty confident (3)’, ‘A little confident (2)’, and ‘Not at all confident (1)’. The participants completed the final survey approximately one year after participating in the in-person training program and once all training activities had concluded for their cohort. Table 1 provides a summary of the survey questions. Changes in confidence levels reported in these surveys were evaluated using paired t-tests (p = 0.05).

In addition to the confidence questions, the one year survey inquired about how the training program contributed to their research outcomes, media representation, or community discussions. Additional questions, including open-ended items, were included to evaluate overall program effectiveness and sustainability of the research projects and partnerships, from the participants’ perspective. Table 2 provides a summary of additional questions included in the one year follow-up survey. All survey data were collected through REDCap. The pre and post surveys were encoded to each individual participant along with demographic and contact information. However, the end of program survey was collected anonymously.

Trainees were also asked to complete anonymous evaluative surveys following each monthly webinar. Participants were able to slide a scale of 1-100 to indicate how strongly they agreed with statements about the quality of various aspects of the webinar, including value and usefulness, expertise of the presenter, engagement, and clarity of learning objectives.

Focus groups: Focus groups were conducted by two different facilitators who were medical student research assistants and therefore unknown to participants, allowing for honesty and openness during the focus group discussion. They were conducted, recorded, and transcribed via Zoom approximately one year after the in-person training program had occurred, and shortly after the conclusion of the final webinar for each respective cohort. Participants were invited to share their perspectives on how RISE Communities impacted their individual projects and understanding of community engaged environmental health research. The focus group guide is provided as a supplemental appendix. Qualitative data obtained from the focus groups were analyzed using a thematic analysis approach where data were coded in order to construct thematic patterns across participant responses, with the intent of identifying key themes across cohorts.

3. Results

Participants: A total of 31 trainees participated in cohorts 1 and 2 of the RISE Communities program. Of these, 58% were community partners, 32% academic partners, 7% local government officials, and 3% students at a university associated with the academic partner. The majority of participants (94%) reported having a bachelor’s degree or higher; 32% had a doctorate. Furthermore, participants were 74% women, 23% men, and 3% non-binary, transgender, or gender non-conforming. Participants reported their ethnicity as 13% Hispanic or Latino, and of those that reported their racial background, 29% reported being a minority. Two of the 31 participants reported having a disability by the ADA definition and three participants reported that English was not their first language. Finally, 45% of participants reported coming from a disadvantaged background. Table 3 provides a summary of participant demographics.

Survey Data: The response rates of the pre- and post-surveys were 95% and 87%, respectively. The response rate for the one year follow-up survey was 86%. Analyses of pre- and post- survey data show a significant increase in participants’ confidence levels across several survey domains (Figure 2). Results also show a significant increase in confidence levels between the pre survey and the one year follow-up survey 1 year later. As shown in Figure 2, there was asignificant increase (p<0.05) in the confidence levels between pre and post survey responses for several statements; confidence in the statement ‘Describing the health risks associated with air pollution’ increased by 0.35 points (95% CI [-0.032, 0.73]), ‘Understanding the challenges faced by ‘environmental justice’ communities’ increased by 0.33 points (95% CI [0.0044,0.65]), ‘Analyzing data received from PurpleAir sensors placed in the community’ increased by 0.52 points (95% CI [0.083, 0.96]), ‘Summarizing air quality data in a visual way for others’ increased by 0.54 points (95% CI [0.13, 0.96]), and ‘Engaging community members in environmental health issues’ increased by 0.37 points (95% CI [-0.019, 0.77]). In addition, there was an even more significant increase (p<0.001) in the confidence levels between pre and post survey responses in the following statements: confidence in the statement ‘Using PurpleAir sensors to collect air quality data’ increased by 1.02 points (95% CI [0.60, 1.44]), and ‘Accessing data received from PurpleAir sensors placed in a community’ increased by 0.83 points (95% CI [0.46, 1.21]).

Participants reported continued significantly increased confidence (p<0.05) in the one year follow-up survey in multiple domains; confidence in the statement ‘Effectively participating in a research partnership between a community organization and an academic institution’ increased by 0.31 points (95% CI [-0.020, 0.65]), ‘Understanding the challenges faced by ‘environmental justice’ communities’ increased by 0.40 points (95% CI [0.073, 0.72]), ‘Analyzing data received from PurpleAir sensors placed in the community’ increased by 0.63 points (95% CI [0.16, 1.10]), ‘Summarizing air quality data in a visual way for others’ increased by 0.62 points (95% CI [0.15, 1.09]), and ‘Engaging community members in environmental research’ increased by 0.56 points (95% CI [0.16, 0.97]). There was an even more significant increase (p=<0.001) in the confidence levels between pre and one year follow up surveys in two statements; confidence in the statement ‘Using PurpleAir sensors to collect air quality data’ increased by 1.28 points (95% CI [0.83, 1.73]), and ‘Accessing data received from PurpleAir sensors placed in a community’ increased by 1.04 points (95% CI [0.60, 1.49]). These confidence level responses are further outlined in Figure 2.

Figure 2. Confidence levels on pre, post, and one year follow up surveys among cohort 1 and 2. .

Figure 2. Confidence levels on pre, post, and one year follow up surveys among cohort 1 and 2. .

Changes in confidence across trainee roles (community partners, local government officials, student participants, academic partners) are presented in Table 4. Overall, community members did not show significant increases in confidence levels in various statements that academic partners did show significantly increased confidence in. When the pre survey is compared to the post survey for only academic partners, confidence in the statement‘Describing the health risks associated with air pollution’ increased by 0.67 points (95% CI [0.082, 1.25]), ‘Understanding the challenges faced by 'environmental justice' communities’increased by 0.87 points (95% CI [0.23, 1.50]), and ‘Summarizing air quality data in a visual way for others’ increased by 1.03 points (95% CI [0.31, 1.75]) whereas community partners did not show similar trends. When the pre survey is compared to the post survey, both academic and community groups did show significant increases in confidence levels in the statements ‘Using PurpleAir sensors to collect air quality data’ and ‘Accessing data received from PurpleAirsensors placed in a community’ which are further outlined in Table 4. Furthermore, when the pre survey is compared to the one year follow-up survey, both groups also showed significantly increased confidence in ‘Using PurpleAir sensors to collect air quality data’, ‘Accessing data received from PurpleAir sensors placed in a community’, and ‘Analyzing data received from PurpleAir sensors placed in a community’. These confidence levels are further described in Table 4.

Participants also reported successful personal, legislative, or research outcomes associated with their projects (Figure 3). Fifty percent of respondents reported engagement on social media, 41% reported sharing their project or results with a community-based organization, and 32% reported a presentation at a non-academic conference; further explanations of these outcome on the REDCap surveys showed that many cohort members presented informational talks at community meetings or to stakeholders.

Participants noted that other forms of information sharing not previously stated above included educational dashboards, programs in high school science classes, and providing a training module for students on a Tribal reservation. Participants were also asked to discuss whether they would recommend the RISE training program to others and why or why not. Participants consistently commented on the thoroughness and well-roundedness of the training. Participants also reported that it was helpful to meet other teams and professionals and to be able to learn from each other. Several participants stated that this would be a helpful place to start for organizations curious about starting their own air quality monitoring programs.

The response rate for participants responding to webinar evaluation surveys was 25%. The monthly webinar evaluations revealed that the participants assigned the highest ratings to the “expertise of the presenter(s)” category and assigned the lowest ratings to the “active involvement of participants in the learning experience” category. Overall, the average ratings for each evaluation category of the webinar series increased from cohort 1 to cohort 2.

Focus Groups: Seven members of cohort 1 (four academic partners and three community partners) and three members of cohort 2 (one academic partner and two community partners) participated in the end of program focus groups. Qualitative data obtained from the focus groups demonstrated several key takeaways regarding structure and content of the RISE Communities training program. Focus group participants across both cohorts identified several strengths of the RISE Communities program, including the in-person programming, easy access to experts, increased comfort with community-based research, and enhanced team and community relationships. All participants agreed the provision of air sensors was a critical facilitator in their ability to undertake this type of research. However, trainees also expressed a continued need for training in the technical aspects of air monitoring (from initial sensor set-up to continued deployment over time), sensor data collection and management, and help with managing project timelines, particularly when research team members face competing demands for their time. Participants favored in-person activities and cited the ability to form stronger relationships with their team members, visit community partner sites, and have live demonstrations of air quality monitors. Participants stated that the program allowed for team building. The community-based research components of the training enhanced their ability to connect with the communities they served. When asked to define success and whether their projects had been successful, participants offered a range of responses from deploying air sensors in their communities, training community members in air monitoring, and acquiring funding to continue their efforts. Most participants reported that their goals had shifted throughout the year, but they felt their projects had been successful overall. Participants also reported continuation of RISE projects and project growth beyond initial proposals. Recommendations for program improvement varied by cohort, with the first cohort suggesting that the data training be more tailored to the audience, potentially via breakout sessions for academic partners versus community partners, and more time for teams to work together on their projects. In cohort two, participants requested a more in-depth introduction to different types of air quality monitors with more discussion of the pros and cons of each and enhancement of the engagement and interactivity in the virtual webinars.

4. Discussion

The RISE Communities is a unique training program intended to promote community-academic partnerships focused on air quality in affected neighborhoods. Participants reported statistically significant increases in confidence regarding air quality fundamentals, low-cost air sensor usage, and community engaged research, as evidenced by pre, post, and one year follow up survey data. In addition, participants reported significantly increased confidence levels in applying PurpleAir monitors in their communities and accessing the data from these monitors in the short and long term follow up. This is an especially important outcome because technical challenges related to sensor set-up, use, and data collection may be a roadblock for community and academic organizations looking to deploy low-cost sensors to engage their communities in air quality research [22]. Demonstrating long-term retention of confidence in using low-cost air quality monitors indicates that a short training program can increase the accessibility of these tools [27].

Community-engaged research is an essential approach to ensure community voice and lived experiences are recognized in research and to increase the impact of research to improve individual and community health [11,16,23]. Studies investigating the impact of this community work on research topics such as air quality have shown promising outcomes for engaging young people in advocacy, increasing the skill bases of communities, and providing more insight into how these issues impact the daily lives of the people affected by them [10,15,24]. The findings in this report further evaluate the long-term efficacy of training community members in the technical and teamwork skills needed for a community-based research project. This study also provides a comprehensive review of the short and long term experiences that a training program such as RISE Communities can provide to academic-community research partners; something that is not well documented in the literature.

Another community air quality monitoring project out of Denver noted technical and language barriers with their monitors, citing a lack of user-friendly interfaces and need for more technical training [25]. This is a barrier similarly faced and discussed in our study findings. Community members also seem to have trouble extrapolating the raw data from monitoring into conceptual ideas about pollution, particulate matter, or how environmental actions impact these numbers [26]. Furthermore, participants relate air quality findings to their own personal actions or environments more often than they apply them broadly to an entire environmental justice community, indicating disconnect between the principles of EJ communities and how participants actually use their monitors [25,26]. Another study continued to explore this idea, finding that participants who hosted monitors wanted to know how the data would personally impact them reinforcing the importance of clear expectations around the limits and viability of the data collected from community monitoring [27]. Future air quality monitoring programs should provide clear expectations about how data can realistically be used, explain the collaborative role of a citizen scientist, and provide reminders about the inherent advocacy work that frequently underpins community air monitoring programs.

Results of this study indicate that intensive training in community-engaged research, team function, and air sensor data collection and analysis can contribute meaningfully to both short and long term gains in confidence using such methods. The community partners in this study demonstrated less retention of confidence in data analysis, data visualization, and research partnership skills when compared to academic partners. These results suggest that community members’ learning needs may be different from their academic partners, and that a training program in air sensor technology may need to provide more tailored learning opportunities for both audiences. Despite this, both academic and community member participants in this study provided overwhelmingly positive feedback on the program, citing its availability of experts for questions and brainstorming, in-person introduction to fellow team members and organizations, and continued guidance and checks ins during the year as critical components that contributed to the success of their projects.

In addition to its strengths, there are some limitations to this analysis that should be considered. First, our sample included 31 participants who were members of the first two cohorts of a five-year training grant. Additional data will be collected in cohorts 3-5 and evaluated separately. Webinar evaluation surveys had low response rates overall, limiting our ability to draw meaningful conclusions about the value and quality of the webinar trainings, from the participant perspective. Participants in cohort 1 indicated survey fatigue, so we reduced the number of surveys in cohort 2, but this fatigue could have contributed to lower response rates for the monthly webinars. Due to the nature of the program, we could not reasonably include a comparison group to determine differences in experience from the RISE Communities participants compared to community-academic partners who undertook similar projects without a formal training program to support them. Finally, the results of this study are specific to methods and trainees in the RISE Communities program and are therefore difficult to generalize to other training approaches or audiences.

5. Conclusions

This mixed methods study evaluated the impact of the RISE Communities training program on participant confidence with using low cost air sensors, developing and maintaining effective partnerships between communities and academic partners, and achieving partnership goals within the one year cohort period. Our study found that participants consistently reported increased confidence levels regarding using low cost air sensors, data manipulation, and understanding environmental justice communities immediately following the formal training and at the one year follow up. Our study also demonstrated that the partnerships formed during the training resulted in tangible outcomes such as presentations, publications, and grant applications. Furthermore, participants provided positive feedback on the topics, presenters, and assistance with technical difficulties throughout the year. Constructive feedback mainly centered around the challenges of data management and visualization, demonstrating a further need for instruction on the technical aspects of low-cost air sensor research projects. These findings demonstrate how a formal training program can enhance confidence in community-academic partners who want to conduct their own air quality monitoring projects, more effectively engage in community-based research, and increase their knowledge of other air quality monitoring efforts. This study also demonstrates the long-term effectiveness of formal training, with evidence of retained confidence and continued pursuit of project goals at the one year follow up period. Future training programs should consider this program’s participant feedback on the difficulty of data management and devote more training time to this topic, revisit the topic often, and offer separate or individualized training to participants such as community members who may not have any background in the technical aspects of research. Because we did not see significant retention of data skills when observing only community partners, future studies may investigate how a tailored training impacts retention of these data skills. This study offers a promising look into how a formal training program in community-engaged air quality research using low-cost sensors can positively impact community-academic research teams.