Submitted:
09 July 2025
Posted:
10 July 2025
You are already at the latest version
Abstract
Keywords:
1. Introduction
2. Materials and Methods
2.1. Study Design
2.2. Ethical Approval and Data Sharing
2.3. Needs Identification: Brief Summary of Field Evidence
2.4. FFT Tool Conceptualization and Drafting
2.5. External Expert Validation: Open Call and Delphi Process
2.5.1. Expert Recruitment: Open Call and Professional Outreach
2.5.2. Delphi Round 1: Structured Tool Review
2.5.3. Feedback Synthesis and FFT Revision
2.5.4. Delphi Round 2: Final Consensus Validation
2.5.5. Quantitative and Qualitative Validation Outcomes
2.6. Data Management, Coding, and Accessibility
2.7. Statement on Digital and Artificial Intelligence Tools
3. Results
3.1. Expert Validation Panel: Composition and Depth
3.2. The Delphi Validation Process: Quantitative and Qualitative Insights
3.2.1. Round 1: Establishing the Diagnostic Baseline
- “The most important barrier, according to me, would be untrained CHWs. If they are not proficient, have poor digital literacy, or even if all other resources are available, the output and impact will be poor.”
- “Motivation should be checked objectively, and the regular use of the app can be verified by digital records. Steps 5 and 6 can be swapped depending on field testing outcomes.”
- “Language barrier is not purely a technical problem and should be integrated with proficiency checks.”
- “Grievance redressal timelines should be flexible; seven days may not be realistic in all settings.”
3.2.2. Round 2: Iterative Refinement and Consensus Building
- “The FFT is now clear and highly useful for both block-level and higher-level field visits. The self-check and grievance steps add value. It is fit for scale-up.”
- “For supervisors, this is a practical self-assessment tool to reflect on their preparedness and the adequacy of resources. For development partners, it identifies gaps for targeted capacity building.”
3.3. The Final Validated Fast and Frugal Tree (FFT)
3.4. Synthesis: Overall Patterns and Implications
- Quantitative Synthesis: The FFT moved from a baseline of basic acceptability to robust excellence, with all items validated at I-CVI ≥ 0.78, S-CVI/Ave = 0.901, and substantial gains in S-CVI/UA.
- Qualitative Synthesis: Thematic analysis highlighted recurring strengths (clarity, practical value, contextual fit) and residual challenges (workflow nuances, adaptation flexibility). Panelists’ insights drove targeted revisions, resulting in a tool that is both technically sound and field realistic.
4. Discussion
4.1. Synthesis with Previous Literature
4.2. Advances in Decision-Support for Digital Health
4.3. Practical Relevance and Usability
4.4. Comparison with National and Global Initiatives
4.5. Strengths and Limitations
4.6. Implementation, Scalability, and Policy Implications
4.7. Directions for Future Research
5. Conclusions
Supplementary Materials
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Abbreviations
| Abbreviation | Full Form |
| AES | Acute Encephalitis Syndrome |
| ANM | Auxiliary Nurse Midwife |
| ASHA | Accredited Social Health Activist |
| AWW | Anganwadi Worker |
| CDPO | Child Development Project Officer |
| CHW | Community Health Worker |
| CVI | Content Validity Index |
| FFT | Fast and Frugal Tree |
| HDI | Human Development Index |
| HIS | Health Information Systems |
| ICDS | Integrated Child Development Services |
| IEC | Institutional Ethics Committee |
| I-CVI | Item-level Content Validity Index |
| LMICs | Low- and Middle-Income Countries |
| mHealth | Mobile Health |
| MOIC | Medical Officer In Charge |
| NGO | Non-Governmental Organization |
| NFHS-5 | National Family Health Survey-5 |
| S-CVI | Scale-level Content Validity Index |
| S-CVI/Ave | Scale-level Content Validity Index (Average) |
| S-CVI/UA | Scale-level Content Validity Index (Universal Agreement) |
| SDG | Sustainable Development Goal |
| UHC | Universal Health Coverage |
| UN | United Nations |
| WHO | World Health Organization |
References
- World Health Organization WHO-Global Strategy on Digital Health 2020-2025; 2021; ISBN 9789240020924.
- Bank, W. Ayushman Bharat Digital Mission’s Integrated Digital Health Ecosystem Is the Foundation of Universal Citizen-Centered Health Care in India;
- Labrique, A.B.; Wadhwani, C.; Williams, K.A.; Lamptey, P.; Hesp, C.; Luk, R.; Aerts, A. Best Practices in Scaling Digital Health in Low and Middle Income Countries. 2018, 1–8.
- Welfare, M. of H. and F. National Health Policy India 2017; 2017;
- National Health Authority (NHA), I. ABDM Handbook Ayushman Bharat Digital Mission 2021, 1–67.
- National Health Authority (NHA), I. Ayushman Bharat Digital Mission Draft Health Data Management Policy 2022; 2022;
- Sharma, R.S.; Rohatgi, A.; Jain, S.; Singh, D. The Ayushman Bharat Digital Mission (ABDM): Making of India’s Digital Health Story. CSI Transactions on ICT 2023, 11, 3–9. [CrossRef]
- Sarin, E.; Bisht, N.; Mohanty, J.S.; Chandra Joshi, N.; Kumar, A.; Dey, S.; Kumar, H. Putting the Local Back into Planning-Experiences and Perceptions of State and District Health Functionaries of Seven Aspirational Districts in India on an Innovative Planning Capacity Building Approach. International Journal of Health Planning and Management 2021, 36, 2248–2262. [CrossRef]
- Bhatia, V.; Rath, R.; Singh, A. Developing the Underdeveloped: Aspirational Districts Program from Public Health Point of View. Indian Journal of Community and Family Medicine 2018, 4, 1–3. [CrossRef]
- 2018; 10. NITI Aayog Transformation of Aspirational Districts Baseline Ranking Report by NITI Aayog; 2018;
- Malaviya, P.; Picado, A.; Hasker, E.; Ostyn, B.; Kansal, S.; Singh, R.P.; Shankar, R.; Boelaert, M.; Sundar, S. Health & Demographic Surveillance System Profile: The Muzaffarpur-Tmrc Health and Demographic Surveillance System. Int J Epidemiol 2014, 43, 1450–1457. [CrossRef]
- 2011; 12. Government of India District Census Handbook Muzaffarpur 2011; 2011;
- 2021; 13. Ministry of Health and Family Welfare NFHS 5 District Fact Sheet Muzaffarpur Bihar; Muzaffarpur, 2021;
- Kalne, P.S.; Kalne, P.S.; Mehendale, A.M. Acknowledging the Role of Community Health Workers in Providing Essential Healthcare Services in Rural India-A Review. Cureus 2022, 14, 1–7. [CrossRef]
- Nadella, P.; Subramanian, S. V.; Roman-Urrestarazu, A. The Impact of Community Health Workers on Antenatal and Infant Health in India: A Cross-Sectional Study. SSM Popul Health 2021, 15, 100872. [CrossRef]
- Blondino, C.T.; Knoepflmacher, A.; Johnson, I.; Fox, C.; Friedman, L. The Use and Potential Impact of Digital Health Tools at the Community Level: Results from a Multi-Country Survey of Community Health Workers. BMC Public Health 2024, 24, 1–14. [CrossRef]
- Feroz, A.S.; Khoja, A.; Saleem, S. Equipping Community Health Workers with Digital Tools for Pandemic Response in LMICs. Archives of Public Health 2021, 79, 10–13. [CrossRef]
- Owoyemi, A.; Osuchukwu, J.I.; Azubuike, C.; Ikpe, R.K.; Nwachukwu, B.C.; Akinde, C.B.; Biokoro, G.W.; Ajose, A.B.; Nwokoma, E.I.; Mfon, N.E.; et al. Digital Solutions for Community and Primary Health Workers: Lessons From Implementations in Africa. Front Digit Health 2022, 4, 1–9. [CrossRef]
- Verdezoto, N.; Bagalkot, N.; Akbar, S.Z.; Sharma, S.; Mackintosh, N.; Harrington, D.; Griffiths, P. The Invisible Work of Maintenance in Community Health: Challenges and Opportunities for Digital Health in Supporting Frontline Health Workers in Karnataka, South India. Proc ACM Hum Comput Interact 2021, 5, 1–31. [CrossRef]
- Pgdha, V.K.; Chaudhary, S.; Sharma, A.K. Challenges in Implementing Digital Health Services in Rural India. J Int Soc Telemed eHealth 2023, 11, 1–5. [CrossRef]
- Gadsden, T.; Mabunda, S.A.; Palagyi, A.; Maharani, A.; Sujarwoto, S.; Baddeley, M.; Jan, S. Performance-Based Incentives and Community Health Workers’ Outputs, a Systematic Review. Bull World Health Organ 2021, 99, 805–818. [CrossRef]
- Borges do Nascimento, I.J.; Abdulazeem, H.; Vasanthan, L.T.; Martinez, E.Z.; Zucoloto, M.L.; Østengaard, L.; Azzopardi-Muscat, N.; Zapata, T.; Novillo-Ortiz, D. Barriers and Facilitators to Utilizing Digital Health Technologies by Healthcare Professionals. NPJ Digit Med 2023, 6, 1–28. [CrossRef]
- Siyam, A.; Ir, P.; York, D.; Antwi, J.; Amponsah, F.; Rambique, O.; Funzamo, C.; Azeez, A.; Mboera, L.; Kumalija, C.J.; et al. The Burden of Recording and Reporting Health Data in Primary Health Care Facilities in Five Low- and Lower-Middle Income Countries. BMC Health Serv Res 2021, 21, 1–9. [CrossRef]
- Borges do Nascimento, I.J.; Abdulazeem, H.M.; Vasanthan, L.T.; Martinez, E.Z.; Zucoloto, M.L.; Østengaard, L.; Azzopardi-Muscat, N.; Zapata, T.; Novillo-Ortiz, D. The Global Effect of Digital Health Technologies on Health Workers’ Competencies and Health Workplace: An Umbrella Review of Systematic Reviews and Lexical-Based and Sentence-Based Meta-Analysis. Lancet Digit Health 2023, 5, e534–e544. [CrossRef]
- World Health Organization WHO Guideline on Health Policy and System Support to Optimize Community Health Worker Programmes; WHO, 2015; Vol. 16; ISBN 0123456789.
- Hilty, D.M.; Armstrong, C.M.; Smout, S.A.; Crawford, A.; Maheu, M.M.; Drude, K.P.; Chan, S.; Yellowlees, P.M.; Krupinski, E.A. Findings and Guidelines on Provider Technology, Fatigue, and Well-Being: Scoping Review. J Med Internet Res 2022, 24, 1–15. [CrossRef]
- Bosch-Capblanch, X.; Liaqat, S.; Garner, P. Managerial Supervision to Improve Primary Health Care in Low- and Middle-Income Countries. Cochrane Database of Systematic Reviews 2011, 2019. [CrossRef]
- Rashmi Kundapur, Sumit Aggarwal, Rakhal Gaitonde, Anusha Rashmi, L.S.G.; Bavaskar, Arvind Pandey, Y. Challenges Faced by Frontline Health Managers during the Implementation of COVID-19 Related Policies in India: A Qualitative Analysis. Indian Journal of Medical Research 2023, 21–27, doi:DOI: 10.4103/ijmr.ijmr_206_22.
- Vikas Bhatia, Preetam Mahajan, Swayam P. Parida, S.B.; Sahoo, S.S. Challenges in Supervision, Monitoring, and Reporting in Anemia Programme Implementation in Odisha, India: A Qualitative Process Documentation. J Family Med Prim Care 2019, 8, 1365–1369. [CrossRef]
- Karuga, R.N.; Mireku, M.; Muturi, N.; McCollum, R.; Vallieres, F.; Kumar, M.; Taegtmeyer, M.; Otiso, L. Supportive Supervision of Close-to-Community Providers of Health Care: Findings from Action Research Conducted in Two Counties in Kenya. PLoS One 2019, 14, 1–19. [CrossRef]
- Wolfenden, L.; Williams, C.M.; Kingsland, M.; Yoong, S.L.; Nathan, N.; Sutherland, R.; Wiggers, J. Improving the Impact of Public Health Service Delivery and Research: A Decision Tree to Aid Evidence-Based Public Health Practice and Research. Aust N Z J Public Health 2020, 44, 331–332. [CrossRef]
- World Health Organization WHO Guideline Recommendations on Digital Interventions for Health System Strengthening; 2019; Vol. 2;.
- Gigerenzer, G.; Kurzenhauser, S. Fast and Frugal Heuristics In Medical Decision Making. Science and Medicine in Dialogue 2024, 3–16.
- Martignon, L.; Katsikopoulos, K. V.; Woike, J.K. Categorization with Limited Resources: A Family of Simple Heuristics. J Math Psychol 2008, 52, 352–361. [CrossRef]
- Bonner, G. Decision Making for Health Care Professionals: Use of Decision Trees within the Community Mental Health Setting. J Adv Nurs 2001, 35, 349–356. [CrossRef]
- Martignon, L.; Erickson, T.; Viale, R. Transparent, Simple and Robust Fast-and-Frugal Trees and Their Construction. Frontiers in Human Dynamics 2022, 4. [CrossRef]
- John, A.; Nisbett, N.; Barnett, I.; Avula, R.; Menon, P. Factors Influencing the Performance of Community Health Workers: A Qualitative Study of Anganwadi Workers from Bihar, India. PLoS One 2020, 15, 1–17. [CrossRef]
- Singh, S.; Miller, E.; Closser, S. Nurturing Transformative Local Structures of Multisectoral Collaboration for Primary Health Care: Qualitative Insights from Select States in India. BMC Health Serv Res 2024, 24, 1–15. [CrossRef]
- Braun, R.; Catalani, C.; Wimbush, J.; Israelski, D. Community Health Workers and Mobile Technology: A Systematic Review of the Literature. PLoS One 2013, 8, 4–9. [CrossRef]
- Singh, N.S.; Scott, K.; George, A.; Lefevre, A.E.; Ved, R. A Tale of “Politics and Stars Aligning”: Analysing the Sustainability of Scaled up Digital Tools for Front-Line Health Workers in India. BMJ Glob Health 2021, 6, 1–14. [CrossRef]
- Parajuli, R.; Bohara, D.; KC, M.; Shanmuganathan, S.; Mistry, S.K.; Yadav, U.N. Challenges and Opportunities for Implementing Digital Health Interventions in Nepal: A Rapid Review. Front Digit Health 2022, 4, 1–14. [CrossRef]
- Temsah, M.-H.; Aljamaan, F.; Malki, K.H.; Alhasan, K.; Altamimi, I.; Aljarbou, R.; Bazuhair, F.; Alsubaihin, A.; Abdulmajeed, N.; Alshahrani, F.S.; et al. ChatGPT and the Future of Digital Health: A Study on Healthcare Workers’ Perceptions and Expectations. Healthcare 2023, 11, 1–14. [CrossRef]
- Woods, L.; Eden, R.; Macklin, S.; Krivit, J.; Duncan, R.; Murray, H.; Donovan, R.; Sullivan, C. Strengthening Rural Healthcare Outcomes through Digital Health: Qualitative Multi-Site Case Study. BMC Health Serv Res 2024, 24. [CrossRef]
- Venkatesh, V.; Morris, M.G.; Davis, G.B.; Davis, F.D. User Acceptance of Information Technology: Toward a Unified View. MIS Q 2003, 27, 425–478. [CrossRef]
- Arvind Pandey*, Nandini Roy**, R.B. and R.M.M. Health Information System in India: Issues of Data Availability and Quality. Demogr India 2010, 39, 11–128.
- Meghani, A.; Rodríguez, D.C.; Peters, D.H.; Bennett, S. Understanding Reasons for and Strategic Responses to Administrative Health Data Misreporting in an Indian State. Health Policy Plan 2023, 38, 150–160. [CrossRef]
- Yusoff, M.S.B. ABC of Content Validation and Content Validity Index Calculation. Education in Medicine Journal 2019, 11, 49–54. [CrossRef]
- Denise F. Polit; Cheryl Tatano Bec The Content Validity Index: Are You Sure You Know What’s Being Reported? Critique and Recommendations. Res Nurs Health 2006, 489–497. [CrossRef]
- Min, H.; Li, J.; Di, M.; Huang, S.; Sun, X.; Li, T.; Wu, Y. Factors Influencing the Continuance Intention of the Women’s Health WeChat Public Account: An Integrated Model of UTAUT2 and HBM. Front Public Health 2024, 12, 1–12. [CrossRef]
- Gudi, N.; Lakiang, T.; Pattanshetty, S.; Sarbadhikari, S.N.; John, O. Challenges and Prospects in India’s Digital Health Journey. Indian J Public Health 2021, 65, 209–212. [CrossRef]
- Kostkova, P. Grand Challenges in Digital Health. Front Public Health 2015, 3, 1. [CrossRef]
- Cometto, G.; Ford, N.; Pfaffman-Zambruni, J.; Akl, E.A.; Lehmann, U.; McPake, B.; Ballard, M.; Kok, M.; Najafizada, M.; Olaniran, A.; et al. Health Policy and System Support to Optimise Community Health Worker Programmes: An Abridged WHO Guideline. Lancet Glob Health 2018, 6, e1397–e1404. [CrossRef]
- Okolo, C.T.; Kamath, S. It Cannot Do All of My Work: Community Healthworker Perceptions of Ai-Enabled Mobile Health Applications in Rural India. In Proceedings of the Conference on Human Factors in Computing Systems - Proceedings; 2021; pp. 1–20.
- 2023; 54. Ministry of Health and Family Welfare (MoHFW) An E-Booklet on IT Initiatives of NHM; 2023;
- Ministry of Women and Child Development [MoWCD] Mission Saksham Anganwadi and Poshan 2.0 - Scheme Guidelines 2022; 2022;
- World Health Organisation Digital Transformation Handbook for Primary Health Care Optimizing Person-Centred Point of Service Systems; 2024; ISBN 9789240093362.
- D’Elia, A.; Gabbay, M.; Rodgers, S.; Kierans, C.; Jones, E.; Durrani, I.; Thomas, A.; Frith, L. Artificial Intelligence and Health Inequities in Primary Care: A Systematic Scoping Review and Framework. Fam Med Community Health 2022, 10, 1–10. [CrossRef]
- Rahimi, S.A.; Légaré, F.; Sharma, G.; Archambault, P.; Zomahoun, H.T.V.; Chandavong, S.; Rheault, N.; Wong, S.T.; Langlois, L.; Couturier, Y.; et al. Application of Artificial Intelligence in Community-Based Primary Health Care: Systematic Scoping Review and Critical Appraisal. J Med Internet Res 2021, 23, 1–19. [CrossRef]
- Ahmed, S.; Chase, L.E.; Wagnild, J.; Akhter, N.; Sturridge, S.; Clarke, A.; Chowdhary, P.; Mukami, D.; Kasim, A.; Hampshire, K. Community Health Workers and Health Equity in Low- and Middle-Income Countries: Systematic Review and Recommendations for Policy and Practice. Int J Equity Health 2022, 21, 1–30. [CrossRef]
- Braun, V.; Clarke, V. Using Thematic Analysis in Psychology; In Qualittaive Research in Psychology. Uwe Bristol 2006, 3, 77–101.
- Masis, L.; Gichaga, A.; Zerayacob, T.; Lu, C.; Perry, H.B. Community Health Workers at the Dawn of a New Era: 4. Programme Financing. Health Res Policy Syst 2021, 19, 1–18. [CrossRef]
- Woods, L.; Martin, P.; Khor, J.; Guthrie, L.; Sullivan, C. The Right Care in the Right Place: A Scoping Review of Digital Health Education and Training for Rural Healthcare Workers. BMC Health Serv Res 2024, 24, 1–12. [CrossRef]
- Rodrigues, S.M.; Kanduri, A.; Nyamathi, A.; Dutt, N.; Khargonekar, P.; Rahmani, A.M. Digital Health-Enabled Community-Centered Care: Scalable Model to Empower Future Community Health Workers Using Human-in-the-Loop Artificial Intelligence. JMIR Form Res 2022, 6, 1–15. [CrossRef]
- Chettri, S.K.; Deka, R.K.; Saikia, M.J. Bridging the Gap in the Adoption of Trustworthy AI in Indian Healthcare: Challenges and Opportunities. AI (MDPI Switzerland) 2025, 6, 1–18. [CrossRef]
- Cossy-Gantner, A.; Germann, S.; Schwalbe, N.R.; Wahl, B. Artificial Intelligence (AI) and Global Health: How Can AI Contribute to Health in Resource-Poor Settings? BMJ Glob Health 2018, 3, 1–7. [CrossRef]
- Wibowo, M.F.; Pyle, A.; Lim, E.; Ohde, J.W.; Liu, N.; Karlström, J. Insights Into the Current and Future State of AI Adoption Within Health Systems in Southeast Asia: Cross-Sectional Qualitative Study. J Med Internet Res 2025, 27, 1–16. [CrossRef]
- WHO TEAM Digital Health and Innovation (DHI), M.D. and D. (MDD) Generating Evidence for Artificial Intelligence-Based Medical Devices: A Framework for Training, Validation and Evaluation; 2021; ISBN ISBN: 9789240038462.
- World Health Organization Ethics and Governance of Artificial Intelligence for Health. Guidance on Large Multi-Modal Models; 2024; ISBN 9789240029200.
| Expert ID | Role Description | Organization Type | Geographic Base | Experience (Years) | Key Expertise | Field/Block Experience |
| E1 | Senior Program Manager | National NGO | Western India | >8 | Digital health, HIS | Direct implementation |
| E2 | Program Officer | International NGO | India + South Asia | 3–5 | Digital health, mHealth | Direct implementation |
| E3 | Block Health Manager | Government | North India | 5–8 | Block management | Block management |
| E4 | State Program Manager | Maternal Health NGO | Western India | 5–8 | mHealth, HIS | Block management |
| E5 | Senior Specialist | State Tech Partner NGO | India/Fiji | 5–8 | Digital health, HIS | Block management |
| E6 | Senior Program Manager | National Foundation | East India | >8 | HIS | Direct implementation |
| E7 | Senior Specialist | Government Tech Partner | North India | 5–8 | HIS, block management | Block management |
| E8 | Consultant | UN Agency | Northeast India | >8 | HIS, block management | Block management |
| E9 | Health and Climate Fellow | NGO | East India | 3–5 | mHealth, HIS | Block management |
| E10 | Nutrition Officer | International Agency | National HQ | >8 | Digital health, HIS | Block management |
| E11 | Project Coordinator | International NGO | North India | >8 | Digital health, HIS | Block management |
| E12 | State Officer | Government Tech Partner | North India | 3–5 | Block management | Supervision |
| E13 | Chief Technology Officer | Health Tech Firm | Nepal/India | 5–8 | mHealth, HIS | Supervision |
| E14 | District Coordinator | State Health Mission | North India | 3–5 | Block management | Supervision |
| Step | Supervisor Question | If Yes | If No | Suggestive Actions (Examples) |
| 1 | Is a functional smartphone or tablet available with the CHW? | Go to Step 2 | Exit & Act | Provide spare/repair, and audit devices |
| 2 | Does the device have a functional internet connection? | Go to Step 3 | Exit & Act | Switch networks, offline forms, and recharge |
| 3 | Are all apps installed, and has the CHW received recent training? | Go to Step 4 | Exit & Act | Install apps; schedule training |
| 4 | Is the CHW proficient in all digital applications? | Go to Step 5 | Exit & Act | Assign mentor; refresher training |
| 5 | Is the CHW motivated and satisfied with digital tools? | Go to Step 6 | Exit & Act | Motivational workshops, recognition |
| 6 | Are all applications working without technical issues? | Go to Step 7 | Exit & Act | IT ticketing: escalate issues |
| 7 | Is the data fully updated for the last 14 days? | Go to Step 8 | Exit & Act | Data review: align paper/digital records |
| 8 | Is there a time-bound grievance redressal system? | Go to Step 9 | Exit & Act | Implement a simple ticketing system |
| 9 | Are grievances resolved within agreed-upon days? | Go to Step 10 | Exit & Act | Review in meetings; escalate systemic issues |
| 10* | Are resources at the block level adequate (self-efficacy)? | Review regularly | Exit & Act | Self-assess resource gaps; escalate if needed |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).