Status and Challenges of Medical History-Taking in Developing Countries and an Affordable Digital Solution to Tackle Them

1. Introduction

"History-taking" is a crucial procedure in healthcare, aimed at extracting vital personal, psycho-social, and symptomatic information from patients [1]. This process forms the cornerstone for diagnosing and administering appropriate medical care, given its pivotal role in patient assessment [2]. A study conducted in the USA with 80 outpatients revealed that around 76% of diagnoses depend on a comprehensive medical history [3]. Traditionally, this involved paper-based patient history forms or direct physician inquiries, especially in developing countries. However, analog methods have limitations, with potential inaccuracies due to recall errors and time constraints [4].

In Bangladesh, physicians’ average consultation time is less than a minute, the world’s lowest [5]. The limited consultation time makes the history-taking tasks worse, compounded by the scarcity of physicians and resources. Nowadays, healthcare providers turn to Electronic Health Records (EHRs) to handle patients’ medical records. But existing EHRs often lack interoperability and fail to capture a patient’s life-long medical history, leading to time-consuming data interpretation for physicians.

During the process of taking patient histories, doctors must adhere to several procedural steps. Although there is no fixed number of items for this practice, certain recommended elements exist. This study has outlined ten standard items that are commonly present in the literature on patient history-taking : (1) Greetings to the Patient (2) Inquiry about the Chief Complaint (CC) (3) History of Present Illness (HPI) (4) Acquisition of Past Medical History (PMH) (5) Inquiry about Medications (6) Collection of Family History (7) Assessment of Social History (8) Systematic Review (9) Exploration of the Patient’s Ideas, Concerns, and Expectations (ICE), and (10) Expression of Empathy. Typically, in developing countries, doctors focus mainly on the patient’s chief complaint (CC) and skip most of the steps [6,7,8].

To investigate physicians’ burdens and history-taking challenges, a survey was conducted, aiming to address the following research questions (RQ): (RQ1) What is the present workload situation among physicians in Bangladesh? (RQ2) To what extent do physicians face difficulties during the history-taking process? (RQ3) How do physicians and their organizations manage patients’ medical histories? (RQ4) To what extent do physicians follow all necessary history-taking steps to ensure quality healthcare services for patients? (RQ5) What level of interest do physicians demonstrate in adopting a digital solution for the management of patients’ medical histories?

Covering a comprehensive medical history in a short consultation period poses challenges. Innovative digital health systems like the Portable Health Clinic and the efficient health data management system offer potential solutions to grab necessary medical history within limited consultation time [9,10,11]. This study aims to explore physician challenges in history-taking due to heavy workloads and introduces the Smart Health Gantt Chart(SHGC), an AI-based system for lifelong medical history management, to create clinical impact.

2. Methods

This study addresses the challenges encountered by physicians during medical history-taking in Bangladesh. Details of the methodology are explained below:

2.1. Study Design, Population, and Setting

The study employed a quantitative research design, engaging physicians across diverse settings, including urban and rural areas, both in the private and public sectors. The sample covered General Practitioners (GPs) and various medical specialists, including dentists, with a minimum of 2 years of professional experience. The survey, conducted between July and August 2023 in Bangladesh, welcomed voluntary participation.

2.2. Data Collection Tools and Techniques

Data collection was conducted using an online questionnaire via Google Forms, comprising 22 questions with closed-ended formats—Yes/No, multiple choice, multiple select, Likert scale, and semantic differential scale questions. The questions are summarized in Table 1. Specific queries on experience, working hours, patient count, etc., offered 5–6 selectable slots. Certified physicians in Bangladesh were invited to participate, ensuring anonymity and privacy by collecting emails to prevent duplication and securing prior consent. Duplication and error checks were performed, identifying no errors. All fields were made compulsory, ensuring no missing values or incomplete forms. A total of 104 eligible forms were analyzed. Python programming, utilizing libraries like Pandas and Numpy, was employed for data pre-processing, visualization, and model training. Feature engineering techniques, such as one-hot encoding and label encoding, were applied. Visualization tools like Matplotlib, Plotly, and Seaborn were utilized, and the ordinal regression model was constructed using Statsmodels and Sklearn.

3. Results: Medical History Taking Challenges

This section reports issues by analyzing the data obtained from our survey.

3.1. Respondents’ Demography

Table 2 shows the survey details and respondents’ demography.

3.2. Observation 1: More than 90% of physicians suffer from workload

To answer the RQ1, respondents were asked to rate their cognitive workload experience using a Likert scale with the following options: (1) Very low (2) Low (3) Moderate (4) High, and (5) Very high. It was observed that 92.31% of physicians marked (3) (4) and (5). Sixty-six (66.35%) selected (4) and (5), which means they felt workload every time or almost every time, while 25.96% selected (3) which means they experienced workload occasionally. The average workload rating derived from this scale was around 3.79 out of 5, indicating a moderate workload on average. This finding is significant for understanding an indication of the actual stress and pressure that physicians face in their professional roles.

Figure 1 illustrates the relationship between workload and ultimate working hours. It shows that physicians working more than 13 hours a day experience extreme workload, with 82% of them feeling workload "(5) Very high". For those working 10-13 hours, 54% feel workload " (4) High", and 23% experience it "(5) Very high".

Inquiring about physicians’ work hours revealed a common trend: most work beyond their official duties, often engaging in other clinics or institutions. Around 54.81% work primarily in medical college hospitals, while 19.30% also practice in private clinics. On average, they work about 8.89 hours officially and 9.53 hours in total, with notable variations reflected in the standard deviations of 3.11 and 3.02, respectively. This extended workload may strain their services, potentially impacting their performance and patient care quality.

3.3. Observation 2: More than 80% of Physicians encounter history-taking challenges

A Likert scale question was asked the respondents to understand the frequency of challenges encountered during history-taking, aiming to get the answer to the RQ2. It revealing that, 1.92% indicated "(1) Never", 9.62% "(2) Almost never", 52.88% "(3) Occasionally/Sometimes", 29.81% "(4) Almost every time", and 5.77% "(5) Every time". This led to an overall 88.46% experiencing challenges occasionally, almost every time, or every time. The statistical analysis of these responses disclosed a mean Likert scale score of 3.28, denoting that, on average, participants encountered challenges primarily at an "Occasionally/Sometimes" level. Furthermore, the median, recorded at 3.00, signifies that half of the responses reported challenges at this level or lower, illustrating a balanced distribution. Additionally, the standard deviation of 0.7900 indicates a relatively minor variance from the mean, suggesting a consistent level of challenges encountered during history-taking among the participants.

Furthermore, inquiring about respondents’ perceptions regarding the level of efficiency of their workplaces in facilitating patient history-taking. The Likert scale question answering options were: "(5) Very Good", "(4) Good", "(3) Acceptable", "(2) Poor", "(1) Very Poor". The findings indicated that 46.15% physicians found it "Acceptable", 30.77% rated it as good (4,5), and 23.07 rated it as poor (1,2). These findings indicate that physicians and their organizations both are facing challenges in history-taking issues.

A correlation matrix was produced to assess the significance of factors affecting doctors’ challenges in history-taking. The factors are listed in Table 3, and the question items mentioned in Table 1. Figure 2 displays the selected variables for analysis, excluding (Q4, Q5, Q7, Q10, Q11, Q12, Q13, Q19, Q22) due to high one-hot encoding correlation. The highest correlation (0.51) between Q7 and Q8 led to the removal of Q7 from the ordinal regression model, resulting in improved performance with Q8. Subsequently, an ordinal regression model was trained to evaluate the impact of independent variables on history-taking challenges.

Table 3 includes coefficients, standard errors, z-values, and p-values for each variable. Statistical significance was observed for Q2, and Q9 respectively. This suggests that these factors have a notable impact on the challenges of history-taking. Q3 and Q6 approached significance with p-values of 0.076 and 0.088, indicating potential but not definitive impacts. Other variables, such as Q1, Q8, and Q14, showed higher p-values, indicating that they are less likely to have a statistically significant impact on the challenges faced in history taking.

3.4. Observation 3: Only 4.81% physicians use digital medical history

To answer the RQ3 The findings were: that only 4.81% of physicians exclusively rely on digital formats for maintaining patient records. Shockingly, approximately half of the surveyed physicians (50%) still adhere solely to paper-based records, while 23.08% maintain a hybrid system utilizing both paper-based and digital records. Equally concerning is that 22.12% of the surveyed physicians do not store any patient medical history at all. Not only in developing countries but also in developed countries, there is no complete dependence on digital health data management. A study conducted in Kuwait with a total of 98 participants from different organizations responded to the survey. The study results revealed that 43% of hospitals were using paper-based medical records, compared to 53% that used both paper-based and electronic media [12].

Paper-based records are traditional, hard to manage, error-prone, and resource-intensive. On the other hand, electronic health records (EHR) are recognized as more efficient and user-friendly, enabling doctors to save valuable time [13,14,15]. The low adoption of digital history management systems underscores the pressing need for their widespread implementation. Furthermore, it’s worth noting that enhanced utilization of EHR systems and their delegation were found to be independently associated with heightened levels of physician productivity [16]. Web-based technology and the use of Internet of Things (IoT) sensors can help improve healthcare data management, considering satisfaction, effectiveness, and efficiency [17]. The hypothesis suggested that those maintaining paper-based records or not storing any data might encounter challenges in history taking "(5) Every time".

Figure 3 shows the relationship between history history-taking approach and history history-taking challenges level rated by the physicians. it is surprising to note that among those not storing any data, none face challenges "(5) Every time". Nevertheless, a total of 82% encounter challenges "(4) Almost every time" or "(3) Occasionally". Among those using paper-based documents, 8% face history-taking challenges every time, 28% face challenges almost every time, and 55% face challenges occasionally. The graph indicates that among users of digital records, 60% "(2) Almost never" face challenges in history taking, while 40% face challenges occasionally.

3.5. Observation 4: More than 70% physicians are unable to follow the necessary history-taking steps

To address the RQ4, this study revealed that in the process of history-taking 70.19% could not follow all necessary steps (10 steps) during the consultation period. The step most frequently followed is "Inquiring about the Chief Complaint (CC)" at a rate of 94.23%, while the least followed step is "Exploring the Patient’s Ideas, Concerns, and Expectations (ICE)" at 48.08%. Furthermore, 34.62% of doctors identified inconsistent responses as the primary challenge in obtaining a thorough medical history. Meanwhile, 30.77% attributed a lack of sufficient time due to a high volume of patients as a significant barrier to conducting comprehensive medical history assessments.

3.6. Observation 5: More than 90% physicians expressed a need to use digital systems for patients’ medical history management

Figure 4 shows the relationship between history-taking challenges and the willingness to adopt the proposed digital system. To answer the RQ5, the study found that 64.42% expressed extreme interest, 16.35% were very interested, and 12.50% moderately interested. In total, 93.27% showed high to moderate interest, with an average Likert scale score of 4.37. The median score of 5.00 suggests that half rated themselves as "Extremely Interested." Surprisingly, among those facing history-taking challenges, willingness to adopt the system varied: 83% for constant challenges, 77% for almost every time, 53% for occasional challenges, and even 80% for those rarely encountering challenges. Notably, 50% of those who never faced challenges expressed interest in the system.

4. Digital Solution to Tackle History Taking Challenges

This section contributes by introducing a unique visualization system called the Smart Health Gantt Chart (SHGC). This system aims to alleviate history-taking challenges by offering lifelong medical history management. Further details about its concept and functions are explained below.

4.1. Concept of the SHGC

The healthcare industry has made improvements in the Health Information Systems (HIS). However, several HIS issues still need to be addressed, including the optimum utilization of the system for better management of patient care and dissatisfaction with the current patient records systems, especially in areas like interface and time taken to input data [18]. To overcome those limitations along with history-taking challenges, this study introduces the Smart Health Gantt Chart (SHGC).

This is an AI-based solution designed to visualize a lifelong medical history in a summarized view for busy physicians. This enables them to quickly and accurately understand patients’ medical histories. Data from multiple Electronic Medical Records (EMR), Electronic Health Records (EHR), and Personal Health Records (PHRs) will be connected to this visualization tool. Medical history is typically generated in three categories. To collect standard medical history from these diverse sources, the following features are included in the SHGC, as illustrated in Figure 5: adoption of Optical Character Recognition (OCR), Automatic Speech Recognition (ASR), or Speech-to-Text (STT) technology to convert images and voice recordings into text. Moreover, adherence to the FHIR (Fast Healthcare Interoperability Resources) and OpenEHR standards ensures consistency across all health data. Additionally, data pre-processing and preparation are conducted to perform error checking and eliminate redundant, incomplete, or incorrect data [10]. After the collection of standard medical history, this SHGC is able to present a comprehensive overview in a single chart within one window. The chart will be accessible online, offering patients full control. Patients can share it with their physicians whenever necessary. The SHGC can save physicians time, assist in efficient decision-making, reduce costs for patients, and increase satisfaction for both physicians and patients by efficient management of lifelong medical records.

4.2. Functional Requirements

The SHGC comprises four major components and several additional features. The major components include Patient Profile, Health Event Categories, Health Events, and Detail Health Information. Among the upper features is the Age Bar, while below, there is the Timeline (years from the birth year). These two elements are interconnected with health events, enabling doctors to understand at which age and in which year specific health issues were experienced. The detailed view of the SHGC is shown in Figure 6.

Patient Profile: SHGC’s Patient Profile stores demographic and unchanging health data, including identification details, blood type, gender, marital status, and health-related habits like smoking, alcohol, allergies, diet, sleep, and exercise. This data aids in initial consultations, offering insights into the patient’s overall health.

Health Event Categories: SHGC addresses the lack of standardized health event categorization. By analyzing 100 medical history forms, it proposes eight universal categories, aiming to unify data collection methods used by hospitals and clinics.

Health Events: In SHGC, Health events are defined as "the summary of specific occurrences related to health complexities, preventive measures, and treatment actions". These health events are visualized by tracking age and year to understand when and at what age, specific health issues occurred. Furthermore, future health events can be visualized by analyzing past medical history with the assistance of AI prediction mechanisms.

Detail Health Information: SHGC offers summarized health details on a Gantt chart with mouse-over pop-ups for quick insights. Clickable summaries enable deeper dives. Beneath the Gantt chart, detailed information includes prescriptions, diagnostic and imaging reports, and textual notes, facilitating better understanding for doctors.

4.3. The prototype of the tool

The SHGC system’s front end utilizes Bootstrap and JavaScript for flexibility and performance, while the back end is built with the Laravel PHP web framework, known for its Model-View-Controller architecture and robust features, including a modular packaging system and dependency management. Powered by PHP 8.1 and Laravel 9, with MySQL as the selected database, the system’s initial prototype development didn’t cover all features, such as standardizing past medical history collection, transforming analog-to-digital records, developing AI-based health predictions, and verifying data authenticity. These features are currently being explored in various research domains and planned for integration into the next prototype version. To enhance usability, we are also prioritizing User Interface (UI) and User Experience (UX) design. These are essential for this kind of application. Their perspectives must be considered to design the most positive experience for users. These can be achieved through the following seven UX goals: safety, relationship, usability, inspiration, flexibility, efficiency, and accomplishment [19].

In the health industry, the use of data is growing in importance. Big Data needs to have effective data governance, which includes measures to manage and control the use of data and to enhance data quality, availability, and integrity [20]. In the development of SHGC, we are considering adopting data standardization such as Fast Healthcare Interoperability Resources (FHIR), OpenEHR, and relevant local rules and regulations.

The practical implementation of the SHGC application could address current limitations in history taking, as validated by physician feedback in the results section. Patient feedback will be incorporated in future studies, along with further evaluation and modifications.

5. Discussion

The study highlights challenges faced by Bangladeshi physicians, with 92.67% expressing feelings of being overburdened, which adversely affects their patient care and history-taking. This finding aligns with previous research. Sultana et al. (2020) reported that 21.3% of surveyed physicians in Bangladesh experienced moderate to high burnout due to work pressure [21]. Additionally, this study revealed that the average ultimate working hour for physicians in Bangladesh is 9.53 hours. This is consistent with findings from Roy et al. (2017), who observed daily official working hours within private facilities ranging from 8 to 10 hours, significantly higher than the usual 6.5 hours in the public sector. Notably, 58% of all respondents worked 6 days per week (public 58%; private 64%), 41% worked 7 days per week (public 42%; private 35%), and only 1% of private physicians worked 5 days per week [22]. Challenges identified include limited access to patient records, digitization gaps, and time constraints during consultations. Medical record digitization could greatly assist, yet 23.08% have limited or no access to records. Only 4.81% manage digital health records, emphasizing the need for digital solutions. The SHGC system is proposed to assist physicians in history and efficient decision-making by offering lifelong medical history visualization.

The study encounters several limitations: (1) The sample size of respondents was only 104 physicians, and a larger sample size would enable more accurate observations. (2) Since the questionnaire was made available online to collect responses, the obtained data lacks balance in terms of gender, geographical location, and discipline, among other factors. (3) The responses consist of self-reported organic data. It would be preferable to obtain observation-based data on physicians’ activities from day-to-day operations. Addressing these aspects could be a focus for future research studies.

6. Conclusions

In developing countries, the shortage of physicians forces them to work long hours to accommodate the high number of patients. This study conducted a cross-sectional survey to assess their workload, challenges in history-taking, and their compliance with history-taking steps. A patient’s lifelong medical history visualization system with necessary features is introduced that can serve as a solution to the challenges identified by the survey. Results show that 92.67% of physicians face overwhelming workloads, 88.46% struggle with history-taking challenges, and only 4.81% use digital records. An AI-based Smart Health Gantt Chart (SHGC) system, introduced to physicians, aims to streamline patient histories. Initial feedback suggests that 93.27% of physicians are willing to adopt this system. With its potential to improve efficiency and patient care, the SHGC system can make a significant impact in Bangladesh and other developing countries by benefiting physicians, improving patient-centered care, and creating a clinical impact.