Generative AI for Culturally Responsive Assessment in Science: A Conceptual Framework

1. Introduction

“Sometimes it’s not the student who is failing the assessment—it might be that the assessment is failing to fully assess the abilities of the student”- [1]

In today’s diverse classrooms, one of the biggest challenges educators face is creating science assessments that genuinely reflect the cultural backgrounds of every student [2]. Science education plays a vital role in equipping students to tackle critical issues like climate change, public health, and technological advancements. Therefore, the purpose of science assessment at the K-12 level, as guided by Framework for K-12 Science Assessment [3] and the Next Generation Science Standards (NGSS; NGSS Lead States, 2013), is to develop students’ scientific literacy by ensuring a deep understanding of essential scientific knowledge and the ability to apply this knowledge to real-world problems [4,5]. This approach encourages students to engage in science and engineering practices in a way that is meaningful to their lives, leveraging their cultural experiences as assets in the learning process [5,6]. Specifically, researchers suggest culturally responsive science assessments (CRSciAs) to connect science education to students’ cultural contexts, making learning more relevant and engaging. However, implementing CRSciAs in K-12 classrooms remains challenging. Teachers face difficulties due to increasing classroom diversity and the dominance of traditional, western-centric assessment methods, which often disadvantage migrant, historically marginalized, and indigenous students. Furthermore, CRSciA practices, though essential, are time-consuming and complex, making them hard to scale effectively [7,8].

Recent development of Generative Artificial Intelligence (GenAI) offers a promising solution to CRSciAs by automating assessment generation and scoring, thereby easing the burden on educators and enabling more equitable, culturally tailored assessments that better reflect students’ diverse backgrounds [9,10,11,11]. GenAI can provide multimodal learning opportunities [12], handling a variety of cultural contexts such as adapting to different languages, symbols, and local knowledge more often more effectively than a single classroom science teacher could manage alone. Specifically in science education, GenAI can assist teachers in creating NGSS-aligned assessment tasks and providing customized feedback based on students’ needs [13].

Nonetheless, the use of GenAI in assessment is not without its challenges. There are legitimate concerns about the fairness and consistency of GenAI-generated assessments, especially when it comes to avoiding bias and cultural stereotypes and ensuring that all students are assessed equitably [14,15]. Without careful consideration, there is a risk that these GenAI technologies could hegemonically reinforce the very inequities they are meant to overcome.

To address these challenges, this study seeks to address these concerns by developing a conceptual framework that brings together the capabilities of GenAI and the core tenets of CRSciA for K-12 education, and further develop a GenAI-CRSciA prompt template strategy using the framework. The GenAI-CRSciA prompt template is designed to guide educators in using GenAI to generate assessment items both scientifically and culturally responsive.

2. Literature Review

There are several studies related to culturally responsive assessments in general, and consequently to CRSciA. This section provides an overview of the impact of standardized assessment, which has traditionally dominated science education. It also examines CRSciA, emphasizing the appropriateness and urgent need and the challenges that persist in implementing CRSciA, and the capabilities of GenAI in addressing these challenges.

Impact of Standardized Assessment

One of the key impacts of standardized assessment (traditional standardized assessment) is the high possibility of creating an “achievement gap”. Achievement gap discourse is prominent particularly in the U.S [16]. The term refers to the disparities in standardized test scores between Native Indigenous, Black, Latina/o, recent immigrants, and White students [17]. Recent studies indicate that current science assessment practices are grounded in the language, experiences, and values of the dominant White culture [18]. While this is a concern, Ladson-Billings [16] further posited that even a focus on the gap is misplaced, instead, we need to look at the “education debt” that has accumulated historically over time. She draws an analogy with the concept of national debt, which she contrasts with that of a national budget deficit to argue the significance of historical inequities in education. Moreover, historically marginalized students including Native Indigenous, Black, Latina/o have had accumulated disadvantages, limited opportunities, and a lack of access to equal education for generations.

CRSciA, therefore aims to address the educational debt in science and not just the gap. This implies that the lack of representation of cultures in science assessments has profound and far-reaching effects across every aspect of life in wider society. A biased science assessment that affects a particular group of students can influence their career goals and limit their contributions to society [19]. Studies shows that unfair assessments have broader societal impacts such as contributing to higher school dropout rates [20]. Students who are unfairly assessed may become disengaged in the educational system, and may be pushed out (drop out) from school as described by Johnston-Goodstar and Roholt [21]. This, in turn, can result in a larger number of unproductive citizens, which negatively affects society by increasing the burden on social services and reducing overall economic productivity [22,23,24]. However, recent research shows that unlike high-stakes assessments, culturally response assessments motivate students and promotes authentic, life-long learning [25,26]. Furthermore, students possess well-developed understanding and awareness of cultural issues are ready to engage in CRSciA [27]. GenAI’s ability to generate personalized educational content demonstrates its potential to continue addressing the diverse needs of students through CRSciAs.

Culturally Responsive Assessments in Science Education

Over the years, K-12 science education has faced challenges in engaging all students in science learning. Research has constantly reported that students failed to use scientific knowledge in solving problems or designing solutions for real world problems. Moreover, traditional methods of teaching and assessment have historically led to the exclusion of diversity among successful science students [28,29]. Most assessments in K-12 science education narrow representations, leading to a growing achievements gap for minoritized students based on race, ethnicity, history, socio-economic status particularly in urban settings in countries like, the United States [30]. This call for CRSciA, which recognizes the diverse cultural perspectives and contributions in science, can aid in the achievement of underrepresented students in science.

Even though there has been an initial perception that science education is not suitable for culturally responsive assessments, this misconception has been cleared (Ladson-Billings, 2021). Recent studies have shown remarkable practices of CRSciAs, and even expanded to include Science, Technology, Engineering, Arts, and Mathematics subjects. This proves the value of culturally responsive assessments across all disciplines and educational levels (Ladson-Billings, 2021). Moreover, the developers of the Framework for K–12 Science Education [3] articulated a broader set of expectations to ensure that by the end of 12th grade, all students would possess sufficient knowledge of science to engage in their everyday lives, continue learning about science outside of school, and have the skills to enter careers of their choice. They emphasize on the phrase “all students” throughout this framework to provide equitable opportunities including assessment for all students to succeed in science [31].

Challenges of Implementing Culturally Responsive Assessments in Science Education

The implementation of CRSciAs faces several challenges with their effective adoption in science assessment. One major challenge arises from the limitations stemming from classroom teachers’ identities and biases, as well as their limited knowledge of diverse cultural contexts [32,33]. Teachers’ personal identities, including their race, ethnicity, and cultural background, can influence how they perceive and engage with CRSciAs [33]. For example, in science assessments, a teacher who lacks familiarity with the cultural experiences of their students may unintentionally introduce biases into the assessment process, either by favoring certain cultural narratives or by overlooking others.

Another challenge lies in the integration of CRSciA within the Framework for K-12 Science Education [6]. Though this is a reflective of a broader struggle within educational systems to effectively align CRSciAs with established curricular standards, it critically applies to science assessment [34]. Ladson-Billings [35] further critiques this misalignment by highlighting how many top-down initiatives to implement culturally responsive assessments often miss the mark; as states, districts, and professional organizations attempt to address cultural issues of assessments through various frameworks and guidelines, their efforts frequently fall short of the theory’s original intent. More to the challenges of CRSciAs is inadequate resources. The lack of institutional support also makes it difficult for science teachers to effectively incorporate cultural responsiveness into their assessments, and the necessary tools, guidance, and resources, educators are unable to fully leverage CRSciA practices [36].

These challenges of teacher identity, bias, and misalignment with curricular standards in science education underscore the urgent need for continuous professional development. However, studies indicate that there is currently lack or insufficient continuous professional development available for teacher regarding CRSciAs [37,38]. For instance studies Harris, et al. [6] survey conducted across 18 states in the US revealed that 86.36% of K-12 teachers view the integration of CRA with NGSS positively but suggested a more robust teacher training programs to enhance awareness and effective adoption of both NGSS and CRA in science classroom.

Generative AI and Culturally Responsive Assessment

Generative AI refers to advanced computational techniques that create new, meaningful content such as text, images, and audio from existing data [39]. These capabilities of GenAI can address significant challenges that teachers face in designing culturally responsive assessments, such as the time-intensive nature of creating materials that are both culturally relevant and pedagogically sound [12]. Furthermore, the interactive nature of GenAI-based assessments allows for real-time feedback and adaptation, providing teachers and students with immediate opportunities to learn and correct misunderstandings [40]. For instance, GPT-4’s ability to interpret and generate multimodal content, including visual data like graphs and chemical diagrams, enhances its utility in crafting assessments that are aligned with cultural contexts and could engage students in ways that traditional automatic text-based assessments cannot [41]. This multimodal approach, grounded in the theory of multimedia learning can help overcome the limitations of traditional AI, which has been largely text-bound, by incorporating a broader spectrum of human experience into the assessment process [12,42]

3. Study Approach

This study adopts Jaakkola [16] model type conceptual framework approach designed to create conceptual frameworks that reveal and predict relationships between emerging key concepts. As the intersection of CRSciA and GenAI is relatively new, this approach allows for the untied exploration of this phenomenon without the constraints of data-related limitations. The primary objective of this study is to construct a Framework that predicts how GenAI can be leveraged to effectively address CRSciA [16]. The development started with the identification of key and dominant cultural tenets in science assessment, followed by exploring the potential capabilities of GenAI in relation to the cultural tenets, integrating into a GenAI-CRSciA framework and finally demonstrating with practical examples of the framework while acknowledging the challenges involved.

3.1. Indigenous Language

Indigenous language plays a crucial role in culturally responsive assessment within K-12 education, particularly in science classrooms where students from diverse linguistic backgrounds are increasingly prevalent [27]. Wright and Domke [28] highlight the emphasis on language and literacy in the Next Generation Science Standards (NGSS) and the C3 Framework for Social Studies, noting the importance of supporting students’ disciplinary language development from the elementary grades. Kūkea Shultz and Englert [29] provide a compelling example of culturally and linguistically responsive assessment with the development of the Kaiapuni Assessment of Educational Outcomes (KĀʻEO) in Hawai‘i. Recognizing the cultural and linguistic diversity of the Hawaiian population, the KĀʻEO assessment was designed to be culturally valid, addressing the unique needs of students in the Hawaiian Language Immersion Program. In the context of linguistically responsive assessment, the application of GenAI in educational settings has shown significant potential in addressing the diverse linguistic needs of students. In a study by Latif, et al. [30], G-SciEdBERT was developed to address the limitations of the standard German BERT (G-BERT) model in scoring written science responses. By pre-training G-SciEdBERT on a substantial corpus of German-written science responses and fine-tuning it on specific assessment items, the researchers demonstrated a marked improvement in scoring accuracy, with a 10% increase in the quadratic weighted kappa compared to G-BERT. This finding highlights the importance of contextualized GenAI in science assessment, where specialized models like G-SciEdBERT can significantly contribute to more culturally and linguistically science assessments, particularly in non-English language contexts.

3.2. Religion

Religious beliefs are a critical factor to consider in the development of CRA in K-12 science education. Mantelas and Mavrikaki [31] highlight the intersection of religiosity and scientific concepts, such as evolution, presents unique challenges for educators in CRSciA. Their study demonstrates that students with strong religious convictions may struggle with certain scientific ideas, which can negatively impact their academic performance if assessments do not account for these beliefs. This means CRSciA should allow students to demonstrate their scientific understanding without forcing them to choose between their religious beliefs and academic success [32].

Barnes, et al. [33] further stress the importance of considering religious backgrounds in CRA, particularly for students of color who may rely on religion as a critical support system. Their research shows that strong religious beliefs can influence students’ acceptance of scientific theories like evolution, which in turn affects their academic success in science-related subjects. Culturally responsive assessments must therefore be designed to account for these factors, ensuring that they do not inadvertently disadvantage students whose religious beliefs differ from mainstream scientific views.

Owens, et al. [34] contribute to this discussion by advocating for a “pedagogy of difference” in teaching science, which could be extended to assessment practices in K-12 science education. This pedagogical approach encourages students to explore the relationship between their religious beliefs and scientific concepts, fostering an environment where multiple perspectives are acknowledged and valued. In the light of this, Sumarni, et al. [24] propose a holistic model for integrating religious and cultural knowledge into STEM education, which can serve as a foundation for CRA practices. Their RE-STEM model emphasizes the importance of bridging the gap between religion, culture, and science, suggesting that assessments should be designed to reflect this integration.

Although the intersection of GenAI and religious contexts remains underexplored, GenAI’s potential for engaging in nuanced discussions about religious concepts is promising (Cheong, 2020). For instance, GenAI models like ChatGPT have shown the ability to participate in theological dialogues, offering responses that respect religious traditions (Oxenberg, 2023). This capability allows for the development of assessments that honor students’ religious beliefs, fostering a more inclusive environment. Nonetheless, ethical considerations are paramount as Ashraf (2022) warns that GenAI applications must be carefully monitored to avoid infringing on religious freedoms and to prevent bias or disrespect in digital interactions.

3.3. Indigenous Knowledge

Indigenous knowledge plays a pivotal role in shaping culturally responsive assessments in K-12 science education, offering a means to create an equitable achievement [35]. Trumbull and Nelson-Barber [36] explore the challenges and opportunities in developing CRA for Indigenous students in the U.S., highlighting the limitations of standardized assessments that often disregard Indigenous knowledge systems. They argue that these assessments can be ineffective and even harmful, as they fail to engage Indigenous students or accurately measure their knowledge. This affirms Muhammad, et al. [37] assertion that that traditional assessments and curricula often overlook the historical and cultural contexts of Black and Brown children in the U.S., as highlighted by the National Assessment of Educational Progress (NAEP).

Therefore the concept of “culturally-valid assessment” was proposed by Trumbull and Nelson-Barber [36] to incorporate Indigenous ways of knowing and be responsive to the cultural and linguistic diversity of students. This approach is crucial for creating assessments that support the students’ academic success while also preserving and respecting their cultural identities [38]. Furthermore, Jin [39] systematically reviews educational programs aimed at supporting Indigenous students in science and STEM fields, revealing the positive impact of integrating Indigenous knowledge with Western scientific assessment. The review shows that culturally responsive assessment approaches in these programs lead to improved educational outcomes, as they allow Indigenous students to draw connections between their cultural heritage and the scientific concepts they are learning.

GenAI has the potential to challenge dominant Eurocentric narratives and promote the inclusion of Indigenous perspectives in K-12 science assessment (Washburn & McCutchen, 2024). For instance, GenAI tools can help ensure that science assessments honor and reflect Indigenous cultural identities. However, it is essential that GenAI-generated content is contextually accurate and respects the complexity of Indigenous cultures. Castro Nascimento and Pimentel (2023) emphasize the need for GenAI models to be trained on diverse cultural datasets to avoid perpetuating narrow perspectives. The deliberate integration of IK into GenAI models can significantly enhance the cultural relevance of science education.

3.4. Race and Ethnicity

CRSciA requires a thorough understanding of how race and ethnicity shape students’ learning experiences and outcomes. Atwater, et al. [40] emphasize that traditional science assessments often overlook the diverse cultural backgrounds of students, particularly those from African American, Latino, and Asian American communities. They therefore advocate for science assessments that are inclusive and reflective of the race and ethnicity within classrooms. Similarly, Wells [41] calls for strategic cross-sector collaboration between K-12 and higher education to address the sociocultural factors affecting diversity in education, reinforcing the need for assessments that are sensitive to the varied cultural contexts of students.

The importance of factoring race and ethnicity into science teaching and assessment is further highlighted by Riegle-Crumb, et al. [42], who found that inquiry-based learning is associated with more positive attitudes toward science among students from diverse racial and ethnic backgrounds. When assessments are designed to reflect the competencies of students’ cultural backgrounds, it allows them to demonstrate their understanding through exploration, critical thinking, and problem-solving in STEM [43]

Research by Choudhary [44] highlights the prevalence of racial bias in GenAI tools, including ChatGPT, underscoring the necessity of rigorous auditing to detect and mitigate these biases. In the context of CRA, AI tools must be designed to promote fairness and inclusivity, particularly in the assessment of students from diverse racial and ethnic backgrounds. Warr, et al. [45] provide evidence of racial bias affecting GenAI evaluations, demonstrating that racial descriptors can influence GenAI-generated scores. This underscores the importance of developing transparent and bias-tested AI tools that account for the unique cultural contexts of students. Ensuring that AI-driven assessments are equitable requires ongoing efforts to address and correct racial biases, providing a fair and accurate evaluation for all students.

3.5. Family and Community Engagement

While family and community might not be directly involved in the creation of assessment items, the role of family and community experts in validating cultural factors of assessments is crucial in the CRSciA framework. Family and community involvement are elements providing essential context and resources that directly impact students’ academic performance and engagement. Denton, et al. [46] emphasize the significance of community cultural wealth in STEM education, particularly in K-12 settings. They argue that an assets-based approach, which recognizes the diverse forms of capital, such as familial, linguistic, and social, that students from nondominant communities bring, is essential for developing assessments that truly reflect students’ backgrounds. In K-12 science assessments, this means creating assessment tools that account for the cultural and social capital that students acquire from their families and communities.

Gerde, et al. [47] provide insight into the specific ways that families contribute to science learning at the K-12 level. Their study shows that the resources and opportunities families provide, such as access to science-related books, toys, and community experiences like visits to parks and science centers, vary widely based on parents’ self-efficacy and beliefs about science. This is an indication that CRSciA at this level should not only measure what students know from formal education but also integrate the informal learning experiences that occur within their family and community contexts.

Soto-Lara and Simpkins [48] further elaborate on the role of culturally grounded family support in the science education of Mexican-descent adolescents, focusing on the K-12 educational context. Their findings reveal that parents provide support through both traditional means, such as encouraging their children, and nontraditional methods, like indigenous cultural practices and leveraging extended family networks. They also found out that despite the parents facing challenges such as limited English proficiency and lack of formal science knowledge, they remain actively involved in their children’s education. For K-12 assessments to be culturally responsive, they must recognize and incorporate these unique forms of family support.

Family and community involvement is a cornerstone of student success, particularly in science assessment. Garbacz, et al. [49] stress the importance of involving families and communities through ecological support. GenAI-driven tools can bridge the gap between school and home by providing culturally relevant resources and information, fostering a supportive learning environment. However, Saıd and Al-amadı [50] note challenges in engaging families, particularly in areas with limited digital literacy and technology access. GenAI has the potential to address these challenges by creating accessible communication channels between schools and families. The goal is to leverage GenAI not only for assessment but also to strengthen the connection between students’ educational experiences and their broader family and community contexts, enhancing overall educational support and involvement.

Figure 2. Generative AI Culturally Responsive Science Assessment Framework (GenAI-CRSciA).

Figure 2. Generative AI Culturally Responsive Science Assessment Framework (GenAI-CRSciA).

4. Discussion

The alignment of the CRSciA framework with the unique cultural contexts of Ghana, the USA, and China involves careful adaptation across several key tenets. In terms of language, the questions are presented in English for both Ghana and the USA, where it is the dominant language of instruction, while in China, the questions are delivered in Mandarin [54,55]. This ensures that students can engage fully with the content in their native or dominant instructional language. For instance, the efficiency of the GenAI-CRSciA framework becomes particularly evident when addressing the challenges faced by students in linguistically diverse environments, such as China and Ghana. In China, despite policies aimed at incorporating English as a medium of instruction, research shows that Chinese students often struggle in predominantly English-medium contexts [56]. The GenAI-CRSciA framework effectively mitigates these challenges by tailoring content to specific cultural and linguistic contexts, such as translating content into Chinese and aligning it with the students’ cultural backgrounds and linguistic needs [57]. Similarly, in Ghana, where the language-in-education policy mandates the use of mother tongue (L1) as the medium of instruction in lower primary grades (KG1 to Primary 3) and English (L2) from Primary four onwards, the GenAI-CRSciA framework demonstrates its cultural awareness by generating science assessments predominantly in English. This aligns with the reality of Ghana’s educational system, where English remains the dominant medium of instruction and assessment, particularly in science education [58,59].

Indigenous knowledge is also thoughtfully integrated, with traditional Ghanaian stories and proverbs highlighting the relationship between lions and antelopes, Native American perspectives shedding light on the spiritual and ecological significance of wolves and moose, and Chinese history and folklore contextualizing the interaction between Amur tigers and sika deer. For example, in Ghana, the framework integrates culturally significant symbols like the lion and deer, which are deeply embedded in local folklore and traditions. The ‘Aboakyire Festival’, celebrated by the people of Winneba, involves the symbolic hunting of a deer, an event that carries profound cultural and spiritual significance [60]. Similarly, among the Akans of Akyem Abuakwa, animal symbolism, including that of the lion and deer, plays a crucial role in cultural identity and educational practices, conveying lessons and wisdom that are essential to the community's heritage [61]. Likewise, in China, the framework demonstrates its effectiveness by incorporating ecologically significant animals such as the Amur tiger and sika deer into educational content, aligning the assessments with the local knowledge and values of communities in northeast China [62].

Further adaptations reflect the importance of race and ethnicity, where the symbolic significance of lions is considered within various Ghanaian ethnic groups, differing views on wolves and moose are acknowledged among Native American tribes and European settlers in the USA, and the cultural heritage of Amur tigers is recognized within both the Han Chinese majority and ethnic minorities [63]. Family engagement is encouraged across all regions, with students asked to gather insights and stories from their elders, enhancing the personal relevance of the assessment. Finally, religious beliefs are explored, with Ghanaian students reflecting on traditional animistic beliefs, American students considering how spiritual beliefs shape their views on wolves and moose, and Chinese students examining Taoist or Buddhist concepts of balance and harmony as they relate to the relationship between Amur tigers and sika deer [64,65]. This comprehensive approach ensures that the assessments are scientifically and deeply rooted in the cultural realities of the students.

5. Conclusions and Future Directions

Over the years, science assessment has faced significant equity challenges due to the complexity of culture, time constraints, and the skills required by teachers to generate assessments that are culturally sensitive to each student's dominant background. Despite efforts made within the NGSS, these challenges persist. This study, therefore, highlighted the core and critical tenets (including indigenous/dominant language, race/ethnicity, indigenous knowledge, religious beliefs, and family and community) that constitute CRSciA and aligned them with the capabilities of GenAI. These tenets were instrumental in developing the GenAI-CRSciA framework and subsequently GenAI-CRSciA Template-based prompt to generate assessment items from NGSS while comparing it with standard prompt strategies. The results showed that the GenAI-CRSciA approach outperformed standard prompt strategies when queried to generate CRA aligned with NGSS standards, proving its effectiveness in addressing the ongoing CRA challenges in science assessment.

Even though the GenAI-CRSciA framework presented assessment that covers range of the cultural backgrounds, it presents challenges such as the overgeneralization of students’ backgrounds, compromising of curricula core competencies that vary in differences, that is, overlooking subcultural variations within a country or region, such as differences between urban and rural students, or between different ethnic or linguistic groups within the same nation which can lead to bias and less effective CRA output. To address this, we recommend that teachers provide comprehensive and specific background information about their students in the template, ensuring that the generated content is truly reflective of the students’ unique cultural contexts. In larger class settings situations, teachers could implement comprehensive local data indexing, where detailed student background information is uploaded and retrievable for use in crafting more personalized assessments.

We strongly recommend educators to make conscious efforts to validate the GenAI-generated assessments with input from human experts, including community or family members, teachers, and even student, culturally responsive educators who are knowledgeable about the cultural nuances. Research indicates that current teacher education reforms have not adequately prepared teacher candidates to meet the needs of culturally diverse student populations, as specialized training is often lacking [66,67]. We propose continuous professional development to enhance teachers’ understanding of the cultural tenets and examples that may be generated by the template. This will enable them to better grasp the fundamentals of culture and assessments, ensuring they can effectively utilize the template to create equitable science assessments.