Assessing the Reliability of the Sexual Violence Questionnaire in Sport Among Spanish-Speaking Athletes

1. Introduction

The prevalence of sexual harassment and abuse in sport, has now been extensively documented and reported within academic research [1]. Young athletes in particular are at a great risk of being victims of sexual violence at this age, unacceptable behavior is often accepted as part of the sports culture [2]. In a study involving of 10000 athletes from 6 European countries, it was found that 35% and 20% of athletes had experienced non-contact sexual violence and contact violence respectively before the age of 18 [3]. There are several reasons that make it difficult to identify sexual violence behaviours, including the normalisation of physical contact, the intense relationships between coaching staff and athletes, and the diversity of leadership styles [4]. It is also important to note that sexual violence in sport can consist of overt sexual behavior, such as rape, or less obvious behavior, such as non-contact verbal sexual harassment [3]. Understanding the relationship between sport participation and sexual violence behaviors could be crucial for prevention, as recent studies confirm the existence of risk and protective factors associated with sexual violence in sport [5,6].

Interpersonal sexual violence in sport can be perpetrated by other athletes or by adults in a position of power in the sports context [3]. Coaches are amongst those who are in authoritative roles and are perceived as predominant figures by young athletes. Due to the complex and complicated relationships they have with their athletes female coaches can indeed be common perpetrators [7,8,9,10]. Coaches are in a position of trust and power over young athletes [11,12] and play a key role in the lives of young athletes. Indeed, they can provide young athletes with supportive relationships, based on trust and closeness, as well as. However, close relationships and the creation of strong emotional bonds can be a trigger for sexual violence to occur [13,14]. One aspect or risk factor of the coach-athlete relationship is the potential for role ambiguity where appropriate and inappropriate behaviors are not clearly defined, due to the time spent by both parties in and out of the sporting environment [15,16]. In addition, a coach’s role often involves physical contact in ways that can be instructional (e.g., teaching a start from a swimming pool) or non-educational (e.g., sharing space during a training camp or championship) [17,18]. The literature also addresses other characteristics of the coach-athlete relationship that can significantly increase the likelihood of sexual violence to occur between a coach and athlete, including athlete dependency, age differences, and the drive for success [19,20,21].

Due to the challenges and the grey area in determining what constitutes sexual harassment and abuse in sport, it is suggested that it is important to clearly determine what appropriate and inappropriate coaching behaviours should look like or take place within the coach-athlete relationship [22]. One of the first retrospective questionnaires on sexual violence in sport, called the Sexual Violence Questionnaire in Sport, was designed and developed to identify different types of coach behaviours susceptible to leading to sexual violence towards athletes [21]. Specifically, the questionnaire was first used to measure student athletes’ perceptions and experiences of sexual harassment in sport. Although subsequent questionnaires on sexual violence in sport have been developed, the inclusion of a section specifically addressing perceptions of behaviors and the coach-athlete relationship enables us to explore the nuanced areas of violence in sports. Consequently, understanding these different perceptions of sexual harassment and abuse is crucial for identifying gray areas where athletes’ perspectives are essential to consider [23].

The cultural factor also plays an important role in how certain athletes perceive and process sexual violence [24]. For example, existent studies confirm these cultural variations, concluding that Israeli students considered more behaviors as inappropriate compared to American students and that Indian athletes classified more actions as sexual violence compared to European and North American students, while the perceptions of the Danish athletes were similar to those of the North American students [20].

The suitability of the questionnaire of sexual violence [21] was also considered after introducing small modifications, translated it into Catalan and Spanish [25]. This was done after analyzing the content in a previous pilot study with participants sharing the same characteristics. However, given the cultural differences identified in other studies and the fact that the questionnaire was initially developed in English and adapted to this cultural context, and considering that no reliability analysis was conducted for the Spanish version, this study aimed to analyze the internal consistency and reliability of the questionnaire on sexual harassment in sports, originally designed and developed in English [21] and later adapted to Spanish/Catalan [25].

2. Materials and Methods

2.1. Participants

A total of 146 students from the Bachelor of Physical Activity and Sport Sciences (CAFYD) at the University of the Basque Country (UPV/EHU) participated in this study [female: n = 52 (35.6%); 20.3 ± 1.9 years; male: n = 94 (64.4%); 21.2 ± 2.3 years]. A convenience sample was used, driven by the crucial need of raising awareness of sexual harassment and abuse in sport among these student-athletes, as they are and will continue to be responsible for the sports programs, clubs, and federations of the Autonomous Community. Before administering the questionnaire, all participants were informed of the objectives of the study, as well as the research procedure, and the possible risks. Their participation was voluntary, and they signed an informed consent form. The study was approved by the Ethics Committee for Human Research of the University of the Basque Country (M10/2022/110).

2.2. Procedure

In order to analyze the reliability of the Sexual Violence Questionnaire in Sport, a cross-sectional study design was carried out. The questionnaires were administered in person, during the month of April 2022. Responses were collected 2 weeks apart between the first (Test) and second (re-Test) administration. To maintain anonymity while also being able to match the questionnaires from Test with the re-Test ones a coding system was used. Participants were asked to provide the first letter of their name and the town they lived in and four digits of their phone number.

2.2. Instrument

The Sexual Violence Questionnaire in Sport previously validated and used in other studies [21,25] seeking to gain the study participants’ perceptions and experiences of sexual harassment in sport. In this study we used the Spanish translation of the questionnaire [25] although the translation into Basque was also carried out. Similarly to this study, the current study used a sample of students studying a sport related degree to ensure the comprehensibility and appropriateness of the items. The pilot test confirmed the need to reformulate some terms in the Basque language in order to improve comprehension (e.g., “pinches” or “diminutives”).

The questionnaire was divided into three sections. In the first section, perceptions of sexual harassment were collected in relation to 24 items about the coach behaviors. The unit of measurement for the perceptions was a scale: 0 = I’m not sure if the behavior is sexual violence, 1 = the behavior doesn’t constitute sexual violence at all, 2 = maybe it is, 3 = the behavior is almost certainly constitutes sexual violence, and 4 = the behavior is almost certainly sexual violence. The second section collected respondents’ experiences of the same 24 behaviors. Again, a different scale was also used: 0 = I’m not sure, 1 = it has never happened to me, 2 = yes, sometimes, 3 = yes, often and 4 = not to me, but to a colleague, yes. Both perceptions and experiences are grouped into different categories: behaviors related to sports instruction, behaviors not related to sports instruction, but contextually dependent on it, sexist behaviors, and physical contacts and verbal approaches. Finally, the third section was added to collect both independent variables and demographic data such as age, university degree, sport practiced, number of hours of sport per week and level of competition, among others.

2.2. Statistical Analysis

The results are presented as frequencies and percentages. The Cronbach’s alpha test was used to analyze internal consistency, while the Wilcoxon statistic was used to determine whether there was a significant difference between the test and the re-test in each item of the perceptions/experiences. On the other hand, the Cohen’s Kappa analysis was used to analyze the concordance of the responses in each item. In this study, the interpretation of the scores is based on the classification system developed by [26]: scores of 0.09 indicate very poor agreement, scores of 0.01 - 0.20 indicate poor agreement, scores of 0.21- 0.40 indicate fair agreement, values of 0.41 - 0.60 moderate agreement, scores 0.61- 0.80 indicated substantial agreement, and scores 0.81 - 1.00 indicate almost perfect agreement. The analysis was performed using the Statistical Package for Social Sciences (SPSS Inc., version 26.0, Chicago, IL, USA). Significance was set at p < 0.05.

3. Results

3.1. Internal Consistency of the Sexual Violence Questionnaire in Sport

The internal consistency of the Sexual Violence Questionnaire in Sport was α = 0.891 for perceptions and α = 0.813 for experiences for all participants (see Table 1). In the case of females, the value of Cronbach’s alpha value of perceptions was slightly lower than that of males (α = 0.872 vs 0.891, for females and males, respectively), while the value of experiences was slightly higher (α = 0.839 vs 0.791, for females and males, respectively). It can be noted that the internal consistency of the questionnaire was good in all cases.

3.2. Perceptions and Experiences of All Participants

For perceptions, significant differences between test and re-Test were found for 10 of the 24 items (see Table 2). Similarly, in the case of experiences, significant differences between test and re-Test were in 2 of the 24 items (see Table 3). However, the Kappa index shows that agreement was significant for all items in the perceptions (range = 0.16 – 0.57, from mild to moderate agreement; p < 0.05). In the analysis, all items showed significant agreements with slightly higher values than in the perceptions (range = 0.37 – 0.61, from fair to substantial agreement; p < 0.05).

3.3. Perceptions and Experiences in Female Athletes

In contrast to the results obtained for the whole sample, in the case of females, significant differences between the Test and re-Test were only found in 4 out the 24 items in relation to perceptions (see Table 4). In the case of experiences, significant differences between the Test and re-Test were found only in item 11 (see Table 5). For both perceptions and experiences, Cohen’s Kappa indicated that agreement was significant for all items, except for item 24. The rest of the agreement values range from slight to substantial (range = 0.06 – 0.67; p < 0.05) for perceptions and from fair to substantial (range = 0.28 – 0.79; p < 0.05) for experiences.

3.4. Perceptions and Experiences in Male Athletes

For males, significant differences were found in 7 out of 24 items, a higher number than compared to females (see Table 6). However, as with females, there was only one significant difference between the test and the re-Test, but in this case, on item 9 (see Table 7). For both perceptions and experiences, Cohen’s Kappa indicated that agreement was significant for all items, although the values of agreement ranged from slight to substantial (range = 0.18 – 0.65; p< 0.05) for perceptions and from fair to substantial (range = 0.33 – 0.65; p < 0.05) for experiences.

4. Discussion

This study evaluated the reliability of the Sexual Violence Questionnaire in Sport among Spanish-speaking athletes. Assessing the reliability of this questionnaire is essential to ensure its applicability in other cultural contexts, specifically in Spanish context. The reliability of the Sexual Violence Questionnaire in Sport was analyzed in previous studies (e.g., 21,27,28) showing internal consistency values above 0.65 and below 0.90. These results are in line with those obtained in the present study, where the internal consistency of the questionnaire in Spanish and Basque was slightly higher in both languages (α > 0.791 for both perceptions and experiences in girls and boys). Regarding the reproducibility of the questionnaire, although some significant differences were observed between the test and the re-Test, (10 items in perceptions and 2 items in experiences) a significant agreement was observed with a slight to moderate agreement in perceptions and slightly higher from a fair to substantial agreement in experiences.

Sexual violence in the context of sport can be perpetrated or experienced by both male or female, regardless of the type of sport [3]. Nevertheless, it can be hypothesized that the perceptions and experiences of men and women are different, which may affect the reliability of the questionnaire. Regarding the internal consistency of the questionnaire, the results were higher than α > 0.791 for both perceptions and experiences in the whole sample, as well as for the female and male gender, indicating good values of internal consistency. Furthermore, the questionnaire can be reliable for both girls and boys, with very small differences between the two genders (Perceptions: α=0.872 vs 0.891; Experiences: α = 0.839 vs 0.791, for girls and boys respectively). These values were higher than those previously obtained in the first validation study α > 0.65 [21] and similar to those obtained in another study [28], which recorded values between α=0.77 and 0.89 for the different dimensions of the questionnaire in the perceptions section. These values show similarities with the reference literature in our context; however, it is necessary to carry out this analysis when using the instrument in different or new contexts.

Following established guidelines [29], which indicate that the test-re-Test is one of the most rigorous ways of measuring the reliability of the instrument over time, a comparative and concordance analysis was chosen. In the analysis with all the participants, significant differences were observed between the Test and the re-Test were observed for 10 of the 24 items, but for experiences on the other hand, the number of items showing significant differences was only two. This trend was also observed for the female gender (differences in 4 items for perceptions and 1 item for experiences) and for the male gender (differences in 7 items for perceptions and 1 item for experiences) where more differences were observed for perceptions than for experiences. The analysis shows that the instrument seems to be less reliable for perceptions. It is difficult to compare the data obtained because no studies were found that carried out a reproducibility analysis. However, in this study, the differences observed, especially in perceptions, may be due to obtaining fewer than five data points in an area, which could affect the results [30]. Additionally, perceptions do not have as much stability over time as experiences can demonstrate. This is because, as indicated different authors, [1,31] perceptions of behaviors related to sexual violence in sport may change over time depending on the different sporting situations and levels of sport and how this change; even athletes may be more accepting of violence in more serious sporting situations, while experiences remain constant over time. However, the instrument has been shown to be reliable for experiments, although further studies are needed to analyse the stability of the instrument over time.

The differences in several items of the reproducibility analysis differ from those observed in the reliability analysis, where the Kappa index showed significant agreement for all items in perceptions and experiences, although the range of agreement was from slight to substantial agreement. In the case of girls, there is no significant agreement on item 24, either in perceptions or experiences (it proposes sexual relations in exchange for privileges). As explained earlier, the fact that there is very little data in some responses may have affected the results. However, for the rest of the items, the agreement was significant with slight and substantial agreement in perceptions and experiences. As with the female gender and in the total sample, the reliability of the questionnaire showed significant agreement for all items, with values ranging from mild to substantial for both perceptions and experiences. Despite the differences in the comparative analysis of the Tes-re-Test, significant concordance relationships were also observed, albeit with low values in most cases. Further research is therefore needed in this area.

Reliability studies tend to assume that if the samples used have an equal number of boys and girls, the overall result when comparing the instruments could be valid for both genders, so several questionnaires have been validated with an exclusively female sample (e.g., 32), however , this is somewhat questionable, as there are really few studies that have attempted to evaluate the reliability of questionnaires for both genders [34]. In this study, the internal consistency values for men are similar to those for women with α = 0.891 for perceptions and relatively lower for experiences, with a value of α = 0.791. As in the case of women and in the sample, there were again significant differences in 8 items (7 perceptions + 1 experience. With regard to the reliability of the instrument, Kappa shows significant agreement for all items, with values ranging from mild to substantial (0.18– 0.65; p < 0.05) for perceptions and from fair to substantial (0.33 – 0.65; p < 0.05) for experiences. These values determine the reliability and temporal stability of the instrument, although they are relatively lower than those obtained with the instrument validated to measure the harassment in team sports [35].

This study is not without its limitations: 1) the number of participants in the study, 2) the distribution of the sample according to gender (92 vs. 54), 3) although the Sexual Violence Questionnaire in Sport was anonymized, it was given in person, so it is possible that some participants gave socially desirable or biased answers, and this they could have influenced the results, especially in terms of perceptions.

5. Conclusions

This study assessed the reliability of the Sexual Violence in Sport Questionnaire, showing a solid internal consistency across the sample and in gender segmentations. Gender-segmented analysis provides a deeper understanding of perceptions and experiences of sexual harassment in sport. Although half of the perceptions items do not appear to be stable over time, the experiences are stable over time, both in the sample as a whole and when differentiated according to gender. The Cohen’s Kappa analysis suggests concordance between responses at different times, which reinforces the validity of the Sexual Violence Questionnaire in Sport as a measurement tool. Despite these limitations, the results support the usefulness and reliability of the Sexual Violence Questionnaire in Sport and highlight the need for further research to analyze the reliability of the instrument on the basis of different population types.