Development and Validation of an Assessment Tool for Evaluating Cross-sectoral Collaboration in Developing a Child-Friendly City in Makassar

1. Introduction

On September 5, 1990, Indonesia ratified the Convention on the Rights of the Child, demonstrating its commitment to guaranteeing a brighter future for every child. Since then, significant progress has been made: more children go to school now than in the past, children have become actively involved in making decisions concerning their lives, and important laws protecting children have been drafted. However, there are still barriers, particularly in some disadvantaged regencies and cities, since the results achieved are not equal (1).

In accordance with human dignity and status, child protection guarantees children’s rights to a healthy life, development, and participation. The goal of this protection is to guarantee that children’s rights are upheld, particularly with regard to their health and education. In this context, it also includes a defense against sexual assault (2). Moreover, there is no disputing that children are legal subjects with a right to protection from their infancy, in terms of age, physical, and life experience. Thus, children require protection because of their conditions, as they are considered fragile.

The Indonesian government implemented a number of public policies in 2011, one of which is the child-friendly city (CFC) policy. The CFC is a regency or city that has integrated the resources and commitments of the public and private sectors to establish a development system based on children’s rights. In order to guarantee children’s protection and rights fulfillment, this system is meticulously and sustainably planned in the form of policies, programs, and activities (3). According to the Indonesian government, the CFC project by UNICEF is essential to the development of the country. This is due to the fact that children represent approximately one-third of Indonesia’s population, which is the fourth-most populous nation in the world (4). The UNICEF plan was subsequently adopted and implemented in Indonesia under the Regulation of the Minister of Women’s Empowerment and Child Protection Number 11/2011 on policies for the development of child-friendly regencies/cities. In addition, the ministerial regulation served as a foundation upon which the regional government based the establishment of child protection.

Recently, there have been 349 regencies and cities in Indonesia that have been designated as CFCs. Moreover, in 2019, 247 regencies/cities and 15 provinces in Indonesia received CFC awards from the Ministry of Women’s Empowerment and Child Protection. In particular, South Sulawesi Province was awarded the 2021 CFC award for 15 regencies/cities in the Pratama category and 2 regencies/cities in the Nindya category. Regencies/cities that were recognized as CFCs were evaluated using 25 sub-tension indicators in addition to six institutional indicators. These indicators fell into five categories that are related to children’s rights: civil rights and freedoms; alternative family environments and caregiving; health and basic welfare; education, leisure activities, and cultural events; and special protection. Additionally, there are five categories in which regional CFC implementation is appreciated: CFC, Main, Nindya, Madya, and Pratama (3).

The CFC program addresses some recent health concerns, including the increasing occurrences of malnutrition, child-friendly, suboptimal medical facilities, and unhygienic or inadequate living conditions for children in particular neighborhoods and areas (1). Furthermore, the WHO reported that the majority of childhood illnesses are strongly linked to the settings where children live (at home), learn (at school), and play (in the community). Moreover, there is also a risk of violence. The number of child rights violations in Makassar increased from 2016 to 2019. This has a negative impact on mental illnesses on the level of physical and psychological factors, which affects children’s growth and development significantly (5).

In addition, research conducted in Makassar by Junaedi (6) revealed that despite having adequate human resources at their disposal to implement the CFC program policy, their performances are nevertheless regarded as being of subpar quality due to their lack of knowledge and comprehension regarding child protection and CFC integration. It was additionally discovered that there remained a lack of perception and comprehension of the legal system and that children’s cities are not widely recognized by the general public as a result of unequal socialization and the elderly’s ignorance of the government’s involvement in this program or the program itself.

The aforementioned difficulties highlight that the issue of cross-sectoral collaboration, which is a method for resolving issues within the CFC program, is inextricably linked to the CFC program. The cross-sectoral collaboration in Makassar is explicitly mentioned in Makassar Mayoral Decree Number 2 of 2017 concerning CFC and Makassar Mayor Regulation Number 967/472.3.05/2019 on the establishment of a CFC task group in Makassar. Additionally, creating an urban environment that promotes children’s growth, development, and well-being is a shared objective of the cross-sectoral collaboration process that focuses on establishing a children’s city, which entails a number of steps and actions involving different stakeholders (7). All stakeholders participating in the cross-sectoral collaboration process to establish a children’s city must have a strong commitment, effective communication, and good coordination. Therefore, cities may improve as environments for children to grow up and develop happily and healthfully if there is good collaboration.

In order to determine the degree to which cross-sectoral collaboration efforts have contributed to the CFC program’s success, it is regarded as essential to comprehend the cross-sectoral strategic role. Moreover, there are currently no instruments or procedures available to evaluate the effectiveness of cross-sectoral collaboration in the current CFC program. Therefore, the significance of this study lies in its summary and analysis of the issues experienced by the development of the CFC, particularly in Makassar. In addition, this study developed indicators for evaluating the cross-sectoral role in CFC program management; thereby, it was expected that the indicators developed would be useful for periodic evaluations of the growth of cities and regencies, particularly in South Sulawesi Province.

This indicator’s primary goal is to assist local governments in measuring and valuing their efforts to fulfill children’s rights by establishing child-friendly regencies and cities. Additionally, it was intended to serve as a guide for the national government, provincial government, and regency/municipal governments to plan, implement, monitor, and assess development policies, programs, and activities that are aimed at achieving the establishment of CFC and fulfilling children’s rights (8).

2. Materials and Methods

2.1. Study Design and Data Collection

This study used a cross-sectional design. Moreover, the research team developed a paper-based questionnaire that was employed to collect data. Afterwards, some enumerators who have been trained in gathering data from students of the Faculty of Public Health at Hasanuddin University conducted the data collection. Additionally, direct data collection was done by contacting respondents at their workplace or respective agencies. To ensure data completeness, after the data were collected, data cleaning was performed first. Furthermore, the respondent was contacted over the phone if there was a missing answer’. In addition, it took about 15–20 minutes to collect data on one respondent.

2.2. Participant and Setting

This study was conducted in Makassar city from September 2023 to October 2023. Makassar is one of the metropolitan cities in Indonesia and the capital city of south Sulawesi province, with approximately 1.4 million living in the city making Makassar as the fifth largest city in the country. The sample consisted of 63 representatives (1-3 persons) of the institutions involved in the development of a child-friendly city in Makassar City in accordance with Mayoral Decree Number 267/472.3.05/2019. These representatives included the Commission II of the DPRD in the field of economic development and finance, the Head of Health Service, the Head of Social Service, the Head of the General Work Service, the Head of the Department of Education and Culture, the Director of the Food Security Service, the Leader of the Environmental Agency, the Head of Statistics of Makassar, the Head the Fisheries and Marine Service, all sub-district (camat) heads in Makassar (15 sub-districts), Bank Sulselbar Makassar Region, the Director of Daya Hospital, and the Head of the Family Empowerment and Welfare Mobilization Team (TP PKK), the Section Head of the Children’s Institution BPPPA and Women’s Organization (Dharma Wanita). In addition, the enumerator conducted interviews with each respondent and requested that they fill out a questionnaire.

2.3. Sampling and Sample Size

The purposive sampling technique was employed in this study. The most knowledgeable and active involvement of the respondents in cross-sectoral collaboration for the development of a child-friendly city (CFC), based on indicators of health and well-being in Makassar City, served as the criteria for choosing the sample. Moreover, the criteria resulted in the selection of 63 individuals who represented each agency to be used as samples for this study.

2.4. Tool Development

The indicator was developed into a tool or instrument after a preliminary qualitative procedure. Moreover, in-depth interviews and paperwork were employed to gather the data in order to obtain indicators of assessment and measurement of cross-sectoral collaboration on the development of the child-friendly city (CFC) in Makassar. The informants included the Head of the Child Institution in Women’s Empowerment and Child Protection, the Head of the Health Service, the Head of Education and Culture Service, Commission II of the DPRD Economic and Development and Finance, and other stakeholders involved in the development of CFC based on indicators of health and well-being, as well as experts who have expertise in their respective fields, i.e., 1 expert in the field of the child-friendly city, 1 specialist in the field of health planning and evaluation, and 1 expert on health policy.

The process of data reduction and categorization involved selecting, simplifying, and transforming raw data from interviews. Afterwards, the data was categorized into thematic categories based on the statements of each informant. Following the interview process, indicators of CFC collaboration were identified, and expert discussions were conducted to improve and refocus the indicators that informants had developed so that the measurement could meet the required standards.

Those processes resulted in 17 indicators of cross-sectoral collaboration in the development of the CFC in Makassar. These indicators were categorized into five dimensions: governance, administration, autonomy, mutuality, and norms. The first dimension of governance has three indicators (7 items), namely membership, institutional rules, and structural change. The second dimension of administration has six indicators (14 items), namely channels of communication, clarity of roles and responsibilities, coordination of policymakers and implementers, interagency coordination in the implementation of programs, the availability of data as a medium of information, and evaluation and monitoring of programs. The third dimension of autonomy has two indicators (9 items), namely programs and budgets. The fourth dimension of mutuality has three indicators (6 items), namely the exchange of resources, reward, and training of human resources. The fifth dimension of the norms has three indicators (5 items), namely trust, commitment, and reward. In addition, these indicators (41 items) were then compiled and developed as a tool that was employed to measure the cross-sectoral role in the development of the CFC program. List of the items and dimensions is shown on Table 1.

2.5. Data Analysis

Confirmatory factor analysis (CFA) was performed using Analysis of Moment Structures (AMOS) version 8.5. Factor analysis is a method of analysis to determine if there are one or more latent variables (not directly observable) that cause a set of variables to correlate with each other. Subsequently, this method of factor analysis as a “statistical model” is known as “confirmatory factor analysis” (CFA) (9). Moreover, it is possible to test the validity of a psychological measurement using CFA due to its “confirmatory” nature. In addition, it can be tested using CFA to the extent to which the whole item of the test actually measures information about only one thing, that is, what is being measured.

2.6. Ethical Consideration

The study was approved by the Health Research Ethics Committee (KEPK) of the Faculty of Public Health at Hasanuddin University (Number: 4912/UN4.14.1/TP.01.02/2023). Additionally, individuals who were willing to participate in this study must sign an informed consent form prior to the interview being conducted, and they were requested to fill out the questionnaire.

3. Results

A total of 63 participants were enrolled in the research study. The participants were mostly female (68.3%), with age range was 40-49 years old (38.1%), only a small percentage of participants over the age of 60. In terms of educational background, the vast majority of participants (58.8%) have an undergraduate/bachelor degree. Information regarding participant characteristics is shown in Table 2.

Before performing or obtaining a fit model for CFA analysis, several items were removed since they were determined not to match the model. As shown in Figure 1, for the governance dimension, there are 5 items remaining (2 items removed); for the administration dimension, there are 10 items remaining (4 items removed); and for the norms dimension, there are 2 items remaining (3 items removed). However, for the autonomy and mutuality dimensions, the number of items remained fixed (no item was removed). Thus, the total number of items that fell into the model is 32.

3.1. CFA Analysis Result

One of the main objectives of the CFA is to test the degree of validity of the latent variable/construction of the proposed measurement model theory. The convergent validity test itself is a test to determine whether the variables of the indicator used are really significant in reflecting structural or latency variables. The following are some measurements of the convergent validity test:

3.1.1. Standardized Loading Factor (SLF)

A good convergent validity property is indicated by a high SLF value. According to Hair, JF. (10), a value of SLF ≥ 0.5 indicates that a good convergent validity property has been achieved. In Table 3 we present all SLF values > 0.5, implying that the convergent validity properties based on a good SLF measurement has been achieved.

3.1.2. Construct Reliability (CR) dan Average Variance Extracted (AVE)

Table 4 shows that the CR and AVE values for the four variables/dimensions—governance, administration, autonomy, and mutuality—are above 0.5 for CR or CR > 0.5 and above 0.7 for AVE or AVE > 0.7. This indicates that the four variables/dimensions have achieved convergent validity properties based on CR and good AVE sizes. However, the CR and AVE values for the norms dimension are below 0.5 for CR and below 0.7 for AVE, implying that the convergent validity properties based on the size of CR and the AVE properties were not achieved by the norms. Additionally, the CR value > 0.7 indicates good reliability.

4. Discussion

This study provides a test instrument that may be employed in evaluating cross-sectoral collaboration in the development of the CFC, as well as used as a benchmark to gauge the effectiveness and impact of the CFC’s cross-sectoral collaboration initiatives in Indonesia. Moreover, five dimensions were identified by this study as being useful for evaluating cross-sectoral collaboration in the development of CFC in Indonesia. These dimensions were then broken down into 17 indicators, resulting in 32 items in total.

4.1. Governance

The results of this study revealed that all items (5 items) in the governance dimension have an SLF value > 0.5, a CR value > 0.7, and an AVE value > 0.5, indicating that a good convergent validity property has been achieved or indicates good reliability. This result also implies that the five items used were really significant in reflecting the governance dimension.

From a conceptual standpoint, collaboration is the fundamental component of governance. The nature of governance suggests that public service delivery in the future will be determined by the collaboration between governmental and non-governmental organizations, with both playing a critical role (11,12).

In order to accomplish shared objectives that cannot be attained independently, a number of public, private, and civil society organizations collaborate as stakeholders under the framework of collaborative governance, which is defined as deliberative consensus and collective decision-making (13,14). Furthermore, collaboration and the formation of institutional and intersectoral governance can be initiated and governed through collaborative leadership arrangements, autonomously structured and networked administratively, or initiated and led externally (15).

4.2. Administration

The results of this study revealed that all items (10 items) in the administration dimension have a SLF value > 0.5, a CR value > 0.7, and an AVE value > 0.5, indicating that a good convergent validity property has been achieved or indicates good reliability. This result also implies that the ten items used were really significant in reflecting the administrative dimension.

In this regard, the administration took into consideration the agency’s communication channels, as evidenced by data derived from the coordination meeting’s outcome, discussions conducted about the development of the CFC, and the attendance of agencies other than the task force team at the meeting. Furthermore, it was discovered the significance of each sector’s tasks and responsibilities in carrying out the CFC program’s development in Makassar City. Additionally, a health program will affect how the institution works if there is effective communication (16).

4.3. Autonomy

The results of this study revealed that all items (9 items) in the autonomy dimension have a SLF value > 0.5, a CR value > 0.7, and an AVE value > 0.5, indicating that a good convergent validity property has been achieved or indicates good reliability. This result also implies that the nine items used were really significant in reflecting the autonomy dimension.

In this study, autonomy refers to the institution’s capacity to take advantage of its own resources in conjunction with those of its members who collaborate to promote the CFC. It was evident from how each institution or organization planned its activities, how it focused program aims, how it coordinated planning, and how institutions are involved in the evaluation program activities. Furthermore, indicators of independence include the provision of budgetary resources for the establishment of the CFC in each institution or agency within the South Sulawesi Province, as well as the alignment of the planning process with the current budget.

Despite the fact that strategic investment in administrative capacity is crucial in public management practice, organizations occasionally disregard the significance of establishing a policy agenda or program and allocating funds to ensure the program’s success (15,17). Then, if funds are available, it is preferable that the agency plan and estimate program-related activities according to the study of the requirements needed for an activity to support the development of CFC.

4.4. Mutuality

The results of this study revealed that all items (6 items) in the mutuality dimension have a SLF value > 0.5, a CR value > 0.7, and an AVE value > 0.5, indicating that a good convergent validity property has been achieved or indicates good reliability. This result also implies that the six items used were really significant in reflecting the mutuality dimension.

In this study, “mutuality” refers to the institutions involved and represents the benefits and mutual support that institutions that collaborate on the CFC program share. Moreover, collaboration between several institutions or organizations is necessary for the development of CFC in order to support one another and facilitate the implementation and accomplishment of goals. A single agency is unable to implement the CFC program by itself. Therefore, interagency collaborations are crucial (18).

An overview of this dimension could be discovered by assessing the indicators of resource exchange, the impact of other institutions’ operations on their counterparts, and interagency appreciation. According to in-depth interviews with informants, only a few institutions involved in the development of CFC in Makassar City were discovered to have carried out other sectors in the particular institution program with external benefits between each institution in one work program. This implies that, in spite of efforts made at mutual support across the institutions, interagency collaboration remains necessary to optimize it.

4.5. Norms

The results of this study revealed that all items (2 items) in the norms dimension have an SLF value > 0.5, indicating that a convergent validity property based on a good SLF has been achieved. However, the results also showed a CR value < 0.7 and an AVE value < 0,5, implying that the convergent validity properties based on good CRs and AVEs have not been reached by the norms dimension.

The norms dimension was evaluated using metrics for trust, dedication, and rewards granted to institutions that have made significant progress in the establishment of CFC in Makassar City. Additionally, the effectiveness of program convergence in a given area was determined by the ability, proactive role, and cooperative pattern developed among actors, individuals, and institutions, which were in line with their roles and authority (19). Furthermore, trust and incentives have an impact on the degree of an organization’s commitment. Moreover, there was a strong correlation between organizational commitment and the active role of the organization. In this instance, a high level of program management commitment from the organization will increase the effectiveness of convergence.

4.6. Study Limitation

One potential bias in this study is related to the screening process. As the child-friendly city collaboration has been going on for almost ten years, some of the respondents who are currently serving as officials have not had enough exposure to cross-sectoral collaboration in the child-friendly city (CFC) program. In addition, a successful collaborative process takes time; however, this study used a cross-sectional design, which took less time.

5. Conclusions

The objective of this study was to measure the effectiveness of CFC programs with cross-sectoral restrictions since there is currently no instrument available for conducting proper assessments. Moreover, the final model of the five-factor scale, which included 17 indicators and 32 items, was successfully developed. Additionally, the CFA results showed that the five-domain model corresponded with the data quite well (SLF >0.5), convergent validity was acceptable). It was expected that the results of this study would serve as a benchmark indicator, enabling the assessment process to gauge the significance with which a regency or city is developing with cross-sectoral restriction measures. It was also suggested to test the tool’s use on a larger population or subject in order to further assess its reliability. In addition, other researchers may create a far more extensive tool that still utilizes the ones developed in this study.