Machine Learning Model for Improvement of Project Team Members Skillset through Software Governance

1. Introduction

According to UNESCO’s Artificial Intelligence (AI) and education guidance for policy makers “Education is UNESCO’s top priority because it is a basic human right and the foundation on which to build peace and drive sustainable development.” (UNESCO, 2021) Education is the primary source for continuous skillset development.

Education and skillset development are applicable at all levels of human development, and this study focuses on the machine learning model for skillset development of project team members in a software organization. According to a recent survey (Sheoraj & Sungkur, 2022), 72.7% of the participants assessed an artificial intelligence (AI)-based governance framework as effective. Most organizations implement AI in their workplaces for seamless task transitions.

As per Gartner “IT governance (ITG) is defined as the processes that ensure the effective and efficient use of IT in enabling an organization to achieve its goals”. Irrespective of size and complexity, every project demands a clear approach to the skillset development of team members. This is vital for successfully implementing business strategies as well as regulatory and mandatory requirements. Most IT companies have had a great impact due to “The Great Resignation,” and around 48 million quit their jobs in 2020. According to McKinsey, lack of career development and advancement is the major reason people quit their jobs (Dowling et al., 2022). AI-based skillset development helps to align team members with the objectives of the project and organization. The learning and development of team members are applicable across the domains of IT Governance, such as value delivery, strategic alignment, performance management, resource management, and risk management.

Employees will be able to support the organization in today's global market with the right opportunities for skill development and employee performance evaluation strategies. [4]. The proposed approach recommends a Learning Management System (LMS) to communicate and follow up with the team members. The LMS provides a well-defined structure to implement the overall skillset development plan according to the project requirements. These tools fetch measurable results for the project team members. These customized plans can help align closely with project requirements. An increased project success rate helps to increase the return on investment and stakeholder confidence in the project's completion.

Precise learning and development plans can be derived based on successful completion of day-to-day activities. In this article a dataset with machine learning techniques was used to determine the upskilling and reskilling status of the team members. Only recurring tracking process can align governance policies with the learning/development of project team members. This is essential for the successful completion of IT projects.

2. Problem Statement

Finding an ideal team with required skills and knowledge is a major challenge for project managers. The likelihood of failure increases when the team members are unskilled. This places organizations in a risky situation.

• The primary focus of many organizations is on their strategic objectives. These strategic objectives are mostly aligned with short-term or long-term business opportunities (Cabral & Dhar, 2019)
• Skillset development of team members are not considered an explicit objective in most organizations.(Rodriguez & Walters, 2017)
• Virtual and hybrid workspaces in IT companies are key challenges to the skillset development of project team members.(Aroonsrimarakot et al., 2022)

The lack of standard educational policies in the IT governance process leads to inconsistencies in the skillset development of project team members. This study focusses on the software governance process and day to day project activities. This approach helps to clearly determine the upskilling and reskilling status of the team member. Provide customized development plan to improve the skillset of the team members and increase the overall success rate of the software projects.

3. Research Questions

Following are some of the key research questions that helped to define the problem statement

• What effect does software governance have on the skillset development of the team members?
• How machine learning model can be used to improve the skillset of the team members in the software governance process?
• How to determine the upskilling and reskilling status based on the day to day project activities?

4. Related Works

Numerous studies have focused on the software governance framework and skillset development. Software governance is one of the key factor to improve the success rate of the software projects. Continous optimization of software governance with logical predictive model can potentially improve the success rate of the software projects. This initial model proposes the potential list of organiztional software standards, governance policies, past and current project details that can be used to improve the success rate of software projects. (Ranesh & Samuel, 2019)

Follow up study on the IT governance framework proposed to use Artificail Neural Network and Balance Score card to improve the success rate of the software projects. Customized and confidential governance pattern are used across various software organizations hence there are very limited real time data across the industry. This study infered that ANN with Balance Score Card was effective in providing the optimal solution to increase the project success rates.(M Maria Antony Ranesh; S. Justin Samuel; R. Natchadalingam; P. Jeyanthi, 2022)

Subsequent analysis in software governance highlights the utilization of classification techniques into governance proces. Naïve Bayes algorithim has been used validate the aligment of governance policies and project details. Proposed to use governance assessment form to validate the project details through classification techniques (Antony-Ranesh & Samuel, 2022)

A recent systematic review of integrative learning in software development teams in 2022 highlighted the key drivers and future agenda items. This analysis demonstrated how a team's activities can be tracked to integrate learning. This study highlights the importance of team learning in understanding inter-team and intra-team settings”(Mehta et al., 2022).

Another study in 2022 regarding comparative effects in knowledge sharing showcases, Capability Maturity Model Integration (CMMI) based improvements in software development. Skillset development contributes significantly to software process improvement at all maturity levels. Hence every organization must focus on knowledge sharing to ensure consistent project success. (Chen & Lee, 2022)

Research paper on software task assignments inferred that the sustainability of development teams over the long term is enhanced by self-adaptive task assignments. Thus, it could potentially be counted upon projects and their managers when there is a potential knowledge loss as a consequence of developer turnover. (Etemadi et al., 2022). A study report in 2022, emphasized the focus of the entire company culture. This affects organizational skillset development. These findings highlight that implementing knowledge management strategies can encourage flexibility and innovation. (Scaliza et al., 2022)

Study in 2022 examined the essential aspects that promote knowledge exchange in the IT industry. According to this study, affiliation, association, and attitude are the only three determinants of knowledge-sharing behavior.(da Silva et al., 2022) A software organization model was developed in 2021. This research article on conflict dynamics and conflict management procedures found that team disputes were a major cause of software project failure. This can have a significant impact on the performance and software quality.(Nunkoo & Sungkur, 2021)

In 2021, a research paper on software governance analyzed the tactical alignment of software projects. This study emphasizes that an effective software governance tool can align with strategic objectives only through governance practices. Only effective governance practices can help align the strategic business objectives.(Maciá Pérez et al., 2021). In 2020, a study on maturity models determined the focus of software governance. Maturity models are useful tools for communicating a substantial amount of information. However, maturity model research and development tools are limited. These models should be implemented to encourage knowledge sharing across software organizations.(Jansen, 2020)

A 2019 case study discussed a framework for distributed software teams and risk-management approaches. In modern risk management practices, the most prominent risk encountered by team members is lack of communication. Hence, the risk-management process should be integrated into the overall framework.(Wan Husin et al., 2019). An alignment process model with three phases, exploring, constructing, and extending, was created as a result of a study conducted in 2018. This approach has helped increase sustainability across organizations. (Yeow et al., 2018)

In 2015, effectiveness of implementing COBIT5 in information security framework was analysed and concluded that “Implementing COBIT5 provides various advantages in the supply chain management system (SCMS) and enterprise resource planning (ERP)” Hence governance framework is very critical in resource planning(Wolden et al., 2015). An extensive literature review between (1993-2011) and 2015 regarding knowledge sharing in software development teams indicated the demand for a classification framework for knowledge sharing. They also established a classification framework for software teams from the perspective of organizational change. (Ghobadi, 2015)

2015 study that examined the impact of the Internet on business found that Internet use creates less-competitive industry structures. Hence, limited exposure to the Internet can help increase overall productivity(Wang & Zhang, 2015). Another 2014 in study regarding the internal control framework showcases a procedure to develop an internal control framework that can be applied within an enterprise resource planning (ERP) system. This study helps to ensure the overall compliance of ERP systems.(Chang et al., 2014)

In 2018, a study on IT infrastructure libraries highlighted a framework for IT Governance. They recommended using this strategy as a key component of enterprise governance. The management of IT resources, with an emphasis on prioritization and rationale, has also improved .(Gërvalla et al., 2018). Another 2013 study insisted that senior management support was important for success and was far more important than any other success factor.(Young & Poon, 2013). To enhance business processes and company performance, a 2010 method suggested IT-related competencies with IT governance activities. Customer services and company performance are associated with improvements at the internal process level.(Prasad et al., 2010)

Research on collaborative technologies and knowledge exchange was conducted in 2009. This demonstrated a procedure that included the implementation of a knowledge codification method and a customization strategy by the co-located and Globally Distributed Teams (GDT).(Gupta et al., 2009)

5. Methods

We have used multimethodological approach that includes both quantitative and qualitative data. The triangulation technique is a process for analysing research findings using bulk data. It serves three important purposes: improving validity, creating a clearer image of a study problem, and examining various approaches to problem-solving. This triangulation technique was used to analyse based on the status of the activities and to derive the process of skill development.

JIRA Dataset

JIRA is a proprietary issue tracking product developed by Atlassian that allows bug tracking and agile project management. JIRA is a standard system that supports software companies for issue tracking and project management. It is a leading provider with over 180,000 global customers for collaboration, development, and issue-tracking of software. The public JIRA repositories included in this dataset were retrieved using JIRA API.

The Materials and Methods should be described with sufficient details to allow others to replicate and build on the published results. Please note that the publication of your manuscript implicates that you must make all materials, data, computer code, and protocols associated with the publication available to readers. Please disclose at the submission stage any restrictions on the availability of materials or information. New methods and protocols should be described in detail while well-established methods can be briefly described and appropriately cited.

Research manuscripts reporting large datasets that are deposited in a publicly available database should specify where the data have been deposited and provide the relevant accession numbers. If the accession numbers have not yet been obtained at the time of submission, please state that they will be provided during review. They must be provided prior to publication.

Interventionary studies involving animals or humans, and other studies that require ethical approval, must list the authority that provided approval and the corresponding ethical approval code.

Figure 1. shows the overall skillset development process with JIRA dataset.

Figure 1. shows the overall skillset development process with JIRA dataset.

We propose a six-step approach with overall skillset development process using software governance policies.

Step 1: Define educational policies in Governance

Software governance is one of the most important tools businesses employ to ensure that software development projects are in line with their business objectives. Alignment of team educational policies within software governance can help improve the success of day-to-day activities. The strategic objectives of the organization help to track the skill development progress of team members. Skillset development of team members through standard upskilling and reskilling is the major pillar for improving the learning of team members.

• Upskilling is a training process of acquiring new skills.
• Reskilling is a training process that ensures that the current level of skill is retained for better contribution.

Table 1 provides 4 types of educational policies that can be validated in the Atlassian JIRA dataset. To validate the accuracy of skill development for project team members, upskilling and reskilling categories are defined. This is based on the issue resolution status and resolution time of bugs, defects, improvements, and new features in JIRA dataset. These governance policies are aligned with the skill development of team members. This is validated through effective pre-processing techniques as part of the subsequent steps in this approach.

Step 2: Pre-processing of JIRA Dataset

Table 2 provides a summary of the Atlassian datasets from JIRA extract. This table highlights the overall number of bugs, improvements, and new features of Atlassian JIRA extract. Table 3 provides the number of fixed bugs, improvements, and new features in the Atlassian JIRA extract.11504 out of 16286 records were in a fixed status; hence, approximately 71% of the total issues were fixed in 14 projects in the Atlassian dataset.

A pictorial flow chart of feature extraction from JIRA dataset is shown in Figure 2. This Figure 2, displays the overall data flow from the JIRA extraction. Feature codes are allocated based on the dataset. Overall records and percentage are calculated for the upskilling and reskilling categories. Status of Bug. Improvement and New Feature represents the average of the day to day status of the activities and very closely aligned with the project objectives. Hence mapping the status of the bug, improvement and new feature would help to determine the skill development of the team members.

Step 3: Identify upskilling and reskilling status

This step ensured that the educational policies defined by the software governance team were validated using the Atlassian dataset based on binary conditions. The upskilling and reskilling of team members are based on the “Resolution status” and “Resolution Duration” fields. Figure 3 is the visual representation of resolution status and resolution duration. Agreed SLA (Service Level Agreement) is one of the key parameters to benchmark the educational policies. The key features updated in Table 4 are used to validate the Atlassian JIRA dataset. Issue identification number and project id are used to avoid duplication at program or portfolio level

This workflow can be represented by the following formula

• Upskilling: If (IR) = “Fixed” OR” Completed” and Resolution Duration = “Agreed SLA”
• Reskilling: If (IR) ≠“Fixed” OR” Completed” OR Resolution Duration ≠“Agreed SLA”

The resolution duration can be customized and used according to the type of activities and strategic objectives of the organization. Equation 1 the procedure of the fixed or completed activities to determine the upskilling and reskilling categories.

Step 4: Accuracy of the upskilling and reskilling status

5.1. Governance Policy Comparison in JIRA – Atlassian Dataset

The following is an overall summary of the Atlassian dataset from the JIRA database: This helps to identify the overall quality and quantity of the Atlassian dataset to validate the upskilling and reskilling status. Figure 4 represents the total number records used for educational policy 1 (EP1) and edcational policy 2 (EP2)

Figure 5 displays the overall percentage of educational policy 1 (EP1) and educational policy 2 (EP2 ). This is based on the issue resolution status of the Atlassian dataset. Overall upskilling is required for 71% of project team members and reskilling is required for 29% of project team members.

EP3 and EP4 are based on the status of the issue resolution data in the Atlassian dataset. Figure 6 represents the upskilling and reskilling records based on EP3 and EP4 and Figure 7 Upskilling and reskilling percentage based on EP3 and EP4. Upskilling is required for 48% of project team members, and reskilling is required for 52% of project team members.

Figure 8 showcases the overall skilling % for educational policies EP1, EP2, EP3 and EP4

Step 4.2 Pre-processed Atlassian datasets from JIRA are loaded into WEKA.

WEKA is an open-source tool for data preprocessing and the implementation of Machine Learning algorithms. This can also help visualize datasets and develop machine learning techniques. Various classification algorithms, such as J48, Random Tree, Random Forest, Decision Table, Logistics, and Naïve Bayes, are used to determine the accuracy of the Atlassian dataset from the JIRA extract.

An algorithm is considered successful when its mapping and predictions are accurate. The outputs of these classifications were compared to determine an accurate technique for determining the upskilling and reskilling status of project team members.

The overall data workflow of this approach is represented in Figure 9

The following are the key features of the algorithm used for of the upskilling and reskilling status:

• J48 - To comprehensively and continually evaluate the data
• Random forest - a widely used algorithm for supervised machine learning. This is used to solve classification and regression issues. To produce the final output, the Random Forest algorithm combines the results of many (randomly produced) Decision Trees.
• Decision Tables - Software testing tool used to examine how the system responds to various input configurations.
• Logistics: This is used to calculate or predict the probability of binary (yes/no) occurrence.
• Naive Bayes: Supervised learning approach for classification issues based on Bayes’ theorem

Table 5 and Table 6 provides the values of the correctly and incorrectly classified instances of all the 4 educational policies. The accuracy of the techniques was determined based on the TP Rate, FP Rate, and precision.

• TP Rate: Provides the percentage of accurate forecasts in positive class predictions
• FP Rate: The Ratio of the number of negative events incorrectly classified as positive occurrences.
• Precision: The capacity of a classifier to avoid classifying anything that is negative as positive.
• Recall: Classifier's capacity to accurately identify all positive instances
• F-Measure: Metric for the precision of a model on a dataset. It is used to assess binary classification methods.

Table 7 and Table 8 provides the classification results of EP1, EP2, EP3 and EP4

Overall, the J48 algorithm provided the most precise results, with a maximum true-positive rate. The results of J48 help predict whether the upskilling and reskilling percentages are accurate in determining the learning and development of project team members.

Step 5: Determine upskilling and reskilling strategies

The upskilling and reskilling process also helps to identify new skill areas based on the strategic objectives of the organization .Figure 10 displays the list of potential skill development strategies that can be used for the upskilling and reskilling of the team members.

6. Conclusions

Software Governance is a framework that includes both standard policies and procedures. These standard policies and procedures are of paramount importance in upskilling and reskilling process. The methodology and approach defined in this study are proposed the upskilling and reskilling processes to align with the successful completion of a project’s activities. The accuracy of the skill development data was derived using various machine learning techniques, and it was concluded that J48 provided the maximum accuracy, followed by Random Forest and decision table algorithms.

Hence, it can be concluded that the J48 algorithm can be applied to multiple projects/programs to monitor and track the skill development process. It is one of the best machine learning algorithms for categorizing and continuously examining data.

J48 is based on a top-down strategy and helps understand patterns in the majority of isolated projects because of project-specific challenges in requirements and risk. Consistent communication with the learning and development plan for project team members would help retain team members in the organization. This would help the organization plan long-term projects and increase stakeholders’ satisfaction. This study helped to aling team members and develop a customized development plan based on the status of the day to day activities. Accuracy of this development plan is compared with various machine learning models and proved with J48 algorithm