Improved E-learning Based Recommender Systems: A Survey

1. INTRODUCTION

A recommender system is used to filter information according to the user’s needs. They play a key role in improving the user’s experience and are being used in different fields like e-commerce and entertainment. Recommender systems are generally used to help the user identify relevant information from massive data chunks hence helping the user [1] to make a better decision. One of the very early recommender systems presented by Goldberg et al in the early nineties used collaborative filtering to recommend information to the user. The model presented in the earlier systems is still being used in some hybrid systems for generating recommendations in various fields. The current interest of research in recommender systems lies in fields like e-commerce [2], music [3], movie recommendation [4] specifically Netflix [5]. Education is also an emerging area for recommendations. Most recommendation systems from the past focused on higher education [6] because universities offer several electives and students need counseling to choose appropriate courses. Some of the existing work also considered kindergarten to 12sth grade education to recommend career path for young students [7]. Online learning is another subarea where recommender systems can play a key role in improving the learning experience of the students [8]. The increasing trend of e-learning has introduced the concept of Massive Open Online Courses (MOOCs). MOOCs are a massive collection of free resources available online. COronaVIrus Disease (COVID-19) pandemic, that hit most of the world in the year 2020 is still forcing educational institutions to shut down to contain the Severe Acute Respiratory Syndrome COronaVirus 2 (SARS-COV-2) [9]. The sudden shift from traditional classroom to online mode has also dramatically increased the number of new enrollments in MOOCs [10]. More students turning towards online learning has increased the demand of recommender system for MOOCs.

Figure 1. Hybrid methodology to improve learning path recommendation [11].

Figure 1. Hybrid methodology to improve learning path recommendation [11].

Recommendation systems for e-learning are different from traditional recommenders because, in traditional education, all courses have specified length, difficulty level and pre-requisites. Furthermore, the dropout rate of students is also low in traditional education as compare to e-leaning platforms [12]. Grade prediction and prior knowledge can play a role in generating recommendations in e-learning but other factors like ratings of a course, rating of instructor, relevancy of course, difficulty level are also important. Massive online courses (MOOCs) are playing a great role in revolutionizing e-learning experience but it comes with some limitations like lack of course sequencing and limited personalization [13].

The current trends of a study conducted in this area are more focused on hybrid techniques rather than using just one methodology. The hybrid techniques help in overcoming the limitation of the older techniques. In one such study, Rawat et al. tried to overcome the limitations of collaborative filtering caused by the sparsity of data by utilizing Jena rules using Resource Description Framework (RDF) [14]. Similarly, Nabizadeh et al focused on improving the quality of learning while considering time constraints by allowing the user to choose the first item then utilizing a graph-based recommender that considers both the available time of the users and their prior knowledge [15]. Another research conducted by Rafiq et al utilized queries given by the user to recommend both the relevant courses and the answers to the questions that were asked by a similar level of understanding [11].

One Research conducted by Niknam & Thulsirman used a hybrid methodology to design an effective recommender system. They utilized fuzzy c-means (FCM) for clustering and Ant colony optimization (ACO) for generating effective learning path recommendations (LPR) for the students which is graphically explained in Figure 1. [16]. In Figure 1, it is displayed that pre-test taken from the user will determine initial understanding of the concepts. Re-tests will be taken side by side to improve the quality of recommendations.

The existing literature in this survey considered different dimensions of the subject under study. This study included but is not limited to MOOC courses. The exiting surveys are summarized in Table 1. They include a study conducted by Mu considered deep learning approaches for all kinds of recommender systems that mostly including music, movies, and books recommender systems [17]. This study did not include existing researches for online course recommender systems.

Another survey conducted by Nabizadeh, Leal, et al focused on LPR in intelligent tutoring Systems (ITS). The purpose of their study was the analysis of the long-term goals of learners in existing literature [18]. Another study [19] included the swarm intelligence methodologies for generating recommendations for e-learning platforms. A study conducted by Nikzad–Khasmakhi et al analysed the work done in area of expert recommendation system [20]. The expert recommendation system finds experts based on the query entered by the user while utilizing Natural Language Processing and Information Retrieval methodologies. The study conducted by [21] reviewed work done in ontology-based models to find most relevant study material while considering personalized preference of the user.

Figure 1. Hybrid methodology to improve learning path recommendation [11].

Figure 1. Hybrid methodology to improve learning path recommendation [11].

Our study majorly includes the recommendation systems that are designed solely for e-learning and does not include other areas like e-commerce and entertainment. The study also includes various techniques and methodologies used for e-learning recommendations. This study will help readers understand different trends of recommender systems in e-learning.

The key contributions of this survey include:

• Exploring existing methodologies to design an effective recommender system in an e-learning environment
• Analyzing existing methodologies and comparing them to identify the most promising area for the future.
• Identify knowledge gaps in the existing literature and identify future challenges.
• Identifying the objective of each research and comparing objectives of existing literature.
• Evaluate the performance of existing methodologies and compare them with each other to identify the one methodology that is outperforming others

The different sections of this study are depicted in Figure 2 that include section 1 that introduces the topic, section 2 that consists of all the literature covered in this study, section 3 contain a comparative analysis of objectives of different researches and comparison of comparable values, section 4 includes gaps and future challenges while section 5 concludes the study with some discussion in detail.

2. Survey Methodology

This survey was conducted on research articles published between the year 2018 and 2020. Query was formulated including keywords recommendation or recommender system, grade prediction and future performance. The data was retrieved from ACM, IEEE, Springer, Google Scholar and Science Direct. The retrieved papers included recommender systems for both online and offline course. We screened the research papers to include only the papers that focused on e-learning and online courses. The resultant relevant papers mostly used different techniques to utilize dataset obtained from log data of various LMS.

The papers relevant recommender systems were further divided into six sub-sections based on the techniques used and were studied separately as shown in Figure 3. There were a few papers that did not fall under any of the major categories so we included them in a separate section and named it as other emerging recommendation techniques in education.

3. Literature Review

Recommender system in online learning is an emerging area of interest due to the increasing demand for online courses [22]. Different techniques are being used to not only improve the recommendations from an available set of courses but recommendations can also be generated for content available within a course. The following sections will discuss this detail to provide better insight to the readers.

3.1. Graph-Based Techniques

Graph-based techniques focus on organizing content in a graphical structure in form of edges and vertices. Traversing through a graph can help in the identification of a path that can provide learners a road map that they can follow to improve their performance. The graph-based technique can effectively recommend content within a course as well hence increasing the efficiency of the students.

One such study conducted by Nabizadeh et al focused on maximizing the performance of learners’ results while taking into account the limited time that a student can spare for a course. They designed a framework that allows users to choose the first item then a graph-based recommender designed using the available time of the users and their prior knowledge based on the item they had chosen first. [15]

Their proposed model focuses on covering all concepts instead of completing all lessons. In the proposed methodology, a two-layered graph is constructed in which one layer consists of lessons while the other layer consists of Learning Objects (LOs). Cluster means and cluster median is used to create groups of slow, fast, and medium learners while matrix factorization estimates the time expected to be taken by the user to complete a specific LO. The path that takes minimum time is recommended to the user.

The experimental results indicated that the cluster median algorithm performed well in most cases by showing around 90% accuracy. The model was implemented, and an online evaluation was also performed. The results showed that the control group performed well only in initial lessons while the experimental group showed average performance in all lessons.

Another similar study was conducted to provide a personalized guideline to the students in the selection of a course by using several techniques like content analysis, assessment, and behavioral analysis [23]. This study is based on Bayesian knowledge tracing (BKT). BKT is an statistical algorithm that helps to infer the current knowledge state of students [24]. BKT is used in educational data mining and learning analytics to model and analyze student learning. The algorithm is based on Bayesian probability theory and is used to estimate the probability that a student has mastered a particular skill or concept based on their responses to a series of questions or exercises. This helps to predict if they have learnt a skill or not.

Four variations of BKT were used for knowledge, tracing in the proposed model including Coarse BKT, multi-grained BKT, Fine BKT, and Historical BKT [23]. Experiments were performed to predict dropout rate and a t-test was conducted on the results which indicated that the multi-grained BKT showed the best performance due to its ability to handle hierarchal structure.

Another study conducted by Wan & Nu focused on the personalization of recommendations using LOs [25]. For existing users, their proposed model stores a profile that can be updated with time while for new users, the system offers a survey that helps in constructing a user’s profile. Recommendations are generated by constructed ontology-based graph using the user’s profile and similarity between LOs. The model also considers learners’ learning styles and learning goals while recommending material.

An Intelligent Tutoring System (ITS) was proposed by Muangprathub, Boonjing, and Chamnongthai. The focus of this research is to adapt the system according to the requirements of the user based learning style. The system combines information of the learner including knowledge state and learning styles with the existing knowledge repositories along with various teaching strategies [26]. The information about the course was extracted in form of objectives, contents, exercises, and tests. A personalized path was generated for each user utilizing the knowledge-based graph using the co-occurrence of different objects. To test the proposed methodology, a web-based application was deployed, and the performance of the system was measured. The results showed that the users who interacted with the proposed model performed significantly better with a mean score of 10 as compared to the control group with a mean score reaching 5.

Similar research conducted by Zhou et al utilized the Long Short-Term Memory (LSTM) model of deep learning along with personalized path generation [27]. LSTM networks were introduced by Hochreiter and Schmidhuber [28]. LSTM networks are designed to overcome the vanishing gradient problem in standard RNNs. LSTM networks have a unique memory cell that allows them to selectively store and access information over extended time periods.

To overcome the cold-start problem, their methodology proposed a separate path where sufficient prior information was not available to generate recommendations [27]. The users are also divided into five main clusters based on their interaction with the platform and then the learning effect is calculated using the LSTM model. The results of their proposed methodology indicated that using clustering along with the LSTM model showed the best performance with a precision value of around 0.45. The value of precision for LSTM after clustering increases as the number of learners’ increases hence showing its effectiveness for large amounts of data.

One research conducted by Niknam & Thulasiraman used hybrid methodology using both clustering and graph-based method to design an effective recommender system. This study focuses on Learning Path Recommendation (LPR) using Ant Colony Optimization (ACO). Ant colony optimization was originally called Ant system and was proposed by M. Dorigo et al. [29] ACO is a kind of swarm intelligence and is a metaheuristic optimization algorithm inspired by the behavior of ants searching for food. The algorithm is often used to solve combinatorial optimization problems. In ACO, a colony of artificial ants iteratively constructs a solution to the problem by moving through a graph or network of possible solutions. In the research conducted by [16] the learning objects are mentioned as concepts that are extracted by using the concept Map tool. In their proposed methodology, learners take a pre-assessment test to identify the concepts familiar to the user. Fuzzy C-means (FCM) clustering is used to assign learners to a cluster. FCM was first introduced by Dunn [30]. The algorithm is based on the principles of fuzzy logic and set theory. It is an extension of the K-means algorithm that allows for soft clustering that allows data point can belong to multiple clusters with varying degrees of membership. FCM was chosen in the research [16] because it doesn’t create rigid clusters but creates softer boundaries. ACO algorithm helps in finding the most effective path for new learners by using data of old learners of the same clusters. The progress of the learners is continuously monitored and change in recommendation is made as and when required by continuously updating the graph. They implemented their algorithm using java and a user interface was created to test it for real-world data. Fifty learners participated in the study for three weeks. In the pre-assessment test, both groups scored and an average of 30 marks while the experimental group performed better in the post-test with an average of 65 marks while the control group students achieved an average of 58 marks.

Some researches in this area are based on query-based system where the user enters a query and system generate a list of recommendation based on that query. One such study was conducted by Shi et al. in which their proposed model used a graphical structure to store LOs and then recommended path according to the profile of a user [31]. The knowledge graph used in this research utilized LOs and recorded the semantic data regarding the relationship between these LOs. Their proposed methodology constructed a multi-dimensional knowledge graph that stores class-wise hierarchical structure that includes basic knowledge, algorithm, and tasks. All these classes include LOs but from different perspectives. Their framework initially inputs a query from user and then collects the target LOs after tokenizing the input query. The semantic relationship between LOs were utilized to generate a more effective path. A system can generate multiple paths for one query. In that case, the recommendation algorithm considers the weight of features for all learning paths while assigning the highest score to the most popular feature. The system recommends the path with the highest score.

They conducted experiment including test and control group. The test group showed much higher user satisfaction with scores reaching up to 0.9 while the control group showed an average reaching up to 0.45.

The research is graph based technology mostly utilized learning objectives to either recommend a path or to generate query response. The Table 2 presents a comparison of results indicating how different methodologies utilized different evaluation measures depending upon the nature of problem and proposed solution. Some methodologies that used similar methodologies such as [16] and [26] indicated that clustering along with swarm intelligence can increase learning gains.

3.2. Content-Based and Collaborative Filtering

Content-based filtering and collaborative filtering are mostly used together in hybrid systems because they help to overcome each other’s limitation when they are used together. Some existing literature used both techniques together while utilized another technique as well to further enhance the performance.

3.1.1. Content-Based Filtering

Content-based filtering is a type of recommendation systems. It generates recommendations to users based on their past behavior and preferences. It works by analyzing the characteristics of items that a user has already liked or interacted with and recommending items that are similar to those items. One of the benefits of content-based filtering is that it can provide personalized recommendations to users even if they are new to the platform. The earliest content based systems can be traced back to mid-nineties when researchers developed a recommendation system called Fab, which used item features and user profiles to recommend movies [32] while another system called GroupLense was developed to recommend news items [33]. That recommender used hybrid system by utilizing both content based filtering and collaborative filtering.

This section of survey will cover the existing research that included content-based filtering along with collaborative filtering to improve recommendation results. Once such a study was conducted by Xiao et al. that utilized content and collaborative filtering for relevant recommendations for MOOC courses [34]. Their proposed framework picks one of the techniques depending upon a threshold value. If the threshold value is met, then the algorithm will take the path towards collaborative filtering but if the dataset does not have enough previous ratings to meet the defined threshold then the algorithm will take the path of content-based filtering.

Another research conducted by Neamah & El-Ameer proposed a framework that utilized the course description, user’s profile, and ratings to determine the most relevant course for a user [35]. They made it mandatory for the instructor to enter a course description. The user’s preference and profile are then compared with the course description to determine relevant courses. On the other hand, the user’s profiles are also updated based on the courses they picked.

Their proposed methodology consists of two phases. The first phase uses K-nearest neighbour (KNN) classifier to determining the most relevant courses based on the user’s profile and this portion of the proposed methodology will be executed for every recommendation. The second phase is optional and is executed only when enough ratings are available in the database for the relevant courses. In the second phase, the Naïve Bayes classifier is used to recommend courses based on the user’s rating.

The results proposed methodology were measured in terms of precision, recall, and f-measures. KNN showed .43 precision, .50 recall, and .46 f-measure value while Naïve Bayes showed 0.75 precision, 0.50 recall, and 0.60 f-measures. Apparently, Naïve Bayes showed better overall precision but it should be considered that naïve Bayes is executed only for relevantly complete data hence showing better results. This also indicates the importance of user’s rating in course recommendations as an important factor for online course recommendation.

The online learning platforms are different from traditional education institutions as they do not have a predefined sequence in which different courses should be taken. This can cause a problem for learners especially the beginner who does not know which course should be taken first. To solve this problem research was conducted by Tarus et al that utilized a hybrid recommender system that included content-based collaborative filtering and sequence pattern mining [36]. This helps in identifying a sequence in which courses should be studied.

Their model considered learner’s profile that included both demographics data and the ontology-based data created by recording user’s interaction with the system, LOs that were derived from the learning material and content, weblogs that stored user’s interaction with the system. The proposed framework used both content and context-aware course recommendations by utilizing the weblogs, LOs and user profiles. The generalized sequential mining (GSM) algorithm was used for identifying the recommended sequence of courses. GSM is a data mining technique used to discover frequent sequential patterns in a sequence database. It was first introduced in 1993 by Agrawal et al [37]. GSM first converts database into a set of generalized sequences hence generating a sequence of item sets. Each item set in a represents a set of items that occur together in a transaction or event.

The result of the hybrid methodology proposed by [36] was compared with individual algorithms and the proposed hybrid model showed the best precision and recall values as compared to any of the three methodologies (CF, CB, or GSM) with 0.35 precision and 0.45 recall.

Another similar research conducted by Klašnja-Milićević et al used Apriori to find pre-requisites and utilized different learning styles while recommending online courses [38]. Their proposed methodology utilized tag-based Collaborative filtering or tagging which is an application of collaborative filtering. Tagging-based techniques are natural language processing methods that involve labeling words or phrases in text with tags that represent their part of speech, named entity, sentiment, or other characteristics. In their proposed method, the learners will write tags according to their requirements.

Their proposed framework was implemented on an online learning management system called Protus which is a programming tutor. Initially, the user completes registration which includes a survey that helps in the identification of learning style. Every newly registered user is divided into a cluster based on his/her learning styles and the courses will be presented to the user according to his/her learning style. The tags were created by the user so the results of the proposed solution indicated that only 55% were covered when they were matched with tags created by experts. In the future, some suggested tags can also be utilized keeping in view the experts’ generated tags to ensure vast coverage of LOs.

Another study conducted by De Medio et al introduced another dimension to the existing research by considering the perspective on instructors in course recommendation [39]. They proposed a teacher-centric recommender system that helps instructors in identifying courses similar to those they are interested in teaching. The proposed MoodelREC system utilizes popular LO repositories to crawl LOs. Then both content-bases similarity and collaborative filtering are used to rank the LOs. The instructor enters a query in the provided interface which is utilized to generate recommendations. Their proposed methodology was implemented and tested by comparing the predicted rating with the actual rating of the user. The results indicated a precision of 0.22 and a recall of 0.39. Hence indicating a lot of room for work in this area in the future.

3.1.2. Collaborative Filtering

Collaborative filtering is a type of recommendation algorithm that utilizes the behavior and preferences of multiple users in order to recommend items to a specific user. One of the very early recommender systems using collaborative filtering was presented by Goldberg et al in the early nineties. The presented model presented is still being used in some hybrid systems for generating recommendations in various fields [1] and is based on the idea to collect and analyze data from users having similar preferences in the past and considering that they are likely to have similar preferences in the future. Collaborative filtering uses the ratings or feedback provided by users to predict their preferences for items they have not yet interacted with. There are two main types of collaborative filtering: user-based and item-based. User-based collaborative filtering recommends items to a user based on the ratings of similar users. Item-based collaborative filtering recommends items to a user based on the similarity of the items to items the user has already rated. Collaborative filtering cannot be applied alone due to its limitations like the cold start problem. Different methodologies like learning styles prediction, content-based filtering, association rules mining, sequential pattern mining or deep learning are used along with collaborative filtering to overcome its limitations.

One such study conducted by H. Zhang et al combined a deep belief network with collaborative filtering for more relevant recommendations [40]. For this research, they used a dataset from real users in StarC platform. The dataset after pre-processing is fed into both supervised and unsupervised Deep Belief Networks (DBN). DBN is a type of Artificial Neural Network (ANN) that is composed of multiple layers of interconnected neurons. It is mostly used for unsupervised learning tasks such as feature extraction and dimensionality reduction and was initially introduced by [41]. The proposed methodology in [40] utilized ratings of users and used them for supervised DBN while unsupervised DBN doesn’t consider the user’s rating. They used Restricted Boltzmann machines (RBM) for constructing neural networks for unsupervised learning. RBM is a type of artificial neural network that is used for unsupervised learning tasks [42]. The results showed a minimum RMSE of 67.48% for their proposed methodology hence outperforming the existing methodologies [40].

The dropout rate in MOOCs is higher and greatly depends on user satisfaction. Hence the dropout rate can be reduced by increasing the user’s satisfaction. One such study conducted by Pang et al utilized collaborative filtering along with learning series techniques to maximize user satisfaction [43]. They proposed a methodology that is based on Recommendations with Learner’s Neighbour and Learning Series (RLNLS). The RLNLS model used inverse distance weight to calculate the past achievement of the user which can be used to predict their future achievements. Collaborative filtering technique was used to find the behaviour of neighbor learners while Hawkes process was used to predict the learning intensity. The recommendations hence generated utilized both the information regarding neighboring learners and the capacity of the target learner. The RLNS model showed the most accurate results with a precision value of 0.34, recall value of 0.38, and f-measure value of 0.35.

The data that is being collected from online learning platforms is getting sparse with time. This can compromise the efficiency of collaborative filtering algorithms. A study conducted by Rawat et al. tried to overcome the limitations of collaborative filtering caused by the sparsity of data [13]. Their research used collaborative filtering while utilizing Jena rules using resource description framework (RDF). Their proposed methodology first collects information from the user using Moodle then calculated its RDF representations so it can be further utilized by Jena rules engine. The Jena Rules engine is a forward-chaining rule engine that allows developers to define rules to infer new knowledge from existing RDF data. KNN is applied to further refine the results and then final recommendations are generated. The recommendation list is evaluated and then is presented to the user.

For the evaluation of the proposed methodology, they used RMSE, precision-recall, and f-measures. The active users’ data showed results of 0.25 RMSE, 0.98 precision, 0.90 recall, and 0.93 value for f-measures. The average users’ data showed results of 0.48 RMSE, 0.97 precision, 0.84 recall, and 0.90 value for f-measures. The non-active users’ data showed results of 0.51 RMSE, 0.64 precision, 0.27 recall, and 0.38 value for f-measures.

Research conducted by Obeidat, Duwairi, and Al-Aiad proposed a model that utilizes the courses opted by the student in the past to recommend similar courses to the new students. [22]. They utilized collaborative filtering along with clustering and association rules mining for better course recommendations. They utilized sequence mining that helped to determine prerequisites and in recommending courses in the correct sequence. For clustering, they used k-means algorithm that uses Euclidean distance to groups, similar students, together. Apriori algorithm was used for association rules mining while SPADE (Sequential PAttern Discovery using Equivalence classes) algorithm was used for sequence rules mining. Apriori algorithm works by first finding all frequent itemsets in the dataset [44] while SPADE is used to discover clusters in high dimensional dataset [45]. The coverage, support, and confidence were considered while ranking the generated association rules. The highest-ranked rules were utilized to generate recommendations. An analysis is performed on the prediction of future grades. If students are predicted to score higher, then the course is recommended to the students otherwise it is excluded from the list of recommendations.

The results indicated that 5 is the most optimal value for the number of clusters. Both Apriori and SPADE algorithms showed better coverage of courses after clustering as compared to recommendations generated without utilizing clustering. Apriori showed 60% coverage with clustering while without clustering it showed 48% coverage. Similarly SPADE algorithm showed 56% coverage with clustering while without clustering it showed 51% coverage.

Another study conducted by Murad et al. explored the recommendation of study material within a course at early stages [46]. This research mainly focused on improving the performance of students in an online course by recommending to them the most relevant study material in time. Their proposed solution uses the profiles of old students to predict and recommend relevant study material to the new students. Their proposed framework combines collaborative filtering with rule-based filtering and KNN classification to create a hybrid system. The recommendation hence generated will be context-sensitive and will depend on the choice of the user. The results showed that adding context sensitivity showed the same Mean Square Error (MSE) as the other baseline methodologies that did not utilize context-sensitive information. This indicated the ability of the proposed methodology to add personalization in a system without compromising the quality of recommendations.

On an online platform, different courses with similar titles may include the content of varying difficulty level. One such research was conducted by Segal et al. that focused on calculating the relevance of a course based on its difficulty level [47]. Their proposed EduRank method used collaborative filtering along with social voting to evaluate the relevance of a course. The algorithm first collects data from students and assign ranking based on the answers. For each question, the social choice methodology is applied and for this purpose, the Copeland method is chosen. The Copeland method helps in collecting votes that label a question as difficult, not difficult or tie. Item response theory and Bayesian knowledge tracing were used for calculating item difficulty. If questions were solved quickly then they were easy, it took more attempts than were difficult. AP correlation matrix was used to evaluate the proposed method and perform a comparison with competing methodologies.

The results indicated that the AP correlation for the EduRank algorithm was much higher as compared to the competing methodologies like SVD, EngenRank, and topic-based ranker (TBR) indicating its better performance.

3.1.3. Matrix Factorization

Matrix factorization is a type of collaborative filtering technique that is most commonly used in recommender systems. Its use in educational recommendation system is also prevalent. One such study conducted by Symeonidis & Malakoudis used multi-modal matrix factorization for course recommendation in MOOCs platform [48]. The proposed methodology presented a new dimension of user and course skill set. This matrix calculates either a course provides a particular skill set that a user is looking for. The most frequent items in the neighborhood are ranked the highest and they appear higher on the list while recommending courses to the user. For calculating the similarity between the users, the proposed technique used cosine similarity.

Their results indicated that the user-item matrix generated only by utilizing ratings showed higher RMSE as compares to the matrix that included course content as well as the ratings. The results indicated that ratings with course content and student’s skill set showed minimum RMSE value.

The CF and CB methodologies are widely used in recommendation systems but may use a different methodology to design a hybrid solution. The comparison of existing methodologies are compared with each other in Table 3 that indicates that most of the existing research in this area such as [13], [35] and [36] used precision, recall and RMSE or MSE for evaluation. Some researches such as [22] and [38] also utilized coverage as evaluation measures. The results indicated that using association rules mining along with CF or CB increased precision and recall and decreased RMSE. Association rules mining also improved results for coverage as well.

3.3. Search Query Based Recommendations

Searching relevant material by entering a search string is a popular method to fetch relevant material on the internet. Due to the complex nature of an educational setting and an increasing number of similar online courses, there is an emerging need to interpret the query entered by the user according to his or her personal needs. One such research conducted by Lalitha and Sreeja proposed a method that majorly focuses on the older audience who are self-learning and do not have access to any recommendations from teachers or instructors [49]. Their research aimed to design a system that can accommodate learners who have no idea from where they should start learning. Content based filtering, collaborative filtering, and web mining were used to identify appropriate learning material on the internet which was later narrowed down according to the personal needs of the learners. Clustering is done using KNN to group similar learners in one cluster to maximize personalization. Once a user enters a query, the recommender system consider information extracted from the learner’s profile and the nearest neighbor. The recommendations are generated based on similarity checks.

Another similar study conducted by Rafiq et al utilized the intelligent query optimization technique by introducing multiple agents to optimize generated recommendations [11]. Their methodology focused on prompting the user to enter information and to also give feedback regarding generated recommendations. Association rules mining is utilized for generating the most relevant recommendations. The results of their proposed methodology showed around 90% accuracy, precision, and recall values that fluctuated a little as the number of lectures were increased but all values stayed up to 89%.

The research covered in this section utilized Association rules mining, KNN and knowledge based filtering. Their results are compared in Table 4.

3.4. Educational Recommender System Using Big Data

The increasing number of online learning platforms and learners is generating massive amounts of raw data that can be utilized to generate recommendations for future learning. Big data refers to extremely large and complex data sets whose processing is beyond the ability of traditional data processing and management tools. There are several existing recommender systems in online learning that utilize big data to generate more effective recommendations. One such study was done by Zhang et al proposed a course recommendation model that utilized big data to generate association rules in a distributed environment to generate the most relevant related courses that can help in generating ranked list recommended courses for online platforms [50]. They utilized Hadoop Distributed File System (HDFS) while using Spark for data processing for association rules mining and Sqoop to import and export datasets. Course recommender used the Apriori algorithm to generate recommendations of the most relevant courses.

Another similar study conducted by Dahdouh et al also utilized HDFS and Spark for big data analysis [51]. The purpose of their study was to discover the most relevant course by implementing association rules mining on big data. For this purpose, they used Spark ML libraries to implement the FP-Growth algorithm. For course recommendations, they analyzed the behavior of learners who interacted with the system in past by utilizing course enrollment and log data. The same model was studied by using more traditional systems like weka and R. The comparison with SparkML with Weka and R indicated that SparkML took minimum execution time while execution time was maximum for Weka.

The data collected from online learning platforms is not only massive but also is very diverse in nature due to its access to a global audience. Education is a subjective term so qualitative analysis is also important in this area to generate better recommendations. The research conducted by Hou et al utilized the feedback of the users to improve the model [52]. They proposed a methodology that divides all courses in different clusters. When users enter the systems, they are recommended courses based on their demographic data. For users who have visited the system earlier, a learning profile is also utilized that includes information regarding courses taken in past. A personalized tree is constructed for each user and courses are recommended based on that personalized tree. If the user is satisfied with the recommendation, the recommender system is rewarded for it and it learns it as a success. This information can help the system in future recommendations. Similarly, unsatisfactory feedback will generate regret and the system will also learn and update its recommendations for the future. The resultant recommendation not only covers personalization but also the course sequencing by observing the course selected in a sequence by past students.

Research conducted suing big data mostly utilized Association rules mining and reinforcement learning. Different methodologies are compared in Table 5.

3.5. Recommendations Using Forum Posts

Just like online courses, online forums are another educational resource which is widely popular among the learner’s especially when it comes to fetching quick response. Several online forums provide answers to popular questions. The recommender system in this scenario can help in extracting information from these discussions. One such study conducted by Porcel et al utilized a hybrid technique based on fuzzy linguistics and ontology-based approach [53]. Fuzzy logic is a type of mathematical logic that deals with approximate reasoning [54]. Fuzzy linguistics does not views linguistic expressions as either true or false. It is helpful in capturing the complexity of natural language. An ontology is a formal specification of a shared conceptualization of a domain that provides a common vocabulary and a shared understanding of the entities, concepts, and relationships between them. The purpose of their research [53] was to identify the interest of the users and present recommendations based on their interests. They proposed a framework called sharing notes that is a web-based forum designed to facilitate students by providing them personalized material. Sharing notes utilized collaborative and content-based filtering to rate the material while the fuzzy linguistics model was implemented to perform the qualitative study on the communication that takes place on the forum between different users. Ontologies are used to map the relationship between users and their preferred material. Trust is also calculated for the resources posting material. The final recommendations are generated after obtaining ratings from the trusted users. Sharing notes was tested with the help of 425 users amongst those 88% of the users show satisfaction with their proposed recommender system.

Another similar study conducted by Albatayneh, Ghauth, and Chua focused on the discussion forums of online learning platforms [55]. The previous methodologies were based just on negative or positive ratings and new recommendations were generated using content-based filtering. Their proposed methodology added the layer of semantic analysis as well. Semantic analysis is the process of understanding the meaning of language by analyzing the meaning of individual words, their relationships to other words in a sentence, and the overall meaning of the text. They implemented and tested their model in a form of a case study. The results showed that semantic based content filtering technique with negative ratings showed a minimum MAE of 0.25.

Another forum-based study conducted by Ntourmas et al pointed out the limitation of platform independence in existing research [56]. They aim to solve this limitation by increasing the transferability of the system to make it applicable on other platforms as well. They proposed a solution that performs semantic analysis on messages posted on discussion forums and implemented this methodology on two MOOCs. For this purpose, they divided posts into three categories, content-related posts, logistic-related posts and posts on which no action is required. Support Vector Machine (SVM) was utilized in the end to generate recommendations for relevant posts. SVM is based on the idea of finding a hyperplane that separates the data points into different classes. The data points that lie closest to the hyperplane are called support vectors [57]. The results of their proposed methodology indicated up to 65% accuracy, 0.59 precision, 0.65 recall, and 0.61 value for f-measure.

Most of the research in forum post discussion context used semantic analysis because in forum post discussion understanding the language and emotion part is important. Methodologies like fuzzy linguistics [53] and semantic analysis [55] are the main focus in this area.

Table 6. Comparison of results for forum post discussion.

Reference	Methodology	Performance measure	Results
[53]	Fuzzy linguistics and ontology-based approach	User satisfaction	88%
[55]	Semantic analysis and collaborative filtering	MAE	0.25
[56]	Semantic analysis and SVM	Precision	0.59
		Recall	0.65
		Accuracy	65%

Table 6. Comparison of results for forum post discussion.

Reference	Methodology	Performance measure	Results
[53]	Fuzzy linguistics and ontology-based approach	User satisfaction	88%
[55]	Semantic analysis and collaborative filtering	MAE	0.25
[56]	Semantic analysis and SVM	Precision	0.59
		Recall	0.65
		Accuracy	65%

3.6. Other Emerging Recommendation Techniques in Education

Apart from the techniques and methodologies mentioned in previous literature, some new areas are being explored. One such study conducted by G et al focused on the group of young adults from the generation Z who have different learning styles as compared to older generations [58]. Their proposed methodology takes the data of online courses as input and ranks them into various difficulty levels. Similarly, students create their profiles and their data is also extracted through the log files that record the data of a student interacting with the system.

Another research conducted by Dwivedi, Kant, and Bharadwaj proposed a methodology to optimize the course sequencing which can be a very challenging task for online course recommendations [59]. They collected data from the learner’s profile and used variable-length genetic algorithms for learning path recommendations utilizing data of both new and old students. Collaborative filtering was used to find the popularity matrix which was later utilized by a genetic algorithm for personalized course sequencing.

Another research conducted by Lin, Li, and Lian aimed to solve the problem of goal-oriented generation of recommended courses by using a convex optimization algorithm [60]. Their study focused on university students as their audience and proposed a data-driven approach based on student’s ratings and performance. Their proposed modified clustering algorithm that embeds optimization technique showed more than 90% accuracy by outperforming the existing techniques [76,77,78,79,80,81,82,83,84,85,86,87,88,89].

Other methodologies that used different approached along such as genetic algorithm, convex optimization and SVM along with collaborative filtering utilized different evaluation measure. Their results are compared in Table 7.

3.7. Critical Analysis

All of the literature reviews discussed so far have been summarized in Table 8 which includes techniques and limitations of major researches. The limitation of few types of research was covered by newer researches in the existing literature and they are included in this research. For example faced cold start problem [15] was overcome by utilizing content-based filtering along with collaborative filtering [14].

4. Comparative Analysis

The previous section discussed the existing literature in the recommender system for e-learning in detail. This section will focus on comparing the existing researches with one another to identify the strength of existing techniques.

4.1. Objective Based Clustering

We will go into the detail of objective based analysis of our literature review in this section. The previous sections indicated that there are a variety of objectives and directions that can be followed to enhance the effectiveness of teaching-learning experience on e-learning platforms. The main objectives of our overall literature are summarized in Table 9 which indicates that most of the present literature focus on personalization and accuracy. Some researchers use precision, recall and f-measure while other use Root Mean Square Error (RMSE) as well. Learning path recommendation is also a direction while some researchers considered execution time and memory utilization as a measure of effective recommender as well.

4.2. Dataset analysis

Since most of the research conducted for recommender systems in e-learning area uses wide variety of datasets. The detailed descriptions of various datasets along with links and references where applicable is listed in Table 10. Some researches like [55] constructed their own dataset while some researchers like [50] used public data set for MOOCs. While LMS datasets of universities is also utilized by some researches like [36] and [47]

4.3. Performance analysis

The objective based clustering has helped us in identification of common objectives. It is also important to compare the performance of these methodologies in order to identify the methodologies that are outperforming others in the same domain. We can observe form Table 11 that most of the researches focused on measuring precision, recall, f-measure, RMSE as an accuracy measure. Some researches like [26] focused on execution time as well. The performance analysis table is constructed for comparable values and survey results may vary from scenario to scenario and cannot provide a uniform assessment hence they are excluded from the Table 5. The results show that methodology proposed by [14] shows best precision, recall and f-measure values.

4.4. Comparison of Best Scheme From Each Category Existing Schemes

The existing literature includes various schemes being used for different objectives. A brief comparison of some of the major existing schemes is summarized in Table 12 that shows that performance for most of the existing schemes were measured in terms of accuracy. Results for active user showed better accuracy as compared to non-active users [14] while full course recommendation showed less accuracy as compared to step by step recommendation in some methodologies [26]. In some cases performance stayed constant with increasing size of dataset [52] while in some cases is slightly dropped [14].

5. Research Challenges and Gaps

The existing research in the area of educational recommendation specifically for online learning platforms is of various kinds. All types of recommender system work for a particular objective and area and every area has challenges of its own.

5.1. The Complex Relationship Between LOs

The graph-based methodologies are mostly used in ITS or within course recommendation like [25]. The graph-based methodologies that recommend different courses mostly face cold start problems like [18]. Some query-based methodologies like [27] utilized graph-based methodology for query optimization but did not consider the complex relationship between different LOs.

5.2. Cold Start Problem

The content based filtering was utilized in various researches like [34] which proposed two-phases of research. The phase that included only content based filtering showed less accurate results as compared to the one that combines collaborative filtering with content based filtering [38] and [31]. Some researchers utilized association rules mining [36] while others used sequence pattern mining [35]. The limitations in existing research vary for different methodologies. The content based filtering excludes the item and user based ratings hence ignoring the rankings generated from old users. The collaborative filtering technique has a cold start problem.

5.3. Increased Processing Time

Association rules mining and sequence pattern mining can significantly slow down the system. Recommendations using big data is also gaining popularity as the volume of online learning platform data is increasing. The existing researches needs enhancements to handle ever increasing amounts of data [52] and real-time data [51]. Both of these tasks can be challenging due to the complexity of currently used methodologies. Light weight model can be utilized to reduce processing time in future.

5.4. Limited Availability of Public Data

Query-based recommendations is another emerging area that has been explored by [49] and [11]. The future enhancement can be achieved in this area by utilizing the profile of both the learner and the instructor who is posting the course while generating recommendations. This direction can face challenges due to limited availability of public data.

5.5. Sematic Study

Forum-based discussion presents some emerging trends in research that can be utilized in semantic study to recommend courses in future [53]. Online learning platforms have a discussion forum that can be effective in analyzing the quality of the course [55]. Semantic study is an important area that can be explored using deep learning [56].

6. Optimal solution

The previous section described various problems like cold-start problems and un-availability of datasets. Since the research in this area is scattered and vary from solution to solution hence it won’t be possible to design one optimal solution that works for all kinds of problems, for example recommendations generated by ITS utilizes contents within a course that cannot be used by a generic course recommender system. That is why we shall look for multiple optimal solution instead of one that works for all scenarios.

In the existing literature there are some researchers used graph based methodologies to capture knowledge base [31]. Deep learning algorithms can be utilized in future to capture more complex relationships between LOs as they are being used in other areas of text analysis [61].

The existing methodologies also has cold-start problem in some papers [34] which has been overcame by combining content-based filtering with collaborative filtering. Using LSTM models along with collaborative and content based filtering can help in solving cold-start problems. [27]

Existing researches mostly construct their own datasets due to little availability of public datasets. Datasets for future researches can be constructed by scraping the online resources. MOOCs and online question answers forums like stack-overflow are good sources for scraping data. Video content can also be utilized to construct dataset by utilizing subtitles [23]. Increase in the size of dataset will further require optimized solutions for complex problems that can be resolved by utilizing deep reinforcement learning [52].

Research in the area of semantic analysis is scarce for recommender system. Hence further work can be done to improve existing methodologies by combining sematic study with deep learning algorithms. Forum discussion are rich source of such datasets that can be explored in future. Deep learning algorithms can help in finding context in semantic analysis of such dataset. In conducting a comprehensive survey on improved E-learning based recommender systems, our research builds upon foundational insights presented in [62,63,64,65,66,67,68,69,70,71,72,73,74,75]

7. Proposed Solution

The proposed solution for an optimized recommender system cannot be universal as the nature of problem can vary. Here we considered a solution for MOOCs where we get diverse audience from all around the world searching for different material. In this scenario, same keyword may match the title of various courses hence evaluating description of course is important. Since anyone can post material in MOOCs platform so evaluating the quality of the content in this scenario can be another hurdle. The size of catalog for online courses is increasing and is expected to increase in future hence making the task of generating optimized recommendation more challenging.

The proposed solution summarized in Figure 4 tries to tackle all of the above mentioned challenges by utilizing course rating, comments and descriptions for sentiment analysis, content-based filtering, collaborative filtering and association rules mining. Use of deep learning will improve the results of recommendations while association rules mining can provide a more in depth insight about the relationship between keywords and selection of courses.

8. Conclusion

Recommender systems are gaining popularity in various fields like e-commerce and entertainment. Its use in education is also getting popular with the increasing popularity of online learning platforms. The existing literature in educational recommender system include areas like graph-based methodologies, ITS, query optimization, content based and collaborative filtering, big data and association rules mining. This survey aimed to explore emerging directions of recommender systems in education. This study analyzed existing literature and performed objective-based analysis. Performance analysis was also considered where the values were comparable. Limitations of existing researches were also identified and studied to identify research gaps for future directions.

In the future, the directions that are appearing to be gaining popularity are big data analysis and hybrid recommendation systems. There is an increasing trend of shifting to online learning mode instead of traditional classroom which has paced up due the COVID-19 pandemic. The increasing demand of online learning platforms has also increased the demand of MOOCs and a lot more people are turning towards MOOCs as an alternate source of learning [10]. This as a result, has increased the need of effective recommender systems. More diverse group of people are opting to learn online which demands more personalized recommendations that considers the requirements of particular region along with other factors. Instructors’ profiles on online learning platforms can also be considered to ensure their authenticity. Semantic analysis on online discussion forums of e-learning platforms can also be explored in future to generate more effective recommendations.