Altmetrics
Downloads
216
Views
69
Comments
0
This version is not peer-reviewed
Submitted:
20 December 2023
Posted:
21 December 2023
Read the latest preprint version here
Authors | The topic of the survey conducted | Limitations |
[17] | Recommender systems using deep learning in the existing literature | This survey was limited only to the deep learning techniques and left major areas like content-based and collaborative filtering |
[18] | Survey on existing ITS | It only covered ITS while left all other techniques used for recommendations |
[19] | Surveys on swarm intelligence | It only included researches using swarm intelligence |
[20] | Recommending experts in a field | Limited scope of query-based search and information retrieval. |
[21] | Ontology-based recommender systems in e-learning. | Studied only ontology-based models that mainly focus on personalization |
Reference | Methodology | Performance measure | Results |
---|---|---|---|
[15] | Cluster mean | Accuracy | 90% |
[16] | FCM and ACO | Learning gain | 33 |
[23] | Historical BKT | Area under curve | 0.7695 |
P-value | 0.0003 | ||
[25] | Ontology based graph | Average Computation time | 0.07s |
[26] | Formal concept analysis | Learning gain | 10.63 |
[27] | LSTM | Precision | 0.45 |
[31] | Knowledge graph | User satisfaction | 0.9 |
Reference | Methodology | Performance measure | Results |
---|---|---|---|
[13] | CF and jena rules using RDF | Precision | 0.98 |
Recall | 0.90 | ||
RMSE | 0.25 | ||
[22] | Apriori with clustering | Coverage | 60% |
Apriori without clustering | 48% | ||
SPADE with clustering | 56% | ||
SPADE without clustering | 51% | ||
[35] | KNN and naïve bayes | Precision | 0.75 |
Recall | 0.50 | ||
[36] | Hybrid (CF, CB and GSM combined) | precision | 0.35 |
Recall | 0.45 | ||
[38] | Tagging and Apriori | Coverage | 55% |
[39] | CB and CF | Precision | 0.22 |
Recall | 0.39 | ||
[40] | DBN and RBN | RMSE | 0.67 |
[43] | RLNLS | Precision | 0.34 |
Recall | 0.38 | ||
[46] | CF and KNN | MSE | 21.4 |
[47] | Copeland method and Bayesian knowledge tracing | AP correlation (higher is better) | 0.77 |
NDPM (lower is better |
0.28 | ||
[48] | Matrix factorization | RMSE | 0.491 |
Reference | Methodology | Performance measure | Results |
---|---|---|---|
[11] | Association rules mining | Accuracy | 90% |
[49] | KNN and knowledge based filtering | Recall | ---- |
Reference | Methodology | Performance measure | Results |
---|---|---|---|
[50] | Apriori | Analysis of generated rules | ---- |
[51] | FP-growth | Execution time | 34s |
[52] | Reinforcement learning | Average regret comparison | Decrease in regret as rounds increased |
References | Methodology | Performance measure | Results |
---|---|---|---|
[58] | SVM | User satisfaction | Satisfactory performance |
[59] | Collaborative filtering and genetic algorithm | Learner’s score | 90% |
[60] | Convex optimization algorithm | Accuracy | 90% |
Author | Technique | Limitations and future work |
[14] | Their research used collaborative filtering while utilizing Jena rules using resource description framework (RDF). After utilizing the RDF and Jena rules engine, an enriched rating matrix will be generated. KNN is applied to further refine the results to generate final recommendations. The recommendation list is evaluated and then is presented to the user | Their proposed methodology utilized manual method to update RDF factbase and this method can be automized in future. [14] |
[15] | Their proposed model focuses on covering all concepts instead of completing all lessons. Their proposed method also monitors the learner’s progress with the passage to time and if the recommended path is not showing satisfactory results, then the recommender recommends auxiliary LOs. The auxiliary LOs are the highest ranked LOs which has not been seen by the user | Their proposed methodology faced coldstart problem for initial LOs that can be solved in future work by automatically selecting the first LO. [15] |
[11] | Their methodology focused on prompting user to enter information and to also give feedback regarding generated recommendations. Association rules mining is utilized for generating most relevant recommendations. | In future, Swarm intelligence and genetic algorithm can be utilized to improve results [11]. |
[16] | Their research used hybrid methodology using both clustering and graph-based method to design an effective recommender system. This study focusses on LPR using ACO. Fuzzy C-means (FCM) clustering is used to assign learner to a cluster. | The proposed FCM and ACO are computationally intensive. They can be parallelized in future for more effective performance. In future, the same model can be tested for large audience with more diverse demographics [16]. |
[22] | They utilized collaborative filtering along with clustering and association rules mining for better course recommendations. They also utilized sequence mining that helped in recommending courses in correct sequence. For clustering, they used k-means algorithm that uses Euclidean distance to groups similar students together. Aprioir algorithm was used for association rules mining while SPADE algorithm was used for sequence rules mining. If students are predicted to score higher, then the course is recommended to the students otherwise it is excluded from the list of recommendations. | ---- |
[23] | They utilized several techniques like content analysis, assessment and behavioral analysis. Four variations of BKT were used for knowledge, tracing in the proposed model including Coarse BKT, multi-grained BKT, Fine BKT and Historical BKT. | In future, words embeddings can be used to construct knowledge graph instead of word co-occurrence to improve the quality of constructed graphs [23]. |
[25] | This study is based on the concept of self-organizing LOs. The recommendation is based on both user’s explicit requirements and implicit needs that are captured by analyzing user’s behaviour. Recommendations are generated by constructed ontology based graph using user’s profile and similarity between LOs. | The one case study was not enough to test their proposed method. In future more case studies can be conducted to verify the effectiveness of their proposed methodology [25]. |
[26] | An ITS was proposed in this research that aimed to adapt the system according to the user’s requirements based on his or her learning style. A personalized path was generated for each user utilizing the knowledge-based graph using co-occurrence of different objects. | Their proposed model has been tested only on limited set of students. More extensive evaluation can be performed on proposed model in future. |
[27] | Their methodology proposed a separate path when there is no sufficient prior information to avoid cold start problem. The users are also divided into five main clusters based on their interaction with the platform and then learning effect is calculated using LSTM model. | In future, data streams of learners can be utilized to improve the generated learning paths. |
[31] | Their proposed methodology constructed a multi-dimensional knowledge graph that stores class wise hierarchical structure that includes basic knowledge, algorithm and tasks. Their framework inputs a query form user, tokenize it and then collects the target LOs. It is possible for a system to generate multiple paths for one query. In that case the system recommends the path with highest score. | In future this system can be implemented for subjects other than computing and user’s profile can also be considered in further researchers. |
[34] | Their methodology utilized content and collaborative filtering for relevant recommendation for MOOC courses. Their framework picks one of the techniques depending upon a threshold value. If threshold is met then collaborative filtering is used otherwise the algorithm will take the path of content-based filtering. | ---- |
[35] | Their proposed methodology consists of two phases. The first phase uses KNN classifier to determining most relevant courses based on user’s profile and this portion of proposed methodology will be executed for every recommendation. The second phase is optional and is executed only when enough ratings are available in the database for the relevant courses. In second phase, naïve Bayes classifier is used to recommend courses based on user’s rating. | Even though one algorithm showed better precision for course recommendation but still the overall f-measure was not showing satisfactory results. Other hybrid algorithms like association rules mining, deep learning and collaborative filtering can be considered to improve results in future. |
[36] | A hybrid recommender system was proposed by them that included content based collaborative filtering and sequence pattern mining to help in identifying a sequence in which courses should be studied. The proposed framework used both content and context aware course recommendations by utilizing the weblogs, LOs and user’s profile. GSM algorithm was used for identifying the recommended sequence of courses. | Emerging tools of machine learning and artificial intelligence can be utilized in future to generate more effective recommendations [36]. |
[38] | Their proposed methodology utilized tag based Collaborative filtering. In their proposed method, the learners will write tags according to their requirements. | In future, some suggested tags can also be utilized keeping in view the experts’ generated tags to ensure vast coverage of LOs. [38] |
[39] | They proposed MoodelREC teacher centric system that utilized popular LO repositories to crawl LOs. These recommendations can be used to design a new course hence the proposed mode provides an outline that an instructor can utilize to design a new course. | There are no standard LOs and LOs may differ in different LORs. Semantic solutions can be used to solve this problem in future. [39] |
[40] | They combined deep belief network with collaborative filtering for more relevant recommendations. RBM are used for constructing neural networks for unsupervised learning. The unsupervised learning RMB model assigns initial values to the back-propagation model and hence eliminate the drawback of traditional method. | --- |
[43] | They proposed a methodology that is based on RLNLS model. The RLNLS model utilized the learner’s interest and capacity to predict their behaviour in future. Collaborative filtering technique was used to find behaviour of neighbour learners while Hawkes process was used to predict the learning intensity. | In future, feature reduction can be performed to extract data only from relevant variables to maximize efficiency of the algorithm [43]. |
[46] | Their proposed framework combines collaborative filtering with rule-based filtering and KNN classification to create a hybrid system. The recommendation hence generated will be context sensitive. | Their proposed methodology does not consider the authenticity of the user posting course. It can be improved in future by analyzing the profile of instructor. |
[47] | Their proposed EduRank method used collaborative filtering along with social voting to evaluate the relevance of a course. The recommended questions are ranked in descending order based on their Copeland score while item response theory and Bayesian knowledge tracing were used for calculating item difficulty. AP correlation matrix was used to evaluate the proposed method and perform comparison with competing methodologies. | Their framework only considered questions and their solution time for difficulty-based ranking while not adding any solution to identify or help weak students. Work can be done in future to help students in coping with their weaknesses while utilizing their strengths to maximize learning. |
[48] | They used multi-modal matrix factorization for course recommendation in MOOCs platform. The proposed methodology calculates either a course provides particular skill set that a user is looking for or not. The most frequent items in the neighborhood are ranked the highest. | The precision results showed between 1% to 3% accuracy for sparse datasets [48]. The lack of available ratings is affecting the accuracy of their proposed model and it be overcome in future by using hybrid techniques. |
[49] | They proposed a method that focuses on the older audience who are self-learning and do not have access to any recommendations from teacher or instructor. Content based filtering, collaborative filtering and web mining were used to identify appropriate learning material on the internet which was later narrowed down according to personal needs of the learners. Clustering is done using KNN to group similar learners in one cluster to maximize personalization. | --- |
[50] | They proposed Course recommender model that utilized big data to generate association rules in distributed environment. They utilized Hadoop Distributed File System (HDFS) while using Spark for data processing for association rules mining and Sqoop to import and export dataset. Course recommender used Apriori algorithm to generate recommendations of most relevant courses. | For future work, they recommended using deep learning for better utilization of big data in generating course recommendations [50]. |
[51] | The purpose of their study was to discover most relevant course by implementing association rules mining on big data. They used Spark ML libraries to implement FP-Growth algorithm. For course recommendations, they analyzed the behaviour of learners who interacted with system in past by utilizing course enrollment and log data. | In future, big data can be utilized to analyze real time data |
[52] | They utilized the feedback of the users and improved the model by using the concept of regret to improve recommendations for the future. | In future work can be done to handle the dynamically increasing courses dataset. (Hou et al. 2018) |
[53] | They proposed a framework called sharing notes that is a web based forum designed to facilitate students by providing them personalized material. Sharing notes utilized collaborative and content based filtering to rate the material while fuzzy linguistics model was implemented to perform qualitative study on communication between different users. Ontologies were used to map the relationship between users and their preferred material. The final recommendations are generated after obtaining ratings from the trusted users. Sharing notes was tested with the help of 425 users amongst those 88% of the users shows satisfaction with their proposed recommender system. | In future their own specific metrics can be used to perform analysis of social networks, and incorporate them in new recommendation approaches [53]. |
[55] | Their proposed methodology added the layer of semantic analysis along with ratings to improve the performance of generated recommendations. The semantics matrix was separately constructed and was used by utilizing cosine similarity. | In future difficulty level can also be added to generated more relevant recommendations [55]. |
[56] | Their proposed solution performed semantic analysis on messages posted on discussion forums. Text pre-processing was performed only on content logistic related posts and semantic similarity calculations were used to performed semantic analysis. SVM was utilized in the end to generate recommendations for relevant posts. | In future, other classification models like LSTM can be utilized to improve accuracy [56]. |
[58] | The proposed methodology takes the data of online courses as input and rank them into various difficulty levels. Similarly, students create their profile and their data is also extracted through the log files that records the data of a student interacting with the system. | Their proposed model did not consider the rating matrix that can include information about how other users interact with the recommender system. |
[59] | They collected data from the learners profile and used variable length genetic algorithm for learning path recommendations. Collaborative filtering was used to find the popularity matrix which was later utilized by genetic algorithm for personalized course sequencing. | In future work can be done to calculate trust for different users and use this value in collaborative filtering [59]. |
[60] | Their solution considered both formatted data like courses and unformatted data like web sources, documents, papers and publications. They performed clustering on the statistics of data rather than on raw data hence improving the processing speed. | In future, deep learning algorithms can also be explored |
Reference | Year | Accuracy | Personalization | precision | Recall | f-measure | RMSE | Reinforcement learning | Serendipity | Normalized Discounted Cumulative Gain | coverage | Optimal course ranking | Transferability | Less execution time | Improved memory utilization | Query optimization | Learning path recommendation | Effective recommender | Early recommendation | To find association between courses | Difficulty based ranking | Student retention | Improvement in learning |
[14] | 2020 | ✓ | ✓ | ||||||||||||||||||||
[15] | 2020 | ✓ | |||||||||||||||||||||
[11] | 2021 | ✓ | |||||||||||||||||||||
[16] | 2020 | ✓ | |||||||||||||||||||||
[22] | 2019 | ✓ | ✓ | ||||||||||||||||||||
[23] | 2019 | ✓ | |||||||||||||||||||||
[25] | 2018 | ✓ | |||||||||||||||||||||
[26] | 2020 | ✓ | |||||||||||||||||||||
[27] | 2018 | ✓ | ✓ | ✓ | |||||||||||||||||||
[31] | 2020 | ✓ | |||||||||||||||||||||
[34] | 2018 | ✓ | ✓ | ✓ | |||||||||||||||||||
[35] | 2018 | ✓ | ✓ | ||||||||||||||||||||
[36] | 2018 | ✓ | ✓ | ||||||||||||||||||||
[38] | 2018 | ✓ | ✓ | ✓ | |||||||||||||||||||
[39] | 2020 | ✓ | |||||||||||||||||||||
[40] | 2019 | ✓ | ✓ | ✓ | |||||||||||||||||||
[43] | 2018 | ✓ | |||||||||||||||||||||
[46] | 2021 | ✓ | ✓ | ||||||||||||||||||||
[47] | 2019 | ✓ | ✓ | ✓ | |||||||||||||||||||
[48] | 2019 | ✓ | |||||||||||||||||||||
[49] | 2020 | ✓ | |||||||||||||||||||||
[50] | 2018 | ✓ | ✓ | ||||||||||||||||||||
[51] | 2019 | ✓ | |||||||||||||||||||||
[52] | 2018 | ✓ | ✓ | ||||||||||||||||||||
[55] | 2018 | ✓ | ✓ | ✓ | |||||||||||||||||||
[56] | 2021 | ✓ | |||||||||||||||||||||
[59] | 2018 | ✓ | |||||||||||||||||||||
[60] | 2020 | ✓ |
Reference | Dataset | Source/platform |
[14] | Retrived dataset from MOODLE server | MOODLE |
[15] | Mooshak and ENKI | ENKI and Mooshak |
[11] | data is collected from multiple online sources like emails, blog, reports, phone calls and social sites | Scrapping of publically available data |
[16] | Dataset was created | UM Learn (University of Manitoba Learning Management System) Tools: a third-party open-source quiz tool called “Savsoft Quiz v3.0” |
[22] | Harvard and MIT dataset | HarvardX-MITx Person-Course Academic Year 2013 De-Identified dataset |
[23] | MOOCs | Coursera courses from Peking University (2 courses) |
[25] | Own dataset | two courses taught by the authors |
[26] | Own dataset | Own dataset |
[27] | Junyi Academy -- http://www.junyiacademy.org/ a Chinese e-learning website which is established on open-source code released by Khan Academy (https://www.khanacademy.org/) in 2012 | Haw-Shiuan Chang, Hwai-Jung Hsu and Kuan-Ta Chen, "Modeling Exercise Relationships in E-Learning: A Unified Approach," International Conference on Educational Data mining (EDM), 2015 (Link: https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=1198) |
[31] | Crawled data from different educational websites (sources not mentioned) | Own dataset |
[34] | Shanghai Lifelong Learning Network, | Shanghai Lifelong Learning Network, |
[36] | LMS of university (name of university not mentioned) | - |
[38] | An educational dataset, consisting of 120 learners, high school students, participants of the Center for Young Talents project, supported by the Schneider Electric DMS NS company, Novi Sad |
---- |
[39] | (2018). SOFIA, Teachers Formation Platform. http://www.istruzione.it/pdgf/ Last |
Sofia |
[40] | Own dataset | starC MOOC platform of Central China Normal University |
[43] | http://jclass.pte.sh.cn | Mic-video Platform of ECNU |
[46] | BOL repository including records from 2017 to 2018. variables include students’ personal information, learning activity scores, and perception and confidence scores toward each mandatory course |
BINUS online learning (BOL) |
[47] | Koedinger, K.R., Baker, R., Cunningham, K., Skogsholm, A., Leber, B., Stamper, J., 2010. A data repository for the edm community: the pslc datashop. Handbook of educational data mining. pp. 43–55 (https://pslcdatashop.web.cmu.edu/KDDCup) |
KDD cup 2010 and an unpublished data of schools name K12 dataset by authors |
[48] | http://merlot.org | Multimedia Educational Resource for Learning and MACE project (Real world object based access to architecture learning material–the mace experience. In: Proceedings of the World Conference on Educational Multimedia and Hypermedia (EDMEDIA 2010), Toronto, Canada (June 2010) (2010)) and Online Teaching (MERLOT) |
[49] | Own dataset | Own dataset |
[50] | IBM dataset for Apriori algorithm,T10I4D100K (http://fimi.ua.ac.be/data/),T25I10D10K (http://www.philippe-fournier-er.com/spmf/index.php?link=datasets.php)] (generated by a randomiteration of the database), Harvard and MIT published the edX learning data in 2012–2013, HMXPC13_DI_v2_5–14–14.csv (https://dataverse.harvard.edu/dataset.xhtml?persistentId= doi:10.7910/DVN/26147)] |
IBM and edX |
[51] | Moodle. Open-source learning platform|Moodle.org. https ://moodl e.org/. | operational database of ESTenLigne platform that is built on LMS Moodle |
[52] | Created their own dataset | Created their own dataset |
[53] | http:// sharingnotes.ujaen.es/ |
Sharing notes |
[55] | Created their own dataset | Created their own dataset |
[56] | MOOCs | MOOC |
[58] | Source not mentioned | One public one private dataset (source not mentioned) |
[59] | Created their own dataset | Created their own dataset |
[60] | data were gathered from several local Universities (Shandong University of Finance and Economics and Shandong University) |
Local universities dataset |
Reference | Year | Accuracy | precision | Recall | f-measure | Execution time |
[14] | 2020 | 0.98 | 0.90 | 0.93 | ||
[27] | 2018 | 0.45 | ||||
[35] | 2018 | 0.75 | 0.50 | 0.60 | ||
[36] | 2018 | 0.35 | 0.45 | |||
[50] | 2018 | 60s |
Reference | Methodology | Results | Limitations |
[14] | Combined jena rules with collaborative filtering | Showed 0.97 precision for active user and 0.64 precision value for non-active users. | Their proposed methodology utilized manual method to update RDF factbase and this method can be atomized in future. [14] |
[11] | Association rules mining combined with multiple agents | The accuracy of their proposed model initially increased when the number of lectures increased but start to decrease once number of lectures reached 40. Overall the accuracy value stayed above 90% | In future, Swarm intelligence and genetic algorithm can be utilized to improve results [11]. |
[27] | Hybrid methodology including clustering and LSTM | Their proposed model showed up to 45% precision for full course recommendation while up to 75% precision for ten-step recommendation | In future, data streams of learners can be utilized to improve the generated learning paths. |
[52] | Reinforcement learning | The proposed algorithm showed around 0.85 value for reward even for massive dataset. | In future work can be done to handle the dynamically increasing courses dataset. (Hou et al. 2018) |
[56] | Neural networks embedding were to calculate semantic similarity along with SVM. | The accuracy was measured in terms of transferability where world history trained dataset when tested on python course dataset showed 62% accuracy while when model trained on python was tested on world history, it showed 65% accurate results. | In future, other classification models like LSTM can be utilized to improve accuracy [56]. |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 MDPI (Basel, Switzerland) unless otherwise stated