Assessing the Impact of Different Levels of Interactivity on the Effectiveness of Self-Learning

As education becomes more and more important in creating improved societies, many people who do not have access to it are falling behind. To help them catch up, many people, especially those in rural areas and developing countries, are turning to different methods of self-learning, especially those that utilize cheap technology and use interactive methods to teach. Our empirical study tests the effectiveness of an e-learning system that utilizes newer, less tested forms of interactivity and could potentially be used in these areas as a selflearning system and compares it to non-interactive video and textbook selflearning in two different topics. The results of the experiment showed that the increased interactivity provided by the e-learning system achieved significantly better learning performance over both non-interactive video and textbook selflearning. It was also found that students who learned through non-interactive video performed significantly better than those who used textbooks for selflearning.


Introduction & Background
Education has the potential to bring hundreds of millions of people out of poverty, increase individual earnings (and thus boost the economy of an area as a whole), lessen rapid population growth (a major step towards societal development [1]), make children and families healthier, and much more [2,3].
Unfortunately, getting better education to developing countries and rural areas around the world -the places that need higher levels of school enrollment and graduation the most -has proven to be exceedingly difficult, as explained below.Education, for most students, starts in elementary school -but for many families in developing countries, even this first step can be difficult.For example, many schools require students to have certain supplies and uniforms to enter the school, which can be exorbitantly expensive for poor families.If students do not have them, oftentimes, the law dictates that they be refused entry [4].
The school might also be too far away to safely get to, or there might not be public transit to the school.In addition to these barriers, by this age, potential students can often help out with their parents' jobs and make money of their own -and for many poor families, this prospect of extra money now is much more appealing than future returns down the road [4].If the student gets into school, they still have many challenges ahead of them.Government spending per student on learning materials is often very low in these countries -on average, less than $1 USD per year (compared to $100 -$200 in developed countries at the time) [4] -which means that often, schools have very few quality, upto-date educational resources to use while teaching.Many teachers also work in poor teaching environments -in fact, in 29 countries (where information is available, the true number is likely much higher), the average pupil-to-teacher ratio is over 40, and goes over 80 in some of those aforementioned countries [5,6].The UNESCO director-general Irina Bokova has reiterated this point several times, saying that "Access [to education] is not the only crisis -poor quality is holding back learning even for those who make it to school."[7] School teachers are also often not paid very well, forcing many of them to work side jobs to make ends meet; thus, oftentimes, they do not have time to create lesson plans, or sometimes even go to school when they are supposed to be teaching.Additionally, about 25% of school teachers skip teacher training, instead going straight into the workforce, simply because the pay is not high enough to cover the lost time [4].In addition to all of these financial problems, schools often face a significant language barrier between students and teachers in extremely heterogeneous populations, which, along with poverty and poor conditions, is the main driver of school dropouts [4].
All of this has led to over 100 million children worldwide never attending school.Additionally, even students that get to school are much more likely to drop out than not.Students in developing countries have a 40% repetition rate, meaning that a student only statistically has a 40% chance of making it to their next year of school [4].
The Internet, which is full of up-to-date content in all languages, and has something from almost any field one could imagine in any form possible, is a perceived solution to these educational issues; however, many people in developing countries cannot access it.In 2014, 62 countries had less than 20% of their population on the Internet [8,9], and in 2006, over 100 had less than 10% of their population with broadband Internet [10,11] -and unfortunately, none of this is likely to significantly change for decades.

Rural Areas
Outside of the scope of developing countries, there is another large digital divide between rural and urban areas, even in developed countries like the United States.For example, in the United States, 39 percent of Americans in rural areas do not have access to broadband Internet, while in urban areas, only 4 percent do not have broadband Internet access [12].The problem gets far, far worse in US Territories: in rural areas in US Territories, 98 percent of people do not have access to broadband Internet (while in urban areas, 66 percent -a smaller yet uncomfortably high number -do not have access to broadband Internet) [12].Additionally, even in the US, students in rural areas are less likely to go to college (due to societal and financial circumstances, among others), have less opportunities to explore new fields and participate in extracurricular activities, and are less likely to graduate from high school [13,14].These students are not getting the education or access to resources they need, and due to the slow nature of reform, that will not be fixed for decades.[15]) that are now accessible to nearly everyone to run these e-learning systems offline.Interactive e-learning allows for students to be engaged while learning, even without a teacher, and lets them go at their own pace while still being restricted in how they move around the curriculum, similar to a classroom, to make sure they learn everything they need before they move on to more advanced topics.Conversely, students can skip ahead if they have already mastered the basics, giving them more flexibility overall.Additionally, because e-learning can be more personalized, it can cater to individual students who may be at vastly different levels with a subject, and produce improvement in students across the board [16,17,18].

LearnSimply Overview
LearnSimply is an interactive learning system developed in part for this research.This system employs many different layers of interactivity, which will be discussed in the following sections.Wi-Fi hotspot or a separate network; and can run completely offline.In addition to its interactivity, LearnSimply features an incentives system that rewards students with points and medals based on how they do on lessons and how long they stay with the system -however, because of the short length of this study, the incentives system did not play a role in student learning and motivation.

Types of Courses
LearnSimply uses courses -groups of different types of lessons -to modularize the teaching of concepts and keep students on track, similar to how different topics, when taught in a physical school, have separate classes that group similar concepts and focus the student in on specific ideas.There are two different types of courses: • Structured Courses.In a structured course, students must complete lessons in a predefined order, going from lesson A, to B, to C, and so on -skipping is not allowed.This is useful for subjects like programming, where it is important that students do not skip past important concepts.
• Unstructured Courses.In an unstructured course, students do not complete lessons linearly.Instead, the course is broken down into modules -smaller groups of lessons that teach single concepts or focus on specific sub-topics.Although modules have a set lesson order, students can move between modules whenever they want to.This is useful for subjects like language arts, which have groups of topics that are not dependent on each other even though they are in the same year or overarching topic, and can thus be split into modules.
The two courses used in the study were both structured.

Types of Lessons
Courses and modules, as described previously, are made up of different types of lessons, as explained below: • Videos.LearnSimply uses interactive video as its main teaching mechanism.Videos cannot be skipped through, which forces students to watch through entire videos instead of allowing them to skip forward and potentially miss important concepts.Additionally, interactive questions are presented throughout videos in order to keep students engaged, make sure they are understanding the content they are being taught, improve retention, refresh memory, and more (for a full explanation, see Section 2.3).
The video player also has a built-in TryIt Editor (not utilized in study), which can be used for subjects like programming and grammar to allow students to try out the concepts they are learning for themselves.
• Activities.Activities, which are essentially interactive mini-quizzes, make sure students fully understand the concepts taught to them in the previous videos with more complex and in-depth questions than the ones found in videos, along with custom graphics and more answer methods.See   • Final Challenge.At the end of every course, students get to complete a final challenge, which is a large, comprehensive activity that allows students to apply everything they have learned in the course to a real-world scenario.For example, for an HTML course (HTML is a webpage "design" programming language), the final challenge might be to create a webpage about a topic of the student's choice that incorporates all of the different concepts they learned about in the course.
Videos and activities (with a final challenge at the end) make up courses.
The merits of the interactive methods used in each of these types of lessons will be explained in the following section.s is stability of memory, and t is time.Essentially, the forgetting curve is an exponential decay graph curving downward as time goes on and thus information is forgotten [20].Ebbinghaus, using this curve, went on to hypothesize that the best ways to stop this decay of memory (the exponential decay shown by the forgetting curve) were by using better memory representation and repetition based on active recall [20].Memory representation is how a fact is represented in the brain, and devices like mnemonics -or, in the case of LearnSimply, video (which is, in itself, more interactive and expressive than text) with questions and activities -help create a more complex representation of ideas being taught in memory and thus help it stick for longer.Repetition based on active recall -or repetition of concepts while using active recall 3 -has also been proven by several studies to be very effective in consolidating long-term memory [21].It also helps with quicker and more effective information retrieval [22].
The principles of the forgetting curve are also backed up by the spacing effect, which has shown that distributed practice (sporadic practice over a long period of time) produces better long-term recall than learning everything in one time block in over 300 experiments [23].In LearnSimply, the spacing effect is utilized through video questions, which refresh student memory shortly after a concept is taught; activities, which jog student memory a bit later; and final challenges, which make sure that students can apply everything they have learned a long time after learning the basic concepts of their course.

The Testing Effect
Questions are presented in videos and activities as opposed to blocks of text or pre-prepared notes because of the impact of the testing effect, which has shown that questions interspersed throughout (video questions) and at the end of each lesson (activities) allow students to practice retrieval of information, 3 Active recall refers to active stimulation of memory during learning with, for example, interactive questions and activities.which is extremely important for retention.This is much better than giving the student "notes", where both the "question" and answer are right next to each other, which is not as good for retention as testing the student over time [24].
Final challenges, although not questions, also help students practice retrieval of information, which means that they too act in a similar way.In addition to all of this, logically, retrieval of information is what matters in the real world -if one cannot recall learned information, it simply is not useful.

The VARK Modalities
LearnSimply is made to appeal to all types of learners (according to the VARK model [25]) -visual, auditory, reading/writing (from questions, activities, and the final challenge), and kinesthetic (from taking notes and typing answers).Reading textbooks only really works for reading/writing learners, and watching videos only works for auditory and visual learners.Thus, LearnSimply should not only enable more people to more effectively learn and retain the content being taught, but should also help students enjoy learning more, which is yet another important part of retention and learning [26].Even if these learning styles are broad generalizations [27], a system that appeals to more general types of learners will logically be able to connect with more learners than one that appeals to fewer.
The following study tests the above interactive methods utilized in the Learn-Simply e-learning system against other forms of self-learning.

Hypotheses
The following research question guided the creation of this study: does interactive video enhance a learner's understanding compared to passive mediums such as a textbook or video without supplementary content?H1: Given the same amount of time, the improvement levels of those using LearnSimply to learn a topic will be higher than those using textbooks to learn the same topic.
H2: Given the same amount of time, the improvement levels of those using LearnSimply to learn a topic will be higher than those using non-interactive video 4 to learn the same topic.
H3: Given the same amount of time, the improvement levels of those using non-interactive video to learn a topic will be higher than those using textbooks to learn the same topic.

Preparation
Before the day of the study, full courses (with videos, activities, and final challenges) were created for two different topics -economics and the philosophy of religion -to use on LearnSimply to teach the participants of the study.A college-level textbook in each subject was also located for the textbook group, and a website for the non-interactive video group was created with the same videos as LearnSimply (more information on that is in Section 4.2).Additionally, for recruitment, permission forms were given out randomly to students in our classes.If a student returned their form, they were asked to provide information on the days they were available for the study through a survey.30 students turned in their permission forms and were available on the day of the study -and thus, the study ended up having 30 participants.Participants were, as explained later, divided into groups of 10 and rotated once between learning 4 Non-interactive video works like TV, in that it is unpausable and does not contain any elements of interactivity like questions.However, in this case, the non-interactive video is the same video content as LearnSimply.methods, which means that each learning method was tested with 20 people (10 with each of the two topics).

Day of Study
1. ID and Group Assignment.Participants, as they came in, were given a name tag with both a letter and a number -the letter, being A, B, or C, randomly assigned them to a group, and the number, which was a number between 1 and 30, randomly gave them an ID number. 5(There were 10 ID numbers per group, which ensured that people were distributed equally.) Each group started with one of three learning methods: interactive video (LearnSimply), non-interactive video (unpausable video) 6 , or text-based passive learning (textbook).Each group had their own row of tables, and each participant got a computer with their group's learning method (textbook, the non-interactive video hub, or LearnSimply) and a testing portal with all of the tests loaded on it. 7  2. Pre-Test.The participants were all given a 10-minute pre-test to set a baseline for their knowledge in the first topic -in this case, economics.
(See Step 5 for the second topic).

3.
Learning.Participants were then given a short set of on-screen instructions on what to do based on their learning method, and were then given 63 minutes to use whatever learning method they were given.
4. Post-Test.After the learning segment, participants were given a 10minute post-test to quantify how much they learned.
5. Repeat Steps 2-4.After participant groups rotated learning methods, steps 2-4 were repeated with a new topic: in this case, the philosophy of religion. 5The ID numbers were used to randomly identify students from pre to post-test without any connection back to the student. 6Non-interactive video hub available online at https://ls-static.firebaseapp.com.

Data & Observations
Improvement scores from pre to post-test came out as shown in Tables 1, 2, and 3. 8 As shown in Table 1, which looks at improvement in economics, there is a significant difference between average improvement when using different learning methods -an average difference of around 19.4 percentage points between systems (from most to least interactive)!However, unlike in economics, there is almost no difference in improvement between non-interactive video and Learn-Simply in philosophy (Table 2).This seems to be because, firstly, questions about philosophy are easier to get right through pure guessing, and secondly, there were only 9 possible points on the philosophy test, while there were 17 on the economics test -which meant there was far less room for variation in scores on the philosophy test.Combined improvement (   As a way to visualize this data, Figure 3 shows pre-test scores versus improvement in economics for each group, and three "clusters" of points -one for each learning method -can be clearly seen, despite varying pre-test scores.
LearnSimply is on the top, non-interactive video is in the middle, and the text- book is at the bottom.The significance of the trend shown here will be discussed in the next section.

Analysis & Discussion
The economics, philosophy, and combined scores were analyzed using a hypothesis p-value test (Tables 4, 5, and 6 respectively).Table 6 shows that there is a statistically significant difference between learning using a textbook compared to LearnSimply, non-interactive video compared to LearnSimply, and a textbook compared to non-interactive video, and Table 3 shows that these differences are improvements, thus affirming all hypotheses (H1, H2, and H3).
(As shown in  of points and medals, become more important as the system is used for longer.
Third, only around 20 American high school students were used to test each self-learning system.Results should be applicable across populations, but we cannot provide empirical support that they do.Although this study is promising, it does not provide the holy grail for self-learning or education in general.
E-learning may not always be superior to other forms of self-learning, although this study proves that in many instances, it certainly is.

Conclusion
In conclusion, researchers have, in the past, shown that "interactive video" -that is, video with interactive text, random content access, etc. -is more effective for self-learning than non-interactive video, textbooks, and other forms of passive learning.However, very few studies have been conducted on the merits of interactive learning using the technology and methods available today.Our study demonstrates that new forms of interactivity, both in videos with more engaging and interesting graphics along with the more complex interactivity utilized in LearnSimply, offer a significantly more effective learning experience for students, showing that interactive e-learning can truly be a solution for better self-learning in places without well established education systems.

Figure 1 :
Figure 1: Part of a course outline for an unstructured course on literary devices.View focused on the "Irony and Coincidence" module.

Figure 2 :
Figure 2: An example activity from an economics lesson about the Production Possibilities Frontier (PPF).

Figure 1
Figure 1 for an example activity with a custom image and a short-answer question.
This study has several limitations.First, only two topics were covered by the study, and although the improvement due to LearnSimply and non-interactive video compared to textbooks on these vastly different topics is promising, there still may be other topics that are not as well suited for e-learning.Second, each learning period lasted just over an hour, when in reality, a class covering these topics in-depth would last weeks.Further studies are needed to gauge the effectiveness of self-learning methods over these greater periods of time, especially because more interactive features of LearnSimply, such as the incentives system 16 Preprints (www.preprints.org)| NOT PEER-REVIEWED | Posted: 8 January 2019 Preprints (www.preprints.org)| NOT PEER-REVIEWED | Posted: 8 January 2019 doi:10.20944/preprints201901.0056.v1

2 2
With the creation of LearnSimply also came a course creator, which allows people to create fully interactive courses for LearnSimply for offline distribution.It is fully open-source and available at https://git.io/fhttrfor anyone to look at, use, and improve.

8 January 2019 Preprints (www.preprints.org) | NOT PEER-REVIEWED | Posted: 8 January 2019 doi:10.20944/preprints201901.0056.v1
and the lecture -the only place where improvement could be found, both in test scores and satisfaction, was between the three aforementioned systems and interactive video, showing that interactive video is incredibly important in both learning and learning satisfaction[19].Many other studies have also come to the same conclusion -interactive video is one of the best ways of teaching students.However, most of these studies had a much different idea of "interactivity" than what we have today.The University of Maryland study, for example, only showed students an interactive transcript and slide deck, and gave students the ability to pause, replay, and skip through sections of the lecture.This is much different than the type of interactivity used in LearnSimply, as detailed in Section 2.2.2, which is why we decided to run a study similar to the one run by the University of Maryland with LearnSimply to test its effectiveness (Sections 3 -6) -because very few (if any -we have not found any through an extensive search of existing literature) learning systems of this type and depth of interactivity have ever been tested against standard methods of self-learning. is lost over time.It was first created by Hermann Ebbinghaus and can be 8 Preprints (www.

Table 3 )
, as expected, shows the trends from Philosophy and Economics combined, with an ≈ 24 percentage point difference between textbook and non-interactive video improvement, and a smaller (but significant; see Section 6) ≈ 10 percentage point difference in improvement between non-interactive video and LearnSimply.

Table 1 :
Improvement in Economics.Questions on both the pre-and post-test where students did not get enough time to either finish the test or get through the full course during the learning period were dropped from 813 Preprints (www.

Table 2 :
Improvement in Philosophy.

Table 5 ,
Data Row 2, the difference between non-interactive video and LearnSimply in Philosophy was not statistically significant; some potential reasons behind that were detailed in Section 5.)

Table 4 :
Improvement Between Groups in Economics.

Table 5 :
Improvement Between Groups in Philosophy.