Blended Learning and Flipped Classroom in Higher Engineering Education: Students' Perception in the Post-COVID-19 Period

1. Introduction

The COVID-19 pandemic period from 2020-2022 presented significant challenges, but it also brought some positive changes in social activities, particularly in work and education [1,2]. Remote work using the internet became possible and even popular in many technical and technological industries. Higher education institutions had to conduct lectures, classes, and laboratories remotely as well. After the first few weeks of adjustment to the new situation in companies and universities, authorities and administrators were forced to act and change their working and teaching approaches. They also made new work tools available to employees and teachers. Over time, these activities evolved into organized and comprehensive support, including quick courses and training [3,4,5]. This is particularly relevant to the fields of study at technical universities, which are in the focus of research and analysis in this paper. In many cases, these new circumstances were perceived as an opportunity to introduce innovations in teaching and to develop new added values related to the skills of using new software and hardware tools both by students and teachers [6,7,8,9]. The author of this paper has experience in implementing interactive elements as part of classes conducted in the Blended Learning (BL) and Flipped Classroom (FC) formulas, as presented in publications [10,11]. Additionally, in [12] he proposed a modified BL method and described a case study on its introduction in the education of building automation engineers at AGH University of Krakow (AGH University), Poland. A new organizational structure for this method was presented in the paper, along with a discussion of tools for active distance learning that were introduced during the COVID-19 lockdown period.

However, since the pandemic has ended, the external factors that previously determined the organization and conduct of remote classes and lectures have disappeared. As a result, the AGH University authorities have reverted to the traditional model of organizing classes in technical fields. This includes requiring teachers to conduct exercises and laboratories in person, with face-to-face (F2F) interaction with students in classrooms. Nevertheless, it is possible to conduct some classes remotely using e-learning, BL, and FC methods. This is particularly applicable to computer-based classes and laboratories that utilize ICT networks as a tool and element of the educational process. Therefore, it is necessary to redefine modern university education, especially in technical fields, and verify the application possibilities of e-learning methods in the hybrid learning framework. The process of evolution should consider the opportunities presented by university regulations, the availability and familiarity with tools and software that support remote and online work and learning, as well as the preparedness and acceptance of new methods and approaches to the educational process both by students and teachers [13,14,15,16].

This paper presents BL and FC elements that the author has permanently implemented as part of lectures and laboratory classes in the hybrid teaching convention for master's (2nd degree) studies during the last two academic years (2022/2023 and 2023/2024) in the post-COVID-19 period. Moreover, the paper presents the results of surveys conducted among students at the end of the semester of classes in the last two years. The surveys aimed to gather information about students' perception of innovative teaching methods, particularly in terms of diversity and the use of e-learning elements. The following Section 4 discusses the survey results and students' responses, analyzing both positive and negative opinions, as well as pros and cons factors related to the implementation of BL and FC methods. The conclusions presented in this paper are a valuable contribution to the ongoing discussion on the development of hybrid education in higher education [16,17,18,19,20]. It is noteworthy that these conclusions are based on a specific case study from a technical university.

The remainder of this paper is organized as follows: Section 2 provides a discussion of related works, indicating application trends of selected BL and FC methods. Then, Section 3 presents the innovative elements of the BL and FC methods implemented by the author, along with a presentation of selected e-learning and interactivity tools. A discussion of the survey results and student feedback regarding the proposed innovative learning elements is provided in Section 4, and then Section 5 presents the conclusions and future works.

2. State of the Art – Related Works

The idea and development of new educational methods with elements of e-learning had been going on for many years before the COVID-19 pandemic. It was directly related to the development of computer technologies and ICT networks, which became an everyday tool for organizing the scientific and teaching work of university staff. The processes of creating lecture presentations (e.g. Power Point), developing materials and exercise instructions in the form of PDF documents, and finally preparing video tutorials and open educational resources such as Massive Open Online Courses (MOOCs) have become part of the daily work schedule [21,22,23,24,25]. This situation led to the need to systematize teaching methods, which already focused on the synergy of direct F2F work with students and the use of e-learning tools with remote work. Learning Management Systems (LMS) were increasingly implemented at universities, which became a platform for providing learning materials and exercise instructions, interactive contact with students, conducting workshops, group projects, etc. [26,27,28,29]. In addition, BL and FC methods have been developed, in which tools for remote working and electronic sharing of teaching resources and materials have begun to play an important role [30,31,32,33]. It is crucial to emphasize the importance of these teaching methods in technical faculties of higher education, where the use of computers and mobile devices in the work of an engineer has become a significant aspect of both the teaching process and professional practice. Therefore, there is an increasing demand to enhance students' abilities in using various interaction tools and methods for knowledge transfer and verification in an effective and innovative manner [34,35].

2.1. Pandemic Experiments and Experiences

The unprecedented COVID-19 pandemic and lockdown period (2020-2021) determined the need for radical and rapid changes in teaching models. For a period of 2 years, the possibility of direct contact and implementation of all forms of F2F university teaching was essentially eliminated. At the same time, this period was so short that it was impossible to develop systemic solutions imposed by the university authorities. Therefore, university teachers were faced with the necessity of trying to organize distance education essentially on their own, with the support of institutions providing new tools. It was up to the university authorities and technical departments to acquire licenses for software and IT solutions to support the e-learning process. In [5] Alcaraz et al. discuss the implementation of inverted (flipped) lectures and pre-class activities in a digital system learning course. The authors' findings provide valuable insights for improving not only digital systems learning courses. They analyzed the students' opinions after the implementation of the new course format and made several important recommendations. For example, they suggest preparing short, high-definition video tutorials as part of the pre-class activity to match the current way young people perceive knowledge. Video and multimedia are important tools for didactic work in the BL formula, especially FC. One of them is the webinar analyzed by Kumar et al. [36]. The authors point out that a significant component of teaching through webinar lies in the engagement of students throughout the live session and their assessment, and then describe the associated challenges due to the need to maintain the integrity of the learning and assessment processes. In addition, they analyze the effectiveness and usability of various tools and platforms providing webinar options with a discussion of the evaluations of their users - students and teachers.

Another aspect of interactive didactics with elements of e-learning is the use of social media. The authors of the publication [37] suggest that social media can be a very effective tool for building a course community, especially in terms of exchanging official and unofficial information between students. The research conducted by them confirms that the proper use of social media could promote a new era of social learning, social presence, and an alternative platform to foster online learning. One interesting tool to support social learning is the Perusall platform [38]. It is excellent online tool that helps students work through reading and watching assignments together. In Perusall, teachers and students can virtually discuss concepts and issues addressed in digital books, papers, or videos [39,40,41]. In the paper [39], the authors state that it can be utilized in pre-class activities, which can have a more open and interactive format, promoting cooperation and activity among all participants. It is a useful solution for fostering course collaboration and support among students, as well as between students and instructors. When combined with other platforms such as YouTube, Facebook, and Wikipedia, it creates a comprehensive environment for exchanging and acquiring knowledge in a modern format that is open to the younger generation of students [37,42,43]. The papers of this period clearly show an intense search for community building methods and tools that allow a group of students and teachers to interact. The paper [44] focuses on a model to support the process of learning and active knowledge acquisition considering two aspects: (i) the management and assessment of the learning activity as well as the implementation using specifically the collaborative tool Padlet [45] to facilitate student participation, group/team collaboration and feedback. The implementation of similar methods and tools is described in [10,46,47].

A separate issue is that in higher technical education, practical and laboratory classes that require working directly with the equipment, on the workstations, are of particular importance. The lockdown imposed by the pandemic made it essentially impossible to hold classes and laboratory works in this format. As a result, teams of educators took a series of measures and new organizational methods in order to conduct classes and provide students with practical skills as effectively as possible. In [8], the authors describe the activities undertaken to organize teaching in the electrical machine laboratory, pointing to the preparation of short video tutorials and remote visits to the laboratory using cameras. Online meetings with industry experts or virtual visits to industrial facilities were also interesting elements in this approach. However, the latter required additional organizational measures from supervisors and teachers. Another example is the implementation of a laboratory in the field of programming automation systems and controllers. In such applications, the possibilities of virtualization and distance learning are much greater, since in practice students do exercises on computers anyway, possibly using driver modules. In the case described in [3], universal Arduino systems are used, which can also be simulated, e.g. on the Tinkercad Autodesk online platform [48].

It should be emphasized that at the end of the COVID-19 pandemic, many institutions and universities reached a state of developed and standardized working methods, which, until recently, considered new circumstances. At the AGH University of the author of this paper, the teaching staff was primarily provided with licensed access to websites, services and tools enabling the organization of remote work (including Microsoft Office 365 including MS Teams, SharePoint, Google Drive, and others) and effective support of the unit in the field of knowledge about e-learning, remote and hybrid working methods [12]. As a result, many educators have decided to permanently include e-learning elements in teaching processes and as an added value to the education of students in technical fields.

2.2. Post-Pandemic Perspectives, Ideas, Challenges

The post-pandemic period of the last two-three years has been a time of active implementation of new teaching methods in a hybrid mode in technical universities and beyond. It was also a period of research and analysis of the impact of distance learning on cognitive processes and the effectiveness of knowledge acquisition by pupils, students, and teachers. For the purposes of this paper, the review of publications is focused on implementation concepts of e-learning elements, in particular BL and FC methods and their perception by students, effectiveness analyses, trends discussion and development challenges of hybrid education identified by scientists and methodologists.

As a first step, it is worth noting the general summaries in the form of reports on the impact of the pandemic period on various sectors of society, including education, and in particular higher education. The International Association of Universities, in its report [49], analyses the disruption and impact of a number of factors on the functioning of higher education in the immediate aftermath of the COVID-19 pandemic (communication, partnership). At the same time, however, it points the challenges and new opportunities that have arisen as a result of this unprecedented situation, particularly with regard to the organization of distance learning, the increased role of electronic resources and ICT in teaching, and new principles of working in communities and groups. Following this, in 2023, a report [50] was published containing guidelines for the development of modern academic teaching. The report emphasizes the importance of hybrid learning, active use of electronic tools, multimedia formats, and remote communication platforms, enabling work in synchronous and asynchronous modes. It was noted that there is a new trend and opportunities for international cooperation, exchange of knowledge and teaching methods in different countries and cultures. This can be achieved with unprecedented efficiency through communication on the internet and the popularization of remote working tools. However, it was also emphasized that these solutions should be introduced gradually, without making radical, multi-faceted reforms of academic teaching. Several factors determine this approach, which are also identified as challenges on the path to effectively implementing BL elements in higher education. In [51,52] the authors point to several significant challenges: (i) limited facility resources - access to fast Internet connections, the need to maintain high reliability of the power supply network and the university's provision of appropriate server resources and efficient LMS platforms; (ii) lack or limited skills/knowledge in the use of online work tools and the organization of effective lectures and laboratory classes using them, e.g. in the FC formula; and last but not least (iii) activation of students during synchronous online meetings and verification of knowledge and assessment of acquired skills, requiring reorganization of work of teachers and students.

From a practical point of view, the issues of new development trends are indicated in [53]. First of all, the author notes a new trend of treating universities as a service aimed at providing students with knowledge and skills, especially those based on research processes carried out by scientists and aimed at supporting the development of innovation, creativity, and critical and creative thinking. With regard to organizational changes in teaching itself, the author also emphasizes the importance of changes in the mentality of students and teachers in relation to the practical use of online working tools. In this context, she points out that most leaders acknowledged that the pandemic had legitimized online learning and made it more “respectable”. It has highlighted the benefits of online learning and ensured that higher education will become increasingly digital in the future. Learning will be more focused on “stackable segments” and education that is primarily just-in-time, not just-in-case. In addition, the new role of lecturers as "learning facilitators" was emphasized, whose task is to support student groups actively and dynamically in their search for good knowledge and in developing skills in its processing and critical analysis, using both synchronous lecture sessions as well as webinars and other materials. This clearly refers to the transition from the classical F2F formula to a combination of F2F and online knowledge transfer in the BL formula. In terms of practical classes, laboratories - the use of FC techniques, with independent learning, analysis of training materials and guidance before as well as during classes. Furthermore, in [20] Singh J. et al. signal that in BL education, online resources should not be used as a substitute for F2F meetings, but rather to enhance and build the atmosphere and discussion plane in the lecture hall, classroom or in a synchronous online meeting. Work in BL and FC approaches should be organized in such a way that students can take full advantage of the flexibility of the teaching process, the ability to work at any time and at any pace, including discussion and group work, both in F2F meetings and on online communication platforms. This approach will create a new teaching environment, open to innovative solutions and methods of working with students and among students. Some of these methods, both for organizing lectures and tutorials, are discussed in this paper as well. In addition, a SWOT analysis of the development trends in hybrid and BL teaching has been carried out, where the strengths include self-pace, autonomy, self-responsibility, and interaction. Opportunities include creativity, synergy, cloud-based potential, and technological advancement.

Moreover, papers [54,55] discuss possible scenarios for changes and implementation of classes at universities in the BL formula. The authors draw attention to the complementary and expanding possibilities of the role of BL and FC methods. The paper [54] provides suggestions for implementing BL in higher education after the pandemic. These suggestions include introducing institutional guides to determine the percentage of F2F and online meetings, redesigning or modifying curriculum classes, revising university infrastructure, providing training for teachers and students, sharing best practices, and ensuring universal access to tools and resources for both students and teachers. In turn, the paper [55] presents and analyses the results of a survey conducted among 87 academics at 36 institutions in the UK. The survey indicates that most of the lecturers have introduced the BL formula in the delivery of lectures after the pandemic. Additionally, several activities were implemented to support the learning materials presented in lectures including online quizzes and tests, forum and chat discussions, the use of the Padlet tool, engagement checks during and at the end of lectures, and links to external resources such as YouTube and company webinars. Many respondents suggested that lectures should continue to be held online in a synchronous and FC format, while exercises and laboratories should be conducted in-person, F2F with some elements of FC like online guidelines, short video tutorials etc.

The analysis of experiences and lessons learned by university staff during and after the pandemic, as well as students' perceptions of the didactic and organizational changes made, is an important issue for educators and research teams. There are numerous papers and studies on these issues, both general and specific, linked to particular fields of study and disciplines. The geographical location of universities and the associated cultural areas are also important factors. In [56] Benito A. et al. present a comprehensive analysis of the changes that should remain in higher education post COVID-19. The authors collected and analyzed survey results from students and faculty staff at three universities: Istanbul Bilgi University (Turkey), The NorthCap University (the first private engineering college in the State of Haryana, India), and Universidad Latina de Costa Rica (located in San José de Costa Rica). The survey questions addressed the students' experiences related to the transition from face-to-face to online classes, including the format of the classes, the assessment process, and the teachers' engagement. Responses were collected from a total of 2,637 students, including 270 from the NorthCap University, 1,085 from Istanbul Bilgi University, and 1,282 from Universidad Latina de Costa Rica. In turn, the questions posed to faulty staff concerned satisfaction and effectiveness of implementing the F2F mode of student and staff preparation, as well as their engagement. Responses were collected from 66 employees of the NorthCap University, 139 from Istanbul Bilgi University, and 174 from Universidad Latina de Costa Rica (a total of 379 individuals). The paper [56] presents detailed results, but of particular interest are the summary results, presented in Table 1.

It is clearly visible that students prefer the hybrid model, with a significant share of F2F meetings, similar opinions dominate among university staff, but with a visible limitation of the F2F formula. As the authors conclude, an overwhelming majority of them suggest that the digital components of the learning/teaching experience should be enhanced. Moreover, students and faculty staff declare that they keep some of the forms of learning/teaching and interacting they have recently discovered. In particular, students expect greater flexibility in attending lectures physically, as well as the option for digital interaction with faculty and digital access to all learning materials. Given these new challenges, the authors suggest that course designs should shift from a professor-centered approach, where subject expertise plays a predominant role, to a team-based approach that emphasizes flexibility and creativity to provide ever better hybrid learning experiences [56]. Considering aspects related to the implementation of hybrid learning forms, particularly BL and FC, it is essential to emphasize the human factor and the readiness of both teachers and students to use these forms of teaching, which are based on digital and network techniques. In the publications [18,57] the authors distinguish two groups participating in the teaching process: "digital natives", students who naturally use digital devices and tools in learning on a daily basis and are open to new challenges in this area, and the majority of their teachers "digital immigrants" who adhere to traditional teaching methods, and any innovations with elements of digitalization require from them an adaptation procedure, a change in thinking and educational conventions. Therefore, it is inevitable that teachers must change their approach to respond to the expectations and ways of contemporary students. Peimani et al. [18] state that the use of digital technology significantly influences students' perceptions, attitudes, preferences, and expectations regarding higher education providers, and this should be effectively addressed both by lecturers and teachers. In addition, the authors present the results of a survey and analysis of a specific case study at the Welsh School of Architecture (28 respondents, 14 females, 14 males) of the implementation of various teaching elements in the BL formula. The results showed a high percentage (around 79% of satisfied and very satisfied) of student approval of the distance learning elements introduced, in particular synchronous online lectures, with some additional material available asynchronously (e.g. recorded short lecture, webinar). It is important to note that the percentage was even higher for individual forms, namely (i) online delivery of lectures about 82% and (ii) synchronous online thematic discussions about 89%. Students also emphasized that most of them had the appropriate computer equipment, access to the Internet and living conditions to ensure that they could comfortably participate in online meetings. Such results are very promising and indicate the high potential of e-learning and the openness of modern students to this type of knowledge transfer and acquisition.

Interesting observations and comments regarding the practical implementation of FC methods are collected in [58]. Hung L. discusses not only the advantages of the FC approach in organizing classes at universities (more flexible and enabling students to be more self-directed in their learning), but above all points to the problems and inconveniences raised by students related to the teaching practice in this field. method (the heavy workload caused by pre-lesson activities, low levels of self-motivation, insufficient physical interaction, a shortage of relevant and available resources). Importantly, the author also collected a number of suggestions on how to improve the effective implementation of the FC method. In particular, it points to the need to change the role and approach of teachers who, when deciding to organize classes in the FC formula, should change their approach to teaching, becoming guides and facilitators of their students. An important aspect of organizing classes and effective learning is the introduction of group work among students, thanks to which they not only learn, but also acquire social skills, establish relationships and the ability to cooperate with others. Another issue is opening up to feedback as an element of consultation and developing procedures for assessing students' work and knowledge, also with elements of peer assessment [59,60]. These considerations are aptly summarized by Rahmani A. and Samira K. [61] by providing an expansion of one of the elements of the name Flipped Classroom, the word FLIP as an acronym:

F: Flexible Environment: flexibility in terms of assessment as well as selecting the suitable time to learn;

L: Learning Culture Shift: this reflects the change of teachers’ role from being only the source of information to guiding students' learning;

I: Intentional Content: this indicates teachers' knowledge about the actual teaching content provided to learners and students and finally deciding what should be delivered online or in F2F sessions;

P: Professional Educators: teachers are supposed to be creative when designing F2F interaction activities and encourage learner-centered learning to construct new knowledge and understanding among learners.

It should be noted that most of the elements of this development refer to teachers and indicate their leading role and responsibility for the effective implementation and implementation of classes in the FC formula.

Regarding the possibilities and challenges related to the effective use of the BL method in [62], the authors collected the most important issues in a unique SWOC analysis format, in relation to a similar analysis from [20], indicating, among others:

S – Strengths: BL provides students with more learning formats and opportunities, not supported by the traditional lectures (F2F lessons – students interact directly with teachers and peers; online time – students interact with resources and media, external knowledge);

W – Weaknesses: effective use of BL requires the adaptation of a LMS by teachers, students as well as university and moreover additional trainings could be needed;

O – Opportunities: adoption of BL promotes innovations within learning and teaching processes and provides new quality in retaliations between students and teachers as well as students themselves;

C – Challenges: teachers need to develop abilities that allows them to manage time, learning, and communication in an effective way; students should develop self-regulation skills since they need to perform autonomous work and organize their time to be prepared for F2F lessons and classes.

Considering those issues and aspects in the context of conclusions discussed in [17,56] seems clear that classic F2F universities and institutions should enhance the digital component of the teaching and learning process drawing on the experience of the COVID-19 pandemic period. Therefore, universities need to develop a new instructional design, taking students and faculty staff beyond the limited online experience they have had during the pandemic. Moreover, it is reasonable bearing in mind that higher education is now better prepared to accept and include ICT technologies and tools as an essential component of the learning process, that can make it more flexible, attractive, and engaging both for students as well as their teachers. In the last 3 years right after the pandemic universities, technology partners and even the regulatory systems conform a more favorable context to spur innovative approach to education, providing opportunities for broader access to hybrid, more effective teaching, and learning, especially in higher education.

3. Materials and Methods

During the COVID-19 pandemic, the author of this paper introduced many different tools and techniques of assigned work, and with them he developed and implemented several innovative approaches to the implementation of teaching classes, both in the form of lectures and laboratory exercises. On their basis, a modified BL approach was proposed, which, together with the first experiences of its implementation, was described in paper [12]. Using the experience gained during the pandemic and the changes in the organization of teaching mentioned in section 2, which were also introduced at the alma mater AGH University and the Faculty of Electrical Engineering, Automatics, Informatics and Biomedical Engineering, the author decided to continue and further development of innovative teaching paths in a hybrid format, with various interactive elements of BL and FC methods. In particular, these are interactive video materials, tutorials, and webinars, as well as platforms for interactive collaboration of student groups participating in laboratory or design classes, presented in paper [10] at a conference devoted to e-learning in industrial electronics. These methods and tools are now a permanent element of the curriculum and educational path of all subjects taught by the author at the master’s degree studies in Electrical Engineering, specializing in Industrial and Building Automation.

In order to discuss the students' approach and perception of these changes and innovations, in this paper the author selected one of the subjects titled “Building Automation - Implementation in Smart Networks” in which some techniques of the BL and FC methods were introduced in the organization of lectures and laboratory exercises. Subject taught during master's studies (2nd degree). It should be noted that, with the consent of the faculty authorities, lectures for this subject are conducted entirely in the form of online meetings, and laboratories are conducted entirely in stationary form - F2F meetings at laboratory with technical stands [63,64].

3.1. Organizational Changes and Tools

The first and principal rule of hybrid teaching organization is to prepare and provide students with a complete course of the subject on the LMS Moodle platform (named UPEL at the AGH University) and to open a dedicated channel on the MS Teams application for the subject. Both tools are available to all AGH University employees and students who use dedicated accounts in the domain agh.edu.pl. The UPEL course provides electronic (PDF files) instructions for 6 laboratory exercises, most of them with additional information materials (articles, users' guides) or instructional materials in the form of webinars, video tutorials. For all exercises, there are also links to enable students to submit reports in PDF format, with descriptions and laboratory records. A view of aforementioned selected parts of the UPEL course is shown in Figure 1.

Figure 1 also shows a link to the lab class schedule and links to the quiz and colloquium conducted remotely. The mid-semester quiz (colloquium 1) to verify students' knowledge is organized asynchronously, with a specific time slot for students to take the quiz/test. The hours of availability of the quiz for solving are agreed each time with the course participants. Interestingly, students usually request access during the late afternoon and evening. In contrast, colloquium 2 summarizing at the end of the semester is organized in the formula of a traditional F2F conversation and practical verification of knowledge at laboratory stations, which enables verification of engineering skills and theoretical knowledge in application. Each such conversation is conducted with a group of 3-4 students by two laboratory instructors.

Other parts of the UPEL course are the materials related to the lectures. While the lectures themselves are conducted online synchronously on a dedicated MS Teams channel, lecture presentations (slides in PDF format) are made available to students as part of the UPEL course, along with additional teaching materials, partly also discussed during lecture meetings. This block is called "Supporting materials - instead of bibliography" and they include detailed technical information on various building automation technologies discussed during lectures and laboratory exercises. Figure 2 presents parts of the UPEL course described in the paragraph.

All elements of the UPEL course related to lectures have been designed and implemented to provide maximum support to students in the process of independent learning, studying, and exploring issues presented during online lecture meetings. Therefore, in particular, the supporting materials contain numerous indications, markings, and minor comments to guide the learning process. These are tools supporting the BL method. Moreover, it should be emphasized that all lecture meetings conducted online are, with the consent and approval of students, recorded and made available to course participants in video file format within the MS Teams channel and cloud resources related to the dedicated course channel. In this way, students have a full spectrum of sources of substantive and technical information related to the subject program, enabling them to learn effectively, directed by the teacher.

3.2. Innovative Elements of Classes and Lectures

As mentioned earlier, the positive effects of implementing some tools and solutions from BL and FC teaching methods during the COVID-19 pandemic, observed by the author of this paper and described in publications [10,12], prompted their permanent implementation in the educational path. However, in accordance with the principles of the Kaizen method [65,66,67] and the possibilities of its application in teaching, it was decided to implement two innovative approaches and tools.

The first of them are the previously mentioned instructions for laboratory exercises with interactive elements, as an element of the BL method. In practice, for two exercises, a new element of the instructions was introduced - a reference to company technical online tutorials, with instructions on programming selected functions of building automation modules. Moreover, tutorials are provided originally in English, directly from the company’s website. As a result, completing the next steps in the exercises required reading a short instructional video and using it to enter appropriate settings and operating parameters for devices at laboratory stands. The educational goal of this approach is to develop students' skills in critical and creative use of publicly available training materials, technical tutorials, in the implementation of programming procedures and integration of building automation functions. An example of such an instruction is shown in Figure 3.

The second innovative approach is organization of two lectures in the FC formula. For this purpose, fully interactive video materials were prepared using scripts of the H5P tool of the UPEL platform (Moodle). For the first lecture, the author recorded two approximately 40-minute tutorials, each with several interactive links to additional information and a summary with review questions. Selected views from a video tutorial recorded by the author with interactive elements are shown in Figure 4.

For the second lecture, video materials were prepared based on YouTube recordings from the channel of technical seminars organized by the AGH University. As previously, the materials have been supplemented with interactive elements of H5P scripts with notes, explanations, and links to other source materials. Selected views from the video with interactive elements are shown in Figure 5.

All these materials can be viewed by students on their own before lecture meetings organized in the form of an open online problem discussion on the MS Teams channel.

The theoretical and substantive knowledge acquired by students during lectures and laboratory classes is verified by two colloquia conducted during the semester. The first colloquium in the form of a quiz on the UPEL platform is solved by students in a given time in an asynchronous format. The test/quiz uses various question organization formats, with graphic and interactive elements. Example formats are shown in Figure 6.

The second test, more extensive thematically and including verification of practical skills from the laboratories, is carried out at the end of the semester, in the form of an interview and short tasks at laboratory stations. The weight of the assessment of the second test, summarizing the semester, is greater in the algorithm for calculating the final grade for the subject.

All innovative teaching methods, tools and techniques described in this section are the result of their gradual, phased implementation over the last three years. After the first year, still directly related to the period of the COVID-19 pandemic, the author conducted a short survey to obtain students' opinions regarding the innovations introduced. Their results were presented and discussed in two publications [10,12]. Due to the positive results of these opinions and the decisions of the university authorities enabling the implementation of didactics with e-learning tools as well as BL methods, it was decided to continue the changes and innovations also in the following years after the pandemic.

4. Results and Discussion

The continuation of the introduced solutions required an in-depth analysis of the impact of new methods and tools on the teaching process as well as the approach of students to the learning process organized in this way [68]. Therefore, the author decided to conduct extended surveys among students of the previously mentioned subject "Building Automation - Implementation in Smart Networks" after the second and third year of their implementation (two and three years after pandemic). As a result, both after the second and third year of implementation of the BL and FC methods, after completing the entire course in a semester, students received access to a voluntary, anonymous online survey (Google Forms) with six series of single-choice questions. 18 students (out of 20 participants in the classes – 90% of students) took part in the survey after the second year (2022), and 12 students (out of 26 participants in the classes – 46% of students) in the third year (2023).

4.1. Survey Results – Students’ Answers and Assessments

Due to the different number of respondents in 2022 and 2023, the survey results were summarized and compared in percentage terms. The first category of issues that students were asked about was the level of lecturer's involvement and the assessment of his knowledge transfer skills. The obtained results are shown in Table 2.

The positive results of the assessment of the lecturer's involvement in the process of transferring knowledge and the increase in its attractiveness are noteworthy. In both years, 100% of the answers were Excellent and Very good. What is particularly important, as indicated by positive responses, students feel that they are get interested in the subject and the issues presented during lectures. Moreover, from their own experience during the meetings, they assess that the time devoted to them was effectively used by the lecturer.

The decrease in definitely positive responses observed in 2022 in favor of positive and neutral ones in 2023 may indicate a change in students' opinions but may also be the result of a decrease in the percentage of evaluators in relation to all course evaluators. However, the lecturer, the author of this paper, should keep them in mind when planning to update the lecture content and methods of conducting lectures with elements of BL and FC methods, especially to diversify the formula of lecture meetings.

The second category of issues and questions in the student survey concerned the organization of the entire course and elements of innovative learning/teaching methods as well as e-learning in lectures and laboratory classes. The results of this part of the survey, divided into subcategories: (i) organization and form of lectures, (ii) e-learning and FC, (iii) course content and organization (lectures and laboratory), are provided in Table 3, Table 4 and Table 5.

Considering results form Table 3 it is worth noting the very high percentage of Definitely yes responses regarding the issue of flexibility in organizing lecture meetings. These meetings are held online, with several FC lectures and online discussions. Students appreciate the lack of the need to be physically present at a lecture, and the lecturer observes a significantly higher attendance of students than in previous years with the stationary format of lectures (lectures are optional and master's students, often already working). Importantly, their participation in online meetings was active - students asked questions, discussed, and answered short interactive tests and surveys during lectures (tools Slido, Socrative, Poll Everywhere).

The second important issue that was the goal to be achieved by the lecturer was the correlation of lecture content with the technical topics covered during laboratory classes. As indicated by students' evaluations from both years, this goal was rather achieved, although, as for most of the analyzed issues, in 2023 there was a decrease in the percentage of best answers in favor of positive and neutral ones (8% - 1 out of 12 respondents). Moreover, the overall assessment in all analyzed issues is rather positive - no negative answers No and Definitely no. Other aspects regarding students' opinions on the implemented elements of the BL and FC methods are summarized in Table 4. Due to the diversity of the proposed tools and solutions, the survey questions focused on the most significant ones.

The results clearly indicate that the inclusion of elements of e-learning and hybrid learning is generally accepted by students. However, it can be noticed that the representative group of respondents from 2023 mentioned above is not as enthusiastic as the previous one and there are more Yes answers compared to the previous year with Definitely yes answers. Such opinions may also be the result of the fact that the innovative elements proposed by the teacher are not commonly used in other subjects and students approach them with distance, without being convinced of their effectiveness in teaching. At the same time, however, in additional comments (analysis in subsection 4.2) they emphasize that such methods and elements should become the standard of modern didactic implementation paths.

Table 5 presents the results of students' opinions regarding the most important issues regarding the organization of education in the subject as a whole, such as: selected information about the subject syllabus, class schedule and support of the UPEL platform.

Among the issues analyzed in this part of survey, the most important and directly related to the subject of this paper is the aspect of support and organization of the course on the UPEL platform, which is assessed very positively in both analyzed academic years. Other organizational issues are assessed in a diverse manner, clearly dependent on the subjective feelings of each student. Hence, there are single negative opinions, with a clear predominance of general positive or very positive opinions. It should be emphasized that every year the instructor conducts a similar survey and a short discussion with students summarizing the semester of classes and lectures. The feedback is analyzed and in the following year new solutions are proposed or those already implemented are improved.

4.2. Survey Results – Short Summary of Students’ Impressions and Comments

Innovation in teaching is a challenge in itself. It always carries the risk of surprise and inefficiency, and at the same time requires additional involvement of the teacher during the preparation of lectures and classes and, of course, of the students during their implementation. Hence, it is very important to evaluate the effects of innovative approaches in education and to collect and analyze feedback from course participants. Table 6 contains elements of students' opinions and even self-assessments related to their feelings about the impact of the education process in the subject in question on the growth of their knowledge and skills and regarding the requirements placed on them.

The difficulty in responding to the issues presented in Table 6 is the need for students to take a critical, conscious look at their own skills and share information about their change, even in an anonymous format. However, as the results indicate, the respondents rose to the challenge, with the vast majority of them indicating a low and medium level of knowledge before the course and a satisfactory level after its completion. What is also noteworthy is the positive and satisfying feeling regarding the requirements for completing the course with positive results.

An important supplement to the survey was the opportunity for students to write down additional comments, both positive and negative. Over the two years analyzed in this paper, positive comments included, among others (selected ones):

"The diversity of classes. Laboratory classes were conducted using modern technologies, hence the material learned and practiced is up to date (in line with current technological trends) and will be useful in the future. (...) The form of classes differs significantly from standard classes conducted at universities (definitely a plus!)."

"Concise, engaging lectures showing current trends in a given technology and preparing for potential work in the profession."

"During the laboratories, the instructions included some own work, instead of hand-to-hand guidance. Thanks to this, you could do more yourself, instead of mindlessly following the instructions."

"The positive aspects certainly include remote lectures, due to the fact that most of the year I work professionally, which in fact forces me to miss some lectures. The remote form allows you to connect and stay up to date with the material being studied, so the question "what's up" you are actually learning in these studies" is a simple answer. Flipped classroom - in my opinion, it should be a standard in studies (as part of lectures). The lecturer can prepare the substantive material once and for all (...), and the student works on the material on his own so like many courses available on platforms like Udemy[69]."

"Remote, recorded lectures + flipped classroom, structured course structure at UPEL, additional materials."

In turn, the negative comments and remarks included (selected ones):

"Personally, I think that in such subjects the number of laboratory hours should be greater than lecture hours (although I guess the instructor may not have any influence on this). Learning is much more effective when you do something, not just listen about it."

"The instructions on the UPEL platform were sometimes inconsistent with the instructions on the computers at the laboratory stations - later, problems appeared when preparing reports made according to the instructions on the UPEL."

"I would suggest greater intervention in laboratory exercises, where the instructor approaches the station and partially checks the student's preparation, asks why the instructions lead this way, and what if it was done differently. This would certainly encourage more diligent preparation for classes."

"What problems does an integrator encounter on site and how to deal with them? It would be great to see a practical problem and methodology for solving the challenge/fault."

"A lot of reports to complete during the semester (maybe some other form of consolidating knowledge and skills from exercises?)."

Several general conclusions can be drawn from the comments and remarks presented by students. First of all, the 2nd degree master’s students appreciate the opportunity to participate in remote, online lectures, which gives them the opportunity to combine the implementation of the study program with professional work. This formula is also appreciated by the author of this paper - a lecturer, as it ensures greater attendance at lectures. However, as students point out, an important element in limiting such lectures is the variety of methods and tools used to transmit knowledge. Hence their systematic implementation by the lecturer is needed and will be provided in coming years.

Another important issue is innovation in the organization and implementation of laboratory exercises, where students expect interaction, instructions leading to small challenges rather than leading them "by hand", often in the format of "clicking" subsequent settings, algorithms on a computer with integration software. This is quite a challenge from the instructor's point of view, and the answer to it is the implementation of interactive elements in the instructions, the use of video instructions from webinars, etc. Of course, this is just one of the elements of planned organizational changes in work at laboratory stand in the coming years, in accordance with the principle of "small steps".

The last general conclusion concerns the need to maintain the uniformity of the materials provided and made available, both for substantive learning and laboratory exercise instructions. As students' comments indicate, any discrepancies are irritating to them. It should be noted that when switching to the digital form of materials and instructions, the risk of differences in the versions of their files and texts is significant and requires special attention of the teacher conducting classes and lectures.

5. Conclusions

The paper describes experiences resulting from the implementation of innovative elements of BL and FC methods as part of a subject related to building automation systems, carried out at the 2nd degree of master's studies at a technical university, majoring in electrical engineering. In particular, the paper focuses on the presentation of tools and methods introduced into the educational path using e-learning techniques and the analysis of feedback collected from students from two subsequent years of the discussed subject.

The results of students' opinions and assessments presented in the paper indicate that their perception of innovative teaching methods, also with elements of online work (hybrid mode), is generally positive. However, the comments and criticisms discussed in previous section indicate that the effective permanent implementation of such an approach into the teaching process requires special attention and commitment of lecturers and teachers. The selected tools of BL and FC methods should offer diversity and, at the same time, consistency in the transfer of substantive knowledge, closely related to practical skills acquired in the laboratory. Moreover, as students point out, an important issue is to link the topics covered in the subject with current industry trends, technological changes, etc. The latter are relatively difficult due to the need for additional hardware and software investments in the laboratory. Therefore, lecturers and teachers should strive to present universal solutions, the mastery of which by students will allow them to effectively carry out projects and tasks also in technologies other than those they can learn during classes. This is especially important in the dynamically changing market of building automation systems, smart home and building systems, and the Internet of Things. An interesting solution to increase the competences of student engineers may be the introduction of peer work and assessment, or peer mentoring discussed in [70]. However, the results of the analysed case study clearly indicate that students at technical universities are generally favourable towards the implementation of BL and FC methods in the teaching process. At the same time, they are aware that by mastering e-learning tools and remote work rules, they develop additional skills needed in their future professional careers [71,72].

In future works, the author intends to focus primarily on changes in the organization of laboratory exercises, with the implementation of problem tasks to be carried out at stations of various home and building automation technologies. Possibilities of using the FC method are being considered, with the development of materials for students to prepare for laboratory classes independently. Next, based on the laboratory exercise instructions with indication of problem tasks, students will strive to solve problems related to programming functions of automation devices and functional integration of a larger system application on their own and/or with the help of the teacher. Additionally, the author has already started implementing selected elements of peer assessment and is going to include elements of peer mentoring in the teaching process, also with online group work [70,73]. Moreover, a system of surveying and collecting feedback from students will be gradually developed and implemented also in other subjects.