Application of Computer Vision and Python Scripts as Educational Mechatronic Platforms for Engineering Development in STEM Technologies

: This paper presents the process of designing, fabrication, assembling, programming and optimizing a prototype of a nonlinear mechatronic Ball-Plate System (BPS) as a laboratory platform for STEM engineer education. Due to the nonlinearity and complexity of BPS, task presents chal-lenging issues, such as: 1) difficulties in controlling the stabilization of a given position point known as steady state error, 2) position resolution known as specific distance error and 3) adverse environ-mental effects - light shadow error, also discussed in this paper. The laboratory BPS prototype for education was designed, manufactured and installed at the Karlovac University of Applied Sciences at the Department of Mechanical Engineering, Study of mechatronics. The low-cost two degrees BPS system uses a USB HD camera for computer vision as feedback and two DC servomotors as actuators. Due to controlling problems, an advanced block diagram of control system is proposed and discussed. An open-source control system based on Python scripts that allows the use of ready-made functions from the library allows changing the color of the ball and the parameters of the PID controller, thus indirectly simplifying control system and directly the mathematical calculation. The authors will continue their research on this BPS mechatronic platform and control algorithms.


Introduction
Undergraduate STEM engineering students require the practical application of theoretical concepts learned in the classroom in order to master controlling methods and issues. Our goal is to help students learn control theories of systems in an engineering context, through the design and implementation of simple and inexpensive BPS. Students are able to apply tools for computer modeling, control system design and software-hardware implementation in real time while solving the problem of ball position controlling. The entire project development is presented and can be assumed as a guide for replicating results or as a basis for a new approach to the design of mechatronic educational platforms. In both cases, we have at our disposal a tool for implementing and evaluating experimentally controlled strategies that can be further improved in the future. University laboratories and experiments play a very important role in successful education in STEM engineering, especially if the topic is robotics and automatic control applications. The rapid development of BPS applications has recently been noted due to the challenges related to controlling and fast dynamic response that requires short and fast sampling and instantaneous correction of the selected controller. Because controlling fast unstable systems is very important in a variety of practical applications, an educational mechatronic BPS platform can be a successful tool if used in training in robotics and automation control applications and control methods.
Feedback position of a ball is collected by using a camera as shown in [1]. Article is describing a controller synthesis for two-dimensional electromechanical system of the ball and plate intended for a study of the system dynamic and laboratory experiments with different proceeding of the control based on the classical and modern control theory. The system includes the quadratic metal plate, which is movably fixed in the center. Its inclination can be changeable in two rectangular directions. For the inclination of the plate a servo-drive with a controller and two stepping motors have been used. The control problem of the described system is to hold the freely rolling ball in the specific position on the plate. For measuring the ball position, the intelligent video system composed from CCD camera, picture-framing interface and program equipment for real-time picture processing are used.
The BPS was understood also as the two-dimensional movement of the ball and beam system presented in [2]. Author S. Awtar and others presented dynamic characteristics of the BPS, the mathematical model with the corresponding simplified model, and the analysis of the applications of different types of PID controllers. Based on the result of the analysis on different controllers, a controller using a switching mechanism is proposed to control the position of the BPS [3]. Also, F. Zheng in [4] includes hardware design description, sensor and actuator selection, system modeling, parameter identification, controller design and experimental testing.
The authors in [5] proposed a resistive touch screen technique to determine the position of a ball. This could successfully eliminate the lighting effect which can cause an error for camera-dependent control system. For the multivariable and complicated control system of BPS, a touch screen and a rotary pneumatic cylinder are chosen instead of a camera and a step motor, in this work. Simulation results show that it has good dynamic and static characteristics with the proposed control method. Not only fuzzy technique has become a popular choice for the BPS, some works utilized genetic algorithm with neural network or sliding mode controller to solve this nonlinear problem as shown in [6]. The PID neural network (PIDNN) controller based on genetic algorithm (GA) for BPS is proposed in this paper. GA is applied in training weighting factor of multilayered forward neural network, thus the disadvantage of backpropagation (BP) algorithm that easily fall into partial extreme value can be overcome and at the same time, the advantage of PIDNN controller that has simple structure and good dynamic, and static performance that is provided with the simulation results show that the proposed controller has the adaptability, strong robustness and satisfactory control performance in the BPS.
Furthermore, in paper [7] authors Y. Pattanapong and C. Deelertpaiboon proposes a position control technique for BPS using fuzzy logic with adaptive integral control action. The aim is that adaptive integral gain can automatically adjust its value linearly and only active when the ball's position is within the specified distance error. This novel scheme could utilize the ability of integral gain to eliminate steady-state error and adopts fuzzy logic technique because of its simplicity of not having to find mathematical model for this nonlinear system [8]. Current position of the ball is determined by using a webcam positioned right above the plate. Fuzzy controllers as advanced solutions are also described in [9,10]. In papers [11,12] the authors suggested sliding mode techniques (adaptive back stepping control) with the strategy of the fuzzy monitoring. They experimentally founded that adaptive sliding back stepping control is more effectively than the use of conventional SMC because it needs a lot of time to get a favorable tracking accuracy. Furthermore, one paper presents the usage of the neuro-controllers [13] and feedback linearization controllers [14]. Other papers consider advanced Model Predictive Control (MPC) [15].
Some paper presents a virtual and remote laboratory of the ball and plate system [16]. The authors in [17] proposed a control algorithm based on cascade PID and they compared it with another control method. The article shows the results of the accuracy of ball stabilization and influence of applied filter on the signal waveform. The application used to detect the ball position measured by digital camera has been written using a cross platform. Net wrapper to the OpenCV image processing library -EmguCV. The aim of the article [18] is to help students learn the theory of control systems in an engineering context, through the design and implementation of a simple and low-cost ball and plate plant. Students are able to apply mathematical and computational modelling tools, control systems design, and real-time software-hardware implementation while solving a position regulation problem.
Finally, a new method was presented in [27]. Paper reveals an original BPS laboratory process; a simplified process model based on state-space process description. In this work a fast state-space MPC algorithm is discussed. According to authors its main advantage is computational simplicity: the manipulated variables are found on-line using explicit formulas with parameters calculated off-line; no real-time optimization is necessary.
Simplicity in the process of modeling the BPS system as well as avoiding complicated mathematical methods and formulas for stability state analysis led to the design and development of the mechatronic BPS prototype described in this paper. Aiming at achieve low-cost, simply and easy implementation and good controlling precision, this paper proposes computer vision as feedback, Python Open CV script PID controller with adjustable PID parameters to balance the ball for different determine position set points [28].
The contribution of this paper is summarized in several thematic sections: 1. Modeling of BPS prototype based on computer design skills for the purpose of fabrication all robotics and auxiliary parts. 2. Production of all designed parts using 3D printing technology and prefabricated aluminum square tubes to avoid machining of metal parts. 3. Usage of Python OpenCV script with ready-made functions instead of complex mathematical models and settings for a nonlinear system. Through the program code, a control algorithm is proposed and applied in accordance with the simplification of parameter manipulation by introducing the ready-made Python script functions. 4. Introduction of an interactive pop-up window for process manipulation; changing the color and set point of the ball position and controlling the coefficients of PID controller. 5. Discussion related to the proposed improved block diagram scheme.
The article is organized as follows. Section 2 briefly describes the computer design steps and procedures for making laboratory BPS prototype. The section includes the step of design individual robotic parts: servo motor shaft holders, levers and plate joints with as few parts as possible. The Python script algorithm is described in detail in Section 3 with an emphasis on description the ready-made functions involved in generating feedback by converting an image from a USB camera into a set of ball position correction request data. Section 4 deals with visualization and control; the pop-up window software implementation in relation to the HSV standard color palette settings and the PID controller coefficients settings. Section 5 presents an advanced block diagram scheme and discusses the results of experiments in which the proposed scheme is compared with the classical one. Also, part of the paper describes the discussion of the three indicated adverse effects that are perceived as errors in the conversion process and possible ways to eliminate or reduce them. Finally, Section 6 concludes the issues in the article.

BPS computer design and fabrication
This part of the paper discusses the steps of BPS design and production phase. The concept of BPS that was considered, designed and selected for production is actually a replica of similar BPS solutions listed in the works [1, 3, 7, 11, 16-18] basically, but in details similar to [11,27] (Figure 1). For the purpose of computer design, the software solution SolidWorks was accepted, and as printing software Ultimaker Cura. SolidWorks is well known as a software solution for computer-aided design (CAD) and computer-aided engineering (CAE) that is wildly used in all cases of technical and engineering design. Ultimaker Cura is the most popular printing software in the world. But first, the "driving board" for two servomotors or "step motors" was chosen. The logical choice was an Arduino UNO microcontroller board with two matching step actuators.

Arduino UNO driving bord
Arduino UNO microcontroller board is used in this paper exclusively as a Pulse Width Modulation (PWM) driving platform for X-axis and Y-axis DC servomotors. Pulse Width Modulation is a technique which takes a constant steady state DC voltage and produces a train of fixed amplitude ON/OFF pulses whose average DC value is determined by the width or duration of the pulses given by the duty cycle. The current is determined by the impedance of the load being supplied. The Arduino's programming language makes PWM easy to use; by calling function or code analogWrite(pin, dutyCycle), where dutyCycle is a value from 0 to 255, and pin is one of the PWM pins (3, 5, 6, 9, 10, or 11). The analogWrite function provides a simple interface to the hardware PWM, but doesn't provide any control over frequency. It had to be noted that despite the function name, the output is a digital signal, often referred to as a square wave. As we have said before, at the Arduino Uno board, the PWM digital output pins are 3, 5, 6, 9, 10 and 11 as presented in [29]. The frequency of PWM signal on pins 5 and 6 are about 980Hz and on other pins are 490Hz. The PWM pins are labeled with "~" sign and for BPS controlling purpose pins 9 and 11 were selected. Stepper type actuators selected for this purpose was "Tower Pro MG995digi hi-speed" 3 wire, PWM controlled actuators. With operating speed of 0,2sec/60degree under 5 Volt power and metal gear transmission it is a very powerful actuator.

Computer design software
SolidWorks is a software solution for computer design and computer engineering wildly used in all technical and engineering design cases, as mentioned before, and incorporates the Finite Element analysis (FEM) Method. The FEM as computer code method provides a reliable numerical technique for analyzing engineering designs. The design process starts with the creation of a geometric model. Then, the program subdivides the model into small pieces of simple shapes (elements) connected at common points (nodes). Finite element analysis programs look at the model as a network of discrete interconnected elements (mesh model). The Finite Element Method (FEM) predicts the behavior of the designed model by combining the information obtained from all elements making up the model.
Meshing is a crucial step in design analysis. The automatic mesher in the software generates a mesh based on a global element size, tolerance, and local mesh control specifications. Mesh control lets one specify different sizes of elements for components, faces, edges, and vertices.
The software estimates a global element size for the model taking into consideration its volume, surface area, and other geometric details. The size of the generated mesh (number of nodes and elements) depends on the geometry and dimensions of the model, element size, mesh tolerance, mesh control, and contact specifications. In the early stages of design analysis where approximate results may suffice, one can specify a larger element size for a faster solution. For a more accurate solution, a smaller element size may be required. Meshing generates 3D tetrahedral solid elements of one type, unless the mixed mesh type is specified [30].
Cura is 3D slicing software from Ultimaker that prepares a 3D model for printing, suitable for use with students at Upper KS2 and beyond. It is simple but powerful 3D slicing software. The print profiles are optimized for Ultimaker 3D printers, but the software will slice 3D files for any 3D printer brand/model. The software supports STL, 3MF and OBJ 3D file formats and also has a function that will import and convert 2D images (.JPG .PNG .BMP and .GIF) to 3D extruded models. The software will allow one to open and place multiple models on the print bed (each with different slicing settings if required). This allows users to print multiple models at a time, making classroom management of the printing process simpler [31].

Fabrication and mounting
Education BPS prototype because of simplicity consists of dozen parts; such as servomotor arm (  Figure 11b). Furthermore, both servomotor knee-arms also have integrated small magnets and in vertical position supports BPS plate in horizontal position (Figure 11b) firmly embracing the magnetic cups from the bottom of the BPS plate. Details of its construction are also given in student's undergraduate thesis [33].
In the next few images (Figure 2 -9), the design phases of some BPS parts are shown in SolidWorks software as the final files in Ultimaker Cura printing software. Figure 2 shows the first robotic servo-arm of DC servomotor whose purpose is a strong connection to the original output of the DC servomotor shaft at one side (   The crankshaft with the jaws of the second robotic arm of the servomotor is connected to the first servo-handle by inserting the shaft into a small hole ( Figure 4) through both parts. The hole at the left side is a holder for a small magnet.

General BPS design
In order for this paper to have a useful and practical application in student education, and not just a presentation of general theory, this part of the paper describes the problem of computer vision and its application in a mechatronic education. During the realization of the project, i.e. the preparation of the student's practical diploma thesis and later experimentation by the co-authors in this paper, some limitations and imperfections of the prototype and method were noticed, as well as problems in achieving stability of placing the ball in the given position. This paper and project are conceived as a simple and accessible experimental setup for learning, programming and understanding, the issues of feedback management in a real environment.
Mechatronic system described in paper was designed, developed and programmed with help of the student Tomislav Tropčić. The sideway of experimental platform is shown in Figure 13 (top left and right). The system uses a USB camera marked ELP-USBFHD01M fixed 200mm above the controlled platform on the camera holder ( Figure  13  To perform the experiment, three balls of identical diameters but different colors were selected ( Figure 13 bottom and left segment). Black, red and orange 40 mm diameter ball for table tennis were selected in turn. For comparison, a smaller red ball made of silicone mixture of significantly higher mass with a diameter of 20 mm was used. Several materials with the different friction property were used as the basis for the movement of the ball: 3D print material, Plexiglas, white paper and light grey sandpaper (sandpaper 180 particles per inch). The selected materials had different values of roughness and overtime gave unequal resistance during the movement of the test balls. The 150x150 mm white platform is placed on three supports, the middle of which is vertically motionless, and is placed in the geometric center of the square surface of the platform. In order for the platform to be tilted in both horizontal axes, a simple "magnetic" bearing with a ball and a cup on the underside of the platform in the geometric center has been designed. When raising and lowering either of the two vertical robotic arms of the servomotor, the platform actually tilts in firm contact with the robotic handle on one side or the other. Servomotors are mounted with steerable arms with a wedge in the elbow ( Figure 13 above and left) connected to the lower part of the steerable base. They are geometrically at an angle of 90 degrees to each other and the grips are equidistant from the central fixed bearing. The horizontal part of the servomotor handle (first arm) is attached to the servomotor protrusion and the vertical part of the handle (second arm with jaws) has a round metallic ball glued to the top in conjunction with a magnetic cup. Together they form a solid bearing that allows rotation because the cup is fixed in the lower part of the steerable base. With a solid elbow and a shaft with a wedge diameter of 4mm as a dry bearing, a simple robotic lever system has been created by which both servomotors with a rotary angle of ±15 degrees transmit the same angular displacement to the BPS platform.

Computer vision issues
One of the most important issues in the application of computer vision is performance in applications of recognizing patterns, shapes, colors and positions of objects. Given the limited amount of data typical for robotics, the challenges associated with selecting the appropriate substrate, lighting, and methods for assessing image and video quality without reference are important. An important aspect is that the algorithms used concern practical applications, not the derivation of mere theory, although simulations and visualization are important components in the preparatory phase of the scientific setup of an experiment.
Computational vision in mechatronics covers the following unavoidable topics: image formation, camera resolution, advanced image features, real-time sampling frequency, binary vision, optical flow, image filters, object creation, epipolar geometry reconstruction, motion tracking, segmentation, grouping, and recognition object. It is the possibilities of software modeling of image processing techniques and techniques for object localization and geometric measurements that enable advanced research in this scientific field. If the experimental setup uses a standard e.g., ELP-USBFHD01M camera software is becoming a powerful tool for evaluating and implementing image processing functionality.

Image converting technicques
The description of the ready-made functions used in the Python script related to image converting techniques is given in the order in which the image obtained using the USB camera is processed. In order to get more images per second, in the program code the resolution is halved to 640 x 480 pixels, so the number of captured images can be doubled, from 30 to 60 images per second. Ready-made Python image resolution function is defined as: "self.cam_width = 640, self.cam_height = 480". The camera uses a USB connector to power and communicate with the computer.


VideoCapture object -VideoCapture() When launching the application, it is necessary to create an object that will capture a video recorded with a USB camera. The application does not process the stored video (e.g. on the hard disk or memory card) but the stream of data that the camera records in real time (live stream), to download a series of images from the camera (30 images per second), the so-called VideoCapture object. The VideoCapture object only needs to specify the camera number (0=built-in, 1=external USB camera) where the recording comes from. Listing 1 shows a fragment of the code. All other processing (reception, processing and image formation) will be performed autonomously "under the hood" of the ready-made function and thus free the programmer from a big job. In this part of the code, it is necessary to define the dimensions of the images captures by the camera, and it is defined that the image is 480 pixels high and 640 pixels wide.


Color model conversion from RGB to HSV -cvtColor() All colors are obtained by using and combining colors in the color palette. If we use the RGB (R-Red, G-Green, B-Blue) palette then we have 3 basic colors: red, green and blue. If each color is written in 256 shades, then by a combination of available shades we get a palette of 16.7 million colors. Another color representation (or color space) is HSV (H-Hue or Tone, S-Saturation, V-Value or Brightness). The paper uses the HSV palette, so the resulting RGB needs to be converted into HSV space (or model) colors. The switching was carried out in secret because it is easier to get a binary image of the object when it is written in HSV format be divided by subheadings. It should provide a concise and precise description of the experimental results, their interpretation, as well as the experimental conclusions that can be drawn (Listing 2).


Noise image removal -GaussianBlur() The next step is the process of removing noise from each image. The first step is blurring the edges of the image (Blur), using the Gaussian Blur function (blurring is performed using the Gaussian formula). When applied in two dimensions, this formula produces a surface whose contours are concentric circles with a Gaussian distribution from the center point. A matrix of certain dimensions is used to determine the degree of blur -the paper defines that the matrix has dimensions of 11x11 pixels (Listing 2).
 Binary image formation -inRange() The captured image has a certain resolution (640x480 pixels), is converted to HSV color model and noise is removed. It is necessary to translate the image from a colored to a black and white image without shades -where the pixel in the image is colored with either black or white. It is necessary to determine which colors (and there can be 16.7 million in the picture) are converted to black and which to white (Listing 2).
 Binary image noise reduction -erode() The resulting binary image may have certain noises that are usually located at the boundary of the contour of the object (in the binary image). Applying the erode() function of the application will remove certain noise, but the consequence may be a reduction in the contour of the object (Listing 2).
 Contour thickening -dilate() In order to amplify the contour of the object on the binary image, it is necessary to use the dilate() function, which will "thicken" the contour by a certain amount of pixels in order to be more clearly visible (Listing 2). Object localization on a binary image -findContours() After forming the binary image and the object, it is necessary to determine the contours of the object located in the image. The contours are passed to the application as a list of coordinates of the outer points that close the contour. There may be multiple contours in the image (intentionally, by mistake, or so) and then the application will look for the contour that occupies the largest area (Listing 3). Minimal circle within the contour -minEnclosingCircle() After locating the contour of the object, the smallest circle is entered inside it so that the coordinates of the center and the size of the radius of the object can be determined. In this way, the center and edge of the contour on the binary image are determined (Listing 3). The procedure requires that the radius of the contour be a minimum of 10 pixels in length, and after finding the contour, the application displays a circle and its center so that the application user has an idea of where the application has located the center of mass or geometric center of the sphere (Listing 4). After determination, it is necessary to send the coordinates of the center of the contour and the radius according to the function that controls the PID controller self.PID(self.setPointX). After running the script all functions and parameters are prepared to locate and calculate the ball shape and find its geometrical center as start set point (inputX, inputY). The Python script starts the motors and aligns the axes of the platform at the appropriate angles to align the stability with the initial start-position of the ball.

Python control script design
Python's control algorithm requires knowledge of past values. Proportional-integral control, for example, monitors the cumulative sum of differences between a setPoint and a process variable. Because the Python function disappears completely after feedback, the value of the cumulative sum must be stored elsewhere in the code. The problem with coding is figuring out how and where to store this information between call algorithms. For coding reasons, an object generator was created where certain parameter values can be received in parentheses. There are several ways to get value from a so-called number generator. One way is to use the next() function which executes the generator until the next yield expression is encountered and then returns the value.
Python script captures series of camera images at 30 frames per second, approximately every 33 milliseconds, which is the sampling rate of the ball position or the speed of calculating the position correction. Thus, the parameter dT is a time constant that correlates with the image processing speed, which is a PID controller feedback parameter. Listing 5 shows the code fragment where the time variable was defined.

Advanced block diagram of PID controller
During the experiment and the selection of the most suitable color, shape and size of the ball as well as the surface of the plate, it was realized that the block diagram is not as simple as it seemed at first glance. Significant and unavoidable disturbances were observed, i.e., external influences that prevented the stable operation of the mechatronic system and the placement of the ball at a given setPoint. Observed interference functions that cannot be accurately described mathematically, but have been shown to be influential because methods of reducing problems and attempt to eliminate them have led to better results and greater stability. For this reason, an improved block diagram control loop was proposed ( Figure 14) that highlights the locations in the CLC loop and the type of dysfunction or detrimental effect on ball position stabilization. First of all, a dysfunction (accidental disorder) is defined, which is denoted as d1(t), which represents mechanical imperfections and clearances of the handles that contribute to the increase of error.
Furthermore, dysfunction d2(t) describes a group of functions within software that, if inconsistent or unable to perform their task properly, increase position vagueness and introduce uncertainty and directly lead to significant problems and instabilities during position control.
The third influential quantity that contributes the most to the results of the experiment is the amount of scattering or light intensity. The system has been shown to have the greatest stability if light is scattered on the substrate from several sources and the original beam of the lamp is shaded. Each shadow from the light source significantly changes the color shade of the ball on the edge of the ball and changes the contour image, which contributes to poorer recognition of the contour shape and consequently the creation of a binary image. It has been observed that with a single light source although the system has dispersive structure, controller cannot stabilize the ball at all due to the above errors or conversion imperfections. In block diagram view, dysfunction d1(t) has a direct impact on the process (plate position) and form "steady state error". Similarly, dysfunction d2(t) as "internal" uncertainty creates a cumulative effect on the Python output data set (inputX, inputY) before the setPoint calculation process (setPointX, setPointY) and thus forms a "light shadow error".

CLC Error value calculation
The equations built into the Python script make strategic progress, emphasize the ability to create program calculations and manage without real physical hardware (external controller). Error functions for both x and y axes (errorX, errorY) come from the comparison process and are defined in the program code by parameters (Listing 6). The PID control system generally consists of two independent classic PID controllers with one loop. The first of them controls the x position of the ball by manipulating the PWM signal of the first DC servomotor. The second controls the y position using the PWM signal of the second DC servomotor. The PID controller used the same coefficients for both axes assuming the board has two axes uniformity and ideal perpendicularity.
Listing 7: Fragment of the Python code: calculation of both PID outputs for angle correction. # angle variables angleX = self.zero_x + (errorX * self.kP + dErrorX * self.kD + self.kI * self.errorSumX) angleY = self.zero_y + (errorY * self.kP + dErrorY * self.kD + self.kI * self.errorSumY) The coefficients of the PID controller kP, kD and kI were selected in the optimization process and entered into the program as default values, however, they can be changed during the experiment in the control application pop-up window. Proportional kP is used to find out the error between the desired value and actual value and is responsible for the corrective response (Listing 7). Integral kI is applied to calculate all the past values of error and then integrate them to find out the Integral term. When error is expelled from the system this integral stops increasing. The derivative kD is used to predict the expected error values in future based on the present values. Controlling effect can be increased if the system has a rapid rate of change, which is also based on Derivative part. Combining all these three operations gives the total value of the required correction. The constants kP, kI and kD of the PID controller can be entered by the program code but also changed in the graphical visualization window presented in Figure  15.

Visualization and control
In this part of the paper, a discussion focused on visualizing the position of the ball after activating the application and managing the position of the ball. First, during the experiment, it was proved that of the three selected balls, the highest quality conversion to a binary image and the entire image processing covers the case of modified orange color (HSV format parameters -0/77/115/51/253/255) with a slight deviation from the entered "default" value in relation to the value entered in program code (HSV -default 1/77/115/61/153/255). The red color (HSV -default parameters 121/157/86/243/255/255) did not give sufficient response quality, despite of parameter modification, which resulted in an increase in the value of the disturbance function d2 (t) and ultimately too much error and deviation in the calculation, which manifested itself as the possibility of setting the red ball to a given default set point on the platform. The black ball, despite having the strongest color contrast in its parameters (HSV-default 0/0/0/25/25/25), could not be recognized at all as a shape or contour in the HSV standard, probably due to poor lighting quality.

Application "Ball Tracking"
Preprints (www.preprints.org) | NOT PEER-REVIEWED | Posted: 22 December 2021 The interactive "Ball Tracking" pop-up window shown in Figure 15 is designed as the controlling device window of the mechatronic BPS prototype. With the help of designed functions that are performed on the screen, it is possible to control the process with various important parameters. The small white dot, 5 pixels wide, represents the center of mass calculated in Python script as the actual center of the orange ball. Ball tracking window is designed to contain two useful presentations of ball images in the upper right corner, the normal and the calculated binary variant (Figure 15a and Figure 15b). Although the small white dot represents the center of the ball, clicking on a new position on the board on the computer screen, determines the desired position of the ball as a small red dot, also 5 pixels wide. The equations for calculating the PID error values in a Python script automatically calculates the correction value for both axes and thus balance the BPS plate with both actuators ([https://youtu.be/8cQ3oVXJYzA]).
On the left side of Ball Tracking pop-up window are six HSV pallet sliders that allow fine tuning of color shades for the ideal conversion of binary presentation ("Show Thresh"). Figure 15a, in the upper right corner, is a real-time image from a USB camera with an orange ball that showed the best response and presentation of the image live stream during the experiment. Below this section you can see three frames for fine-tuning the controller PID coefficients in steps of 0.001 units or the "Reset PID" option for the default values (stored in script). In the lower left corner are two square buttons for controlling object search (Start/Stop Tracking) and the right button for controlling servomotors (Start/Stop Motors). Furthermore, at the very bottom of the interactive window there is a very useful option to further adjust the horizontality of the plate with respect to the unevenness of the substrate on which the prototype is located. Also, the sampling frequency of the image in millisecond is shown in the upper left corner (insert value 32 in Figure 15a). In Figure 15b, a binary figure of ideal shape shows the same position of the ball. Changing images is allowed by pressing the "Show Thresh"/" Normal View" button. The mechanical "zero position" of the plate, as for the robotic arms of servomotor, is at a default angle of 37-degree for both actuators (Figure 15a and Figure 15b). The "zero position" can be set in 1-degree steps increments within the "Calibrate" X and Y axis frames. Angle control on both axes is limited to ±15 degrees.
In the Python script, two more pop-up screens have been created that indicate the graphical representation of the relevant parameters as well as their numerical matrix representation for future mathematical analyzes. One is 6 seconds time period diagram with a graphical presentation of the actual and selected position set point, and shaft angle value as PID correction response. The second manageable pop-up window in the Python code represents numerical data of parameters in the same time period.

Experiment results
After making the prototype, it was necessary to functionally test the work and optimize all the functions of the Python script with actual components of the BPS prototype. After several iterations, the functional operation of the BPS system was obtained, which enabled the setting of a ball setPoint anywhere of the surface of the plate (150x150mm). But, a "sliding" of the smooth ball on a smooth Plexiglas plate substrate was observed. This is a proven imperfection of the mechanical system and the plate substrate, commonly referred to as "steady-state error" or dysfunction d1(t), as shown in the advanced block diagram ( Figure 14). During experiment typical "overshooting" error values are in domain of ±24 pixels (approx. ±8 mm). Furthermore, during the experiment, a large influence of lighting on the operation of the system was observed. Strong light sources on one side of the ball have been shown to prevent the system from bringing the ball to its desired location. This is a proven feedback imperfection due to an inaccurate image conversion system called "light shadow error" or d2(t) dysfunction in the paper, which is also shown in the advanced block diagram. To counteract this detrimental effect, several smaller discrete shaded light sources were included and then the control accuracy was significantly improved. Figure 17 shows "light shadow error" at the right side of orange ball contour diameter enters physical ball border and binary image is disrupted. Furthermore, a resolution problem was observed, called "specific distance error": the inability of the system to recognize a new set point of the sphere center position near the existing actual set point. A "specific distance error" has been shown to be the length of a red dot radius of 6 pixels (2 mm) that marks the center of mass of a sphere ( Figure 18). Also, Figure 19 shows the "specific distance error" after controller correction process: the setPoint change 12 pixels (4 mm) in the left X-axis direction. Stabilization can be seen after 2 seconds of control and residual dislocation of the actual ball position approximately 2 pixels (0.7 mm) away. Lower graph at Figure 19 shows X-axe servomotor angle correction; after 2 seconds signal noise observed in range ±0,2 degree. To increase the roughness of surface of the plate, sandpaper with a granulation of 180 points was placed. With this background, controlling has been significantly improved. Also, one can see controller order to actuators for correction every 32 milliseconds (lower image on Figure 20). In order to demonstrate the effect of the amount of input light on the feedback efficiency, an additional light source on the other side of the panel was switched on in the third second of the experiment. Improved lighting led to the instantaneous stabilization of the ball as shown in Figure 20. Also, in the first three seconds there was a delay in response or evident sluggishness of the ball, which is not the focus of this paper and which will certainly be taken into account in future scientific research.

Conclusions
This paper discusses the application of the BPS prototype as a laboratory platform for the education of STEM engineers. Moreover, details on the design and implementation of software and hardware it is also discussed. An open-source control system based on Python scripts that allows the use of ready-made functions from the library is characterized by a very short computation time. Simplicity of the calculation is possible due to the OpenCV environment. When applied to the BPS process, the OpenCV algorithm was considered to work, but compared to other papers it is necessary to improve the system to eliminate or partially reduce the impact of size disturbances indicated as errors in the improved block diagram.
Therefore, the authors hope that the presented works may be inspiring for the readers, students, leading them to further development of new methods and applications of machine vision and computer vision methods for industrial and non-industrial purposes, because the authors will certainly continue their research on this BPS mechatronic platform and control algorithms.  Data Availability Statement: The data presented in this study are available on request from the corresponding author.