A comparative study of virtual and insite engineering service-learning implementations

The Smile project is an engineering service-learning initiative carried out through collaboration between Chouaib Doukkali University in Morocco and Pusan National University in South Korea. Since 2016, this project has been conducted annually for engineering students from both universities. Participants are selected through an oral interview, ensuring representation from different majors, years


INTRODUCTION
Service-learning is an educational approach that combines learning objectives with community service in order to provide a pragmatic, progressive learning experience while meeting social needs.In the realm of engineering, service-learning can be likened to a design challenge.It involves seamlessly integrating the subject matter of a credit-bearing educational experience with an organized service activity that addresses specific community needs [1].Through this engagement, students are encouraged to reflect on their service experiences, gaining deeper insights into the course content, developing a broader appreciation for the discipline, and fostering a heightened sense of civic responsibility [2].
Either by choice or by obligation, engineering service learning can be carried out at a distance.In this case, we speak of virtual engineering service-learning or what we call E-service-learning in engineering.This working mode aims to achieve the same objectives as those of the face-to-face mode.
We understand virtual service-learning as a distinct approach where learning and service intertwine within the cyberspace.It represents a merged teaching-research initiative that integrates community service with academic learning through a concise online project.This empowers educators to impart knowledge with social significance while enabling students to build upon experiential learning by addressing real-life J Edu & Learn ISSN: 2089-9823  A comparative study of virtual and insite engineering service-learning implementations (Tarek Riaji) 361 challenges.From this standpoint, virtual service-learning unites research, teaching, and the dissemination of socially valuable knowledge, fostering a cohesive connection among the university's three pillars or missions and directing the institution's efforts towards societal good [3].Smile project is one component of the comprehensive beyond engineering education (BEE) program, which endeavors to leverage engineering for impactful service.Within the BEE program, various projects exist, including creativity station, Smile project, BEE internship, BEE happy project, engineering service committee, dynamic island, and slice project.All these projects share a common educational platform that embraces the engineering service-learning model, centered on the design process to address real-world issues within the project BEE community.As per the vision outlined in project BEE's 2020 strategy, the international committee takes on the role of event organizers from the community service hub center [4], [5].
In the realm of service learning, extensive research has been conducted, with a predominant focus on investigating students' adaptation and alignment within the new online learning paradigm.These studies predominantly delve into the extent to which students successfully acclimatize to virtual service-learning and whether this altered mode of education hinders their positive contributions to society [29]- [32].However, our research extends beyond this level of analysis, delving into a deeper dimension.Specifically, our study transcends the conventional boundaries by assessing the impact of both face-to-face and online engineering service-learning environments on students' competencies and capacities.We aim to achieve this by conducting a comprehensive comparison of outcomes between the two distinct modes of learning.By doing so, our study aspires to shed light on the nuanced effects of the learning environment on students' abilities, leading to a more holistic understanding of the role and potential of service learning in shaping students' experiences and contributions.

RESEARCH METHOD
As professors and managers of Smile project, our study is based on a qualitative methodology in which we adopted questionnaires, interviews, observations, focus groups, and documents.We will take the last version in the insite mode which was organized in 2019 on the one hand, and the version of 2021 in online mode on the other hand.The research employs a mixed-method case study design, aiming to gain a deep understanding of the "Smile" project's transition to online mode and its impact on engineering students' competencies and skills.The study takes place at three Moroccan universities: Chouaib Doukkali University of El Jadida, Mohamed VI Polytechnic University of Benguerir, and Ibn Tofail University of Kenitra, as well as the National University of Pusan in South Korea.The participants consist of 25 students, divided into 5 teams, with 5 students in each team as shown in Table 1.
Data collection involves the use of online pre-and post-surveys conducted through Google Meet.The surveys measure the participants' competencies and skills, with additional questions regarding the criteria and conditions of online mode participation.The pre-survey data are analyzed using SPSS version 28.0.1 (IBM, Chicago, IL) to compare the results with those obtained in the face-to-face version of the project [33].
Furthermore, qualitative data is collected through participant observation.The researcher takes on the role of a manager, engaging in the online activities and dialogues with the participants during the implementation of the Smile project.The observation process occurs throughout the entire process of the project's online implementation, covering stages such as discussing and modeling the strategy, guided practice, and individual practice.Multiple observations over time aim to gain a comprehensive understanding of the participants' experiences in the online mode.Overall, this mixed-method approach allows for a thorough exploration of the "Smile" project's transition to online mode and its impact on engineering students' competencies and skills.The combination of pre-and post-surveys and participant observation provides a holistic view of the phenomenon under investigation, offering valuable insights for further research and program development.

RESULTS AND DISCUSSION
Smile project in online mode has the same objectives and follows the same general process of the insite mode, except some modifications at the level of different stages so that it can adapt to the new working conditions.At the treated issues level: the lecturer is responsible for seeking out issues of interest to the local community and writes detailed reports with photos and videos about the local community and its activities.At the meetings level: participants need to discuss about problem solving using the online platform (Zoom, Google Meet, WhatsApp, Kakao Talk, or Microsoft Teams) as swhon in Figure 1 with team members and mentors.In addition to groups on Facebook, WhatsApp and Kakao Talk are open to facilitate communication in case of need for help and guidance.At the state of progress level: participants are asked to fill out reports with photos and/or videos for each step completed, and upload them to a dedicated online platform, respecting the specified deadlines.First, we will study the impact of this approach in face-to-face mode (2019 version) on the participating engineering students, through the comparison of the different skills of engineering students before and after the project, then we will compare their results with the results obtained in online mode (2021 version).A comparative study of virtual and insite engineering service-learning implementations (Tarek Riaji) 363

Statistical analysis
According to Table 1, we note that the distribution of the teams is very balanced, both in terms of gender, age, and nationalities, and the engineering students' majors are diverse.As in the face-to-face mode, this distribution will help them to complement each other and create an environment of competition and enthusiasm between the teams.Using Google meet, online pre-and post-surveys for all engineering students were conducted to measure the same competencies and skills of the participants as in the insite version of the project, with a few additional questions about the criteria and conditions of working in online mode.The data obtained through pre-survey were processed by SPSS version 28.0.1 (IBM, Chicago, IL) for analysis.The results were then compared with their counterparts in face-to-face mode as shown in Tables 2, 3 and  Figure 2.
Both samples have approximately the same values, where the range in all targeted skills varies from 0% to a maximum of 2%, with a total average range of 0.3%, considered as negligible.The range is the difference between the largest and the smallest values.Thus, the impact of the initial state of competencies and skills of the two groups on post-project outcomes is negligible.Figure 2 illustrates the degree of similarity between the two study groups across both modes of the Smile project.This visual representation provides insights into the level of equivalence observed within the project's different modes for the two study groups.
As Smile project in 2019, we made a comparison between the results obtained in pre-survey and those obtained in post-survey.Table 3 shows the outcomes of this comparison.The results from pre-survey demonstrate that participating students possess at least the minimum required skills and competencies, with all targeted competencies being no lower than 63%.This suggests that participants are carefully selected based on specific skills and abilities.On the other hand, Group 2 consistently exhibits higher results for each competency compared to Group 1, with all target skills not falling below 76%, indicating a positive impact of the project on the participants.The degree of achievement in all targeted competencies is consistently greater than or equal to 13%, and the average degree of achievement for all skills and competencies is 18%, indicating a significant and positive evolution in the participants' capabilities [34].Skills and competencies have been the central focus of extensive academic research, demonstrating their positive influence on service-learning educational outcomes [35].

Post-project outcomes comparison of the two modes
In order to verify if there is a statistically significant difference between the mean samples' values of the two Smile project modes, we chose the independent samples t-test as hown in Tables 4 to 6, after ensuring that the proposed data processing is subject to all conditions of this test.It has almost the same objectives as paired samples t-test, except that it compares two unpaired samples.It should also be noted that this test does not require the same samples size.
An independent samples t-test is used to compare the means of two groups.The data for both groups are interval in nature as shown in Table 4.While there is no strict assumption of normal distribution (if one or both groups have highly unusual distributions, the t-test may not yield accurate results, especially with unequal sample sizes), it is assumed that the two groups have equal standard deviations.If the sample sizes are equal or very similar, the assumption of equal standard deviations is not critical [36].
The group 1 (face-to-face mode) had higher values for almost all the covered skills and competencies which range from M = 3.7813 to M = 4.6250, and SD = 0.35197 to SD = 0.84976, except for the ability to use engineering tools, in which we mark a higher value in group 2 (online mode) (M = 4.1250, SD = 0.70308) with a range from M = 3.6250 to M = 4.2500 and SD = 0.70308 to SD = 0.85502 as shown in Table 4. Levene's equality of variances test shows that equality of variance can be assumed for all competencies, except the interpersonal skill.This results in significant values between 12.8% and 56.8%, which are above the defined significance level of 5% as shown in Table 5. Levene's Test is therefore not significant, and the null hypothesis is confirmed.Thus, there is equality of variance in the samples.The Levene's Test for interpersonal skill obtained a value of 3.7%, which is below the defined significance level of 5%.As a result, we reject the null hypothesis, confirming the alternative hypothesis.Therefore, there is an inequality of variance in the samples for this skill.The results of the two-tailed t-test for independent samples (with equal variances assumed) indicated that the difference between group 1 and group 2 concerning the dependent variables of all skills and competencies was not statistically significant, -0.986 ≤ t ≤ 0.342, 0.086 ≤ p ≤ 0.736, 95% confidence interval [-1.07588 to -0.8732, 0.38838 to 1.26939] as shown in Table 6.Thus, the null hypothesis is retained.To therefore tell how strong the difference is between the two groups, we adopted the effect size t-test for Independent Samples, and its results are shown in Table 5. Due to the small size of the two samples and their difference (Group 1 = 10, Group 2 = 25), the Hedges' g measure is consistent with our case.The effect size values obtained varies between 0.70157 and 0.79724, so all are comprised between 0.5 and 0.8.The results therefore have a medium to large effect.

Qualitative results
In our study on engineering service-learning, qualitative analysis played a crucial role.We utilized it to acquire deeper insights, complementing the survey results with data gathered from interviews, observations, focus groups, and documents.This comprehensive approach enabled us to thoroughly assess the impact and effectiveness of various modes of service-learning in engineering education.In addition to descriptive statistics, the graph in Figure 3 shows the percentage increase of skills achievement in both project modes.Ability to utilize engineering major: even though there is a high development in favor of insite mode, the application of engineering majors in online mode is no less important than in insite mode.Due to the variation of this competence according to the issue treated and the solutions proposed which requires specific specialties that may not correspond to the participants' majors [37].Ability to solve problems: the treatment of a problem in face-to-face mode, gives a concrete aspect of this issue, as well as the possibility of testing each time the proposed solutions and of rectifying the errors, which develops better the competence of problems resolution [18].
Engineering tools usage capability: the use of online engineering tools does not imply a real practice of this skill, especially at the design process level.It facilitates the acquiring of this skill, especially with the use of technological tools, but is not very effective in terms of practical application.Interpersonal skills: although the online mode is an available alternative, the face-to-face mode remains the most relevant environment to practice and develop interpersonal skills that require physical contact with others.
Communication skills: a homogeneous team is necessarily a team that achieves effective communication between its members, in terms of quantity and quality.The quantitative side is strongly present in the online mode, which ensures flexibility in terms of time and space, while the qualitative side is A comparative study of virtual and insite engineering service-learning implementations (Tarek Riaji) 367 more present in the insite mode, which offers a transfer of emotions and feelings through body language.
According to the results of both groups, the qualitative approach has a greater impact on communication skills than the quantitative approach [38].Ability to think synthetically and Self-management capability: these two skills obtained almost the same increase in achievement in both project modes, which shows that this variable does not influence their evolution.Global capability: direct physical communication helps to better normalize the overall relationship between the different students' nationalities and create a smooth atmosphere of cultural harmony [39].In this paper, we attempted to answer the following questions: do projects involving the engineering servicelearning approach have a significant and sustainable impact on engineering students' skills and competencies?at what level can the online mode of such projects replace the insite mode? and what is the difference between them?
The online mode is considered as an alternative mode available in case the project cannot be conducted in the field due to the COVID-19 pandemic for example [40].By following the same design process of the insite mode and with some adaptive modifications for this type of work, we conducted a second study of the project in its 2021 online version, in order to evaluate their results and compare them to the face-to-face outcomes mode.The pre-survey results show a very small and insignificant difference compared to the face-to-face mode, so we consider them of negligible impact.Consequently, the initial skills and competencies level of participants is identical in both modes of the project under study.Although the average values of skills and competencies showed progress in favor of the face-to-face mode, except for the ability to use engineering tools, where the most significant progress was for the online mode, these disparities were insufficient to cause a significant difference.Therefore, we cannot give absolute preference to one mode at the expense of the other.However, due to the small sample size of this study, additional research is required in order to document statistically significant results.
This analysis provided further insight into the impact of Smile project in insite mode and online mode in helping students develop their skills and competencies as civic-minded and socially responsible members of society.The quantitative and qualitative results suggest that for the participants, the objectives of engineering service-learning, which go well beyond competencies and skills to include citizenship and social responsibility, were met.If academics and practitioners are encouraged to fully understand and document the value of service-learning as a pedagogical model in engineering education and as a model for developing skills, competencies, and socially responsible in all disciplines, it will be a strong impetus in the development of engineering education, and therefore the graduation of highly qualified students from engineering schools [41], [42].

LIMITATIONS
The international nature of Smile project work tends to attract students who are willing to participate in the project, and participants are selected through an oral interview to ensure that they possess a minimum of skills and abilities required.The latter may represent great viability for the evolution of learning.Additionally, it is important to acknowledge the possibility of self-selection bias among individuals who may hold a positive inclination towards the Smile project.Furthermore, it is worth noting that confirmatory factor analysis was not conducted in our survey, primarily due to the relatively small team sizes in relation to the number of data points sought.Our survey responses yielded a modest sample size of 10 participants for the insite version and 25 for the online version, which warrants careful consideration while interpreting the findings of this study.
The method that has been adopted to evaluate the different competencies of the participants remains the self-appraisals (by survey), which does not give precise results.The students' self-appraisals may not correspond well to the real performances.Note that this is sometimes referred to as the Calibration Error, which is the difference between values indicated by an instrument, the survey in our case, and the actual values.It has been famously repeated in multiple situations, including the Kruger-Dunning effect.In psychology, this cognitive bias manifests as individuals with limited knowledge or competence in a particular intellectual or social domain tend to significantly overestimate their own knowledge or competence within that domain.This overestimation occurs in comparison to objective criteria or the actual performance of their peers or people in general [43], [44].
In addition, to ensure a more robust and pertinent comparison between insite and online modes and obtain reliable results, it is essential to maintain the stability of variables such as participating members, issues addressed, participants' initial skills and competencies, majors, duration, and other relevant factors.This approach allows us to focus solely on the target variable, which is the mode of work.Finally, let's remind that this is a very small-scale, practitioner-driven study with limited access to time, funds, and staffing resources.As such, this does not seek to deliver generalizable knowledge but rather aims to offer a perspective that is yet to be broached.It is hoped that the methodology of engineering service-learning will

CONCLUSION
The study explores the "engineering service-learning" educational approach, which has the potential to enhance the diverse competencies and skills of engineering students in conjunction with traditional educational methods.In universities, engineering students typically engage in time and location-restricted theoretical and practical lessons.However, the engineering service-learning approach empowers engineering students by offering them opportunities and suitable conditions to apply and consolidate their knowledge gained during their engineering education.This not only benefits the students but also has a positive impact on the local community, providing sustainable and cost-effective solutions for their challenges and fostering the development of their activities and projects.Additionally, this approach allows professors to impart genuine significance to their courses while taking on a guiding and mentoring role.
Applying this approach online is very possible and useful as well, but it yields fewer results compared to the insite mode.It also provides the possibility of benefiting a large number of engineering students, and the possibility of working on a variety of issues for different local communities and with limited financial capabilities, in contrast to the face-to-face mode.In online mode, the fact that the proposed solutions are not realistically installed renders them unsubjects to experimentation and testing, and hence the difficulty in determining their effectiveness and problem-solving ability.Additionally, the absence of the actual construction and installation of the project proposed by the students, and remaining at the prototype stage, means that the local communities have benefited in a very limited way and sometimes do not benefit.In conclusion, engineering service-learning stands as an effective educational approach that can be tailored to suit diverse engineering disciplines, making it applicable in both face-to-face mode under favorable conditions and online mode when face-to-face is not feasible.Furthermore, its scalability allows for improvements by incorporating, removing, or modifying stages and methods, ultimately leading to enhanced outcomes.

Figure 1 .
Figure 1.Team meeting on Zoom platform as part of Smile project 2021

Figure 2 .
Figure 2. Comparison index of the students' skills initial state in the two modes of Smile project

Figure 3 .
Figure 3. Skills and competencies development levels of student engineers participating in the two Smile project modes


ISSN: 2089-9823 J Edu & Learn, Vol. 18, No. 2, May 2024: 360-370 368 be extended by researchers, professors, and evaluators in different contexts to build on the participants' experiences in this project.

Table 2 .
Comparison between the initial state of various competencies measures in insite and online modes

Table 3 .
Descriptive statistics of competencies measures in the Smile project 2021

Table 4 .
Group statistics

Table 5 .
Independent sample test

Table 6 .
Independent samples effect sizes The denominator used in estimating the effect sizes: Cohen's d uses the pooled standard deviation; Hedges' correction uses the pooled standard deviation, plus a correction factor; and Glass's delta uses the sample standard deviation of the control group.