Construct Validity of a Scale to Measure the Job Satisfaction of Professors at Public Universities in Central Mexico during COVID-19*

Validez de constructo de una escala para medir la satisfacción laboral de profesores de una universidad pública en México durante la COVID-19

Received: December 12, 2020
Accepted: April 4, 2021

How to cite / Cómo referenciar
García-Lirios, C. (2021). Construct Validity of a Scale to Measure the Job Satisfaction of Professors at Public Universities in Central Mexico during COVID-19. Trilogía Ciencia Tecnología Sociedad, v. 13, n. 25, e1826. https://doi.org/10.22430/21457778.1826

 

Abstract

Until May 2021, six and a half million deaths had been directly and indirectly associated with COVID-19. This pandemic has impacted the traditional classroom by transferring it to online environments. The objective of this study was to establish the construct validity of an instrument that measures professors’ job satisfaction. A cross-sectional psychometric study was carried out with a selection of 100 professors from public universities in central Mexico. Three main factors were found to explain 56% of the total variance: information need, perceived usefulness, and technology adoption. Although the research design limited the results to the study setting, suggesting the extension of the work towards validity and convergent of the construct.

Keywords: Construct validity, COVID-19, reliability, satisfaction scale.

Resumen

Hasta mayo de 2021 se reportaron seis millones y medio de muertes relacionadas directa e indirectamente con la COVID-19. Esta pandemia ha impactado el salón de clase tradicional, transfiriéndolo a ambientes virtuales. El objetivo de este trabajo es establecer la validez de constructo de un instrumento que mide la satisfacción laboral. Se realizó un estudio transversal psicométrico con un grupo seleccionado de 100 profesores de una universidad pública en México. Se encontró que tres factores principales explican 56% de la varianza total: necesidad de información, utilidad percibida y adopción de tecnológica. Aunque el diseño de esta investigación puede haber limitado los resultados a este escenario específico, estudios futuros podrían mejorar la validez y convergencia de los constructos.

Palabras clave: validez de constructo, COVID-19, fiabilidad, escala de satisfacción.

INTRODUCTION

Until May 2021, the SARS CoV-2 pandemic and the COVID-19 disease had claimed the lives of 3.5 million people worldwide. Adding the deaths from atypical pneumonia indirectly related to the health crisis, that statistic could even reach 6.5 million. These figures can be calculated from the difference between the average annual number of expected deaths and the total number of deaths reported since the emergence of the SARS CoV-2 (World Health Organization, 2021). In Mexico, 250,000 deaths have been reported. That number, added to the 450,000 deaths in excess of the annual average, amounts to a total of 700,000 victims, and more cases are being reported at a mortality rate of more than 2% (Pan American Health Organization, 2021).

Against this background, the impact of the pandemic on the economy, mainly on employment, has been significant in terms of entrepreneurship indices. Entrepreneurship, an indicator of the gross domestic product (GDP), attracts international investment and facilitates the economic rebound of different regions and localities. In the last 20 years, the United States has been number one in this regard, followed by the People's Republic of China. However, the advent of this pandemic changed the course of history. Studies on entrepreneurship worldwide have focused on the impact of the pandemic on micro-businesses (Organisation for Economic Co-operation and Development [OECD], 2021) and found that the most dynamic regions in the world should expect an entrepreneurial redound in accordance with the impact of the pandemic on their local economy.

The Asian region, led by the Chinese economy, projects a growth rate higher than all other economic regions for 2020 (OECD, 2021). In contrast, the eurozone, the most affected by the pandemic, should not expect a rebound of its economy until 2023. A country-level analysis shows that the asymmetry between said the regions is substantially similar. For instance, the entrepreneurial activity in South Korea suffered less from the impact of the pandemic than in European countries, which experienced the most negative effects and whose economic activity has not yet been reactivated or picked up.

In the Asian region, where COVID-19 has had a weaker effect, 21% recognize significant, moderate, and opportunistic effects (OECD, 2021). An instrument applied in 18 countries of the Asian continent measured six dimensions regarding entrepreneurship in response to the pandemic. The instrument included openness to change, limiting paradigms, and resource optimization at organizations. It also featured another institutional dimension that refers to the confidence of investors and entrepreneurs in public management and administration. Thus, it was found that trust is a factor that moderates the relationship between the pandemic and its effects on entrepreneurship.

The green gross domestic product (GGDP) can be used to anticipate the effects of economic crises resulting from health threats. The ranking of green economies is led by the US, followed by Switzerland and Australia (OECD, 2021). In terms of green development, Mexico is at the bottom of the ranking along with Venezuela, Libya, Bangladesh, Chad, and Bosnia-Herzegovina. In the latest ranking of entrepreneurship favorable to the green economy in Latin America, Chile leads the list, followed by Puerto Rico and Colombia. In contrast, Salvador, Venezuela and Honduras rank the lowest.

In spite of the asymmetries between the economic regions of North America and Asia Pacific, as well as the differences between developed countries, entrepreneurship is a local phenomenon that hinges on the trust in local products over those of multinational companies (OECD, 2021). In this relationship, high-performing ventures, which are considered so based on trust in the brand and the product, produce more local growth because they generate a higher quality of life and subjective well-being. Therefore, the places with the highest development indices present the highest quality of life and well-being; in contrast, less developed communities are associated with low levels of health care, education, and employment. In other words, entrepreneurship, due to the pandemic, responds not only to the crisis but also to the local economic structure.

Even in economies with a medium or intermediate income level (such as those in Latin America), the differences lie in the levels of trust translated into perceived opportunities and entrepreneurial intentions (OECD, 2021). While Colombia and Chile exhibit a favorable balance in both areas, Mexico and Puerto Rico present differential gaps in terms of the two confidence factors. That is, entrepreneurship is not only a reaction to risky situations (such as the pandemic), but it also involves responses focused on trust. Thus, cultures with high levels of corruption, opacity, or negligence show lower levels of trust.

Central Mexico is considered the area with the greatest entrepreneurial dynamics in Latin America, reaching growth levels comparable to developed countries such as those in the eurozone. However, since 2008, the entrepreneurship indices of the States that constitute the entrepreneurial center of Mexico have changed (Instituto Nacional de Estadística y Geografía, 2020). For example, the State of Oaxaca presented the lowest entrepreneurship index; the State of Morelos, a medium one; and Mexico City (CDMX), the highest. From 2009 to 2014, that gap between States in the center of Mexico has widened. The entrepreneurial index of the State of Mexico has grown disproportionally compared to that of Mexico City, Veracruz, and Puebla; and the index of Morelos has decreased. In 2015, the level of entrepreneurial confidence in Central Mexico was measured employing investment traffic lights. As a result, Querétaro was in first place, followed by Aguas Calientes and Coahuila; by contrast, Veracruz, Oaxaca, and Chiapas occupied the last places. Nevertheless, the entrepreneurship index of Morelos is still low.

The responses of state governments have been different from those of economic and social organizations because they have focused their public policies on mitigating the pandemic rather than promoting entrepreneurship and economic reactivation (Sandoval-Vázquez et al., 2021). In the State of Mexico, the support has been directed toward sectors affected by unemployment due to the crisis, underemployment due to the opportunism of subsidiaries, and unemployment due to lack of opportunities.

In this risk-prone scenario, the mitigation policy is based on lockdowns and social distancing, which forced the transition from the traditional classroom (with the teacher as the central axis of knowledge dissemination) to its virtual counterpart (where the teacher shares learning platforms). This distinction is radical because it has strong effects on traditional or teacher-centered learning and technology-mediated learning, which requires mode student autonomy (Bustos Aguayo et al., 2021). Professors’ job satisfaction depends on information sharing, which is mediated by their cultural repertoire or contents stored on devices. Furthermore, their satisfaction lies in the convergence of two factors inherent in online teaching: (1) training motivated by learning results and (2) the communication of tasks, assignments, or exams on platforms.

In the education sector, different technologies have attracted unprecedented interest in the promotion of courses, diplomas, congresses, or colloquia. The latter have been held through platforms such as Google Meet, Zoom, or Teams in alliance with Facebook, Twitter, YouTube, Instagram or WhatsApp, who have been disseminators of cultural and educational events, as well as guarantors of real-time access for entrepreneurs, trainers, or trainees emerging from the pandemic (Rincon-Ornelas et al., 2021).

Job satisfaction has been defined as a favorable disposition towards academic, professional, and job training, which is now received online. In this sense, it has been classified as a satisfactory experience, which is derived from a commitment to the vocational instruction, but it also implies a high degree of trust in that union, the organization, or the technology (García Lirios, 2021). More specifically, it is empathy with the leadership and an intrinsic motivation with the occupation. Hence, job satisfaction is a general attitude that results in motivation and commitment to produce a positive experience of personal and group growth.

In the virtual classroom, job satisfaction refers to basic psychological processes of searching, selecting, and processing information. It is a system of perceptions of usefulness and ease of use, as well as satisfactory experiences of compatibility between technologies and teaching knowledge management, production, and transfer skills (Espinoza-Morales et al., 2021). In this context, entrepreneurship is taught using platforms such as Teams, Zoom, or Google Meet.

However, other studies into job commitment in the virtual classroom suggest rather unsatisfactory experiences (Quintero Soto et al., 2021). Face-to-face instruction involves high-risk decisions and activities due to the probable spread of the coronavirus in closed spaces. As a result, telecommuting has been adopted in Mexico because of lockdowns, overcrowding, and the fact that most Mexican families live in confined spaces. In this situation, job satisfaction is related to knowledge management, production, and transfer in academic instruction.

The impact of the pandemic on entrepreneurship has spread to employee satisfaction. McCloskey (1974) studied job satisfaction (alpha 0.89) based on safety rewards (threats and dangers), social rewards (identity and belonging), or psychological rewards (autonomy, responsibility, recognition, and appreciation). McCloskey and McCain (1987) updated this scale with an eight-dimensional structure associated with the three original theoretical dimensions. Mueller and McCloskey (1990) also found that safety rewards have reflective relationships with extrinsic reward satisfaction, scheduling, and work-family balance. In both studies, social rewards refer to satisfaction with co-workers and interaction, and psychological rewards reflect satisfaction with professional opportunities, praise, recognition, control, and responsibility.

Kumar and Majeed Khan (2014) developed a scale to measure job satisfaction composed of seven factors: privileges, cooperation, working environment, patient relationship, facilities, career development, and human resources. In their study, the factorial weights ranged between 0.476 and 0.884, which suggests that the scale should include other indicators related to the satisfaction of health professionals with their working environment.

Lee et al. (2017) established six factors in their job satisfaction scale: salary and well-being, leadership behavior, personal growth, self-employment, interpersonal relationships, and job competence. Based on exploratory and confirmatory factor analyses, they generated six dimensions with the orthogonal rotation technique. Nevertheless, their factor loadings ranged between 0.07 and 0.77, which means that other factors should be included.

Hora et al. (2018) carried out a systematic review of instruments that measure job satisfaction and found three widely used scales: the Job Satisfaction Survey, Minnesota Satisfaction Questionnaire, and the Job Satisfaction Questionnaire. All of them were shown to have acceptable psychometric properties for measuring the topic, were cited in other recent studies, and their items had ben subsequently updated or adapted to multiple research settings.

In general, the instruments that measure job satisfaction are differentiated by their degree of specification for the job function (Quiroz Campas et al., 2020). This is because it is a subjective process rather than a reflection of the culture or work environment. In this sense, it is relevant to observe the job satisfaction of professors in a pandemic scenario, who have radically transformed the way they work in the face of social distancing and lockdowns. Therefore, are there significant differences between the job satisfaction dimensions reported in the literature and the factors established in this study?

The research hypothesis of this study holds that there are significant differences between the findings reported in the literature regarding the dimensions of job satisfaction and the factors found here (Garcia Lirios, 2020). This is because the SARS-CoV-2 pandemic and Covid-19 have changed work climate, commitment, satisfaction, and job performance, generating stress by significantly increasing the demand for remote technologies among administrators, professors, and students (García Lirios et al., 2020a). Due to the impact of the health crisis on the economy in general and employment, the job satisfaction of public university professors has three dimensions that have been established in the literature (Sánchez-Sánchez et al., 2020a): information need (news about lockdowns and the educational system), perceived usefulness (data on professional development opportunities), and compatibility (adoption of virtual classrooms). In the midst of lockdowns and social distancing strategies to reduce disease transmission, receiving information about the end of lockdowns, reopening crowded places, and new outbreaks results in a low or high degree of satisfaction that will have an impact on the culture and work environment of organizations (Juárez-Nájera et al., 2020).

This outpour of information about the costs and advances of lockdowns or social distancing has an impact on the intensive use of technologies, which will generate a degree of dissatisfaction or satisfaction associated with commitment, entrepreneurship, innovation, and job competitiveness (Bustos Aguayo et al., 2020a). In addition to the information need and technological usefulness, the adoption of devices or software implies high degrees of satisfaction with the virtual classroom (Sánchez-Sánchez et al., 2020b). It is a virtuous circle of information exchange in an environment that is favorable to education.

METHOD

Design

Since job satisfaction in virtual classrooms implies scenarios of information needs (search, processing, and dissemination), compatibility between technology and computational skills, and the usefulness of these platforms, these three dimensions (reported in the literature) can be compared with observations in a sample (Bustos Aguayo et al., 2020b). This is possible because these three dimensions can be modeled adopting the theory of job satisfaction. In principle, the theoretical corpus suggests that job satisfaction in virtual classrooms is the result of basic cognitive processes: perceptions, attitudes, motivations, skills, knowledge, intentions, and specific actions (García Lirios et al., 2020b).

These basic cognitive processes converge into three complex factors: information need, compatibility of skills with the accessibility and ease of use of technologies, and technology usefulness. Other studies have examined such convergence of the three factors and reported dimensions (measured using instruments) that refer to the impact of technology on the academia, college education, and professional training (Carreón-Guillén et al., 2020a). Although professors’ job satisfaction can be the result of teaching and professional practice, positive experiences indicate that they also need to improve their skills and specialize in data management. Professors show a propensity to disseminate information on the Internet, which has been associated with the compatibility of computational skills with the accessibility and ease of use of technology, as well as the potential benefits of its use. That is, in the face of the pandemic, teachers activate cognitive processes aimed at accepting technology and collaborative learning that can be disseminated in virtual classrooms (Carreón Guillén et al., 2020b).

Therefore, this cross-sectional psychometric study aims to examine the validity of an instrument that measures job satisfaction, confirm the dimensions reported in the literature, and establish the percentage of variance explained in order to be able to propose a model that measures the three factors and their indicators.

Sample

Purposive sampling was employed here to select 100 professors (Age, M = 34.2 and SD = 10.28; monthly income, M = 8,890.12 USD and SD = 265.29 USD) from a public university in central Mexico considering their experience of the recent change in teaching practices. Since the university was recently founded, most professors have completed undergraduate studies in social sciences, especially social work, communication, languages, and actuarial science.

Instrument

The Job Satisfaction Scale (JSS-21) employed here was constructed based on the systematic review of Hora et al. (2018), who examined three instruments: the Job Satisfaction Survey, Minnesota Satisfaction Questionnaire, and the Job Satisfaction Questionnaire. All three measure the organizational phenomenon of job satisfaction considering three preponderant dimensions: information need, technology usefulness, and compatibility with the virtual classroom. Each item included five response options ranging from 0 (not satisfactory at all) to 5 (satisfactory).

Process

Microsoft Teams software was used to respond to a section of the survey. The selected professors received, via their institutional email address, an invitation and a link to answer the questionnaire, which was composed of 21 items and socio-educational questions, each with a response scale. At the end, they were given a certificate of participation. The confidentiality and anonymity of their responses, as well as the adequate handling and protection of their personal data, were previously ensured.

Analysis

The data were processed in Statistical Package for the Social Sciences (SPSS) version 23.0 to establish the parameters of distribution, reliability, validity, fit, and residual. SPSS was also utilized to test the null hypothesis in this study, i.e., there are no significant differences between the theoretical dimensions (reported in the literature) and the factors of job satisfaction found here.

Normal distribution and discrimination of items

The first condition was estimated using the Mardia coefficient (.9.20), which reached a value that could be located at the multivariate normal distribution threshold (Trógolo et al., 2019). The second condition, kurtosis and bias parameters were estimated considering values lower than unity. Thus, the scale obtained bias (.678) and kurtosis (.892) values lower than those required by the standards.

Exploratory Factor Analysis

The extraction method was main axes and promax rotation (criterion of five subjects per item). The preliminary requirements of adequacy (KMO = .657) and sphericity were estimated ⌠χ2 = 14.21 (14df) p < .05⌡.

Interpretation

Values close to zero were considered to be evidence of spurious relationships; values close to unity, evidence of collinearity; and values between 0.30 and 0.90, evidence of a relationship between the variables. To test the null hypothesis, it was assumed that values lower than 0.05 were evidence of significant differences at settings greater than 0.90 and residuals lower than 0.009.

RESULTS

Table 1 shows the descriptive values of the instrument that measures professors’ job satisfaction considering thresholds to interpret the results. All the values are in a range in which reliability and validity tests can be conducted. Both analysis show a three-dimensional structure that converges to a common factor in the same way as the initiators in each dimension.

Table 1. Item descriptions
R Item M SD A F1 F2 F3
r1 Platform taxonomy for virtual classrooms 3.21 1.81 .711 .432
r2 User traffic of virtual educational platforms 3.29 1.93 .703 .329
r3 Data on connectivity in virtual classrooms 4.13 1.37 .735 .314
r4 Information on keyword search 4.21 1.56 .792 .405
r5 Productivity index of scientific articles 4.08 1.51 .721 .465
r6 Number of downloads of special items 4.27 1.07 .760 .437
r7 Citation index by author during the Pandemic 2.17 1.03 .774 .328
r8 Learning in virtual classrooms 2.10 1.02 .701 .429
r9 Hours dedicated to online education 4.36 1.01 .794 .432
r10 Scientific article preparation time 4.87 1.09 .763 .501
r11 Instrument development time 2.07 1.43 .782 .643
r12 Video conference preparation 4.76 1.42 .774 .328
r13 Virtual classroom management time 3.35 1.78 .790 .543
r14 Real-time data processing 3.28 1.04 .781 .671
r15 Use of platforms for virtual classrooms 4.30 1.53 .775 .439
r16 Use of video conferencing 2.65 1.52 .776 .320
r17 Use of electronic classroom devices 3.14 1.60 .709 .456
r18 Use of alternative online education 2.74 1.85 .762 .531
r19 Use of data storage devices 3.39 1.45 .783 .629
r20 Use of platforms for academic events 2.45 1.63 .759 .543
r21 Use of scientific databases 4.13 1.21 .743 .630
Note: Compiled using data collected in this study. R = Reactive, M = Mean, SD = Standard Deviation, A = Cronbach’s alpha, F1 = information need (25% total variance explained and alpha = .781), F2 = technology usefulness (18% variance explained and alpha = .791), and F3 = compatibility with virtual classroom (13% variance explained and alpha = .765).
Source:
Created by the author.

The exploratory factorial structure of professor job satisfaction includes three dimensions: information need, perceived usefulness, and technology adoption. However, the correlations between the factors and the items are low, which indicates that other indicators should be included. In order to observe the structure of relationships between factors, a covariance matrix was estimated. Said matrix suggests that a reflective model of job satisfaction should be developed using the three dimensions and its indicators (see Table 2).

Table 2. Relationships between factors
M SD F1 F2 F3 F1 F2 F3
F1 21.37 16.21 1.000 1.876 .512 .612
F2 23.51 17.39 .672* 1.000 1.890 .501
F3 28.31 15.21 .529** .546* 1.000 1.671
Note: Compiled using data collected in this study. M = Mean, SD = Standard Deviation, F1 = information need, F2 = perceived usefulness, F3 = technology adoption; * p < .01; ** p < .001; *** p < .0001.
Source: Created by the author.

The correlation and covariance structure shows that other factors, which were not included in the model, could be incorporated if they are identified in the literature, but the percentage of explained variance suggests an orthogonal criterion. In order to observe the relationships between all the variables considering the adjustment and residuals to test the null hypothesis, a reflective model of job satisfaction was developed in this study (see Figure 1).

Figure 1. Structural equation model
Note: Compiled using data collected in this study. F1 = information need, F2 = perceived usefulness, F3 = technology adoption, r = indicator, e = error measured indicator, relationship between errors and indicators, relationship between factors and indicators.
Source:
Created by the author.

The adjustment and residual parameters found here ⌠χ2 = 23.12 (13 df) p > .05; GFI = .997; CFI = .997; RMSEA = .008⌡ suggest that the null hypothesis (i.e., there are no significant differences between the theoretical structure reported in the literature and the structure observed in this study) was not rejected.

DISCUSSION

This study contributes to the state of the art in this field because it establishes the exploratory factorial validity of the Job Satisfaction Scale, although the study design limits the results to the sample.

The number of dimensions in classic instruments that measure job satisfaction ranges between three and seven factors related to resources with which workers carry out their task or function. This study has shown the prevalence of three factors related to the technology, devices, and software that are used in virtual classrooms. This new type of satisfaction is the result of the need for long lockdowns and social distancing. Due to this situation, information about the pandemic and the flipped classroom is increasingly searched, processed, and intensively used. In a flipped classroom, professors no longer occupy central place of knowledge and delegate their instruction to student’s self-management and computer skills, limiting themselves to motivational support, which is increasingly satisfactory as the pandemic deepens.

This result should be confirmed because job satisfaction also depends on the traditional classroom and the centrality of knowledge. The balance between the traditional and the emerging paradigm implies high degrees of professor satisfaction, which could be explained by a balanced approach, the dissemination of knowledge, and the emergence of new capabilities.

In this emerging virtual classroom learning process, the adoption and intensive use of technology can generate high levels of satisfaction that could be reduced with the return to traditional classrooms. Nevertheless, a balance between face-to-face and distance functions can maintain high levels of satisfaction. By studying the profiles and capabilities of professors, educational institutions can anticipate scenarios that can maintain the job satisfaction of their faculty, as well as their academic and research performance.

Therefore, establishing the validity of the job satisfaction construct is a first step towards the convergent validity of the instrument with respect to other scales that measure variables concomitant with the phenomenon under study. This is the case of multi-feature and multi-method validity, which can be used to anticipate the measurement of connected phenomena based on the comparison of heterogeneous, non-standardized, and standardized observations and records. The prediction of job satisfaction in virtual and traditional classrooms brings significant benefits in the generation of intellectual capital beyond the academia and has effects on the professional and work environment of university professors.

CONCLUSIONS

The objective of this study was to establish the validity of an instrument that measures professors’ job satisfaction using three dimensions: information need, technology usefulness, and adoption of virtual classrooms. Although the findings here are limited to the sample, they suggest that future research should investigate construct and scale validity to achieve convergent validity.

However, the proposed model is robust. The non-rejection of the null hypothesis (i.e., there are no significant differences between the theoretical structure reported in the literature and the structure observed in this study) means three things: (1) Other factors should be included to increase the percentage of explained variance and adjust the theoretical model based on the observation of the proposed dimensions. (2) The model should be empirically tested in other risk scenarios, such as insecurity and lockdown, as well as using other samples of workers that present levels of dissatisfaction or satisfaction when forced to use some technology. And (3) future studies should propose a new modeling of the factors and indicators of job satisfaction.

The job satisfaction of professors who forcibly left the traditional classroom and adapted to its online counterpart had not been measured thus far in studies on this topic. Nevertheless, predicting such satisfaction sparks the discussion about the organizational conditions that will allow them to return to face-to-face instruction without leaving online education aside. Future research should clarify this question by anticipating possible scenarios of balance between face-to-face and distance classes.

 

FOOTNOTE

  • arrow_upward * This article is derived from the collaboration project between the Transdisciplinary Academic Network and the Universidad Autónoma del Estado de Hidalgo, registration number ATN-UAEH-CGL-2020-001.
REFERENCES