Computational Thinking in the University Context: A Literature Review of Assessment Instruments

Pensamiento computacional en el contexto universitario: Revisión de la literatura de instrumentos de evaluación

DOI 10.22430/22565337.3394 PDF Table Figure

Received: Enero 28, 2025
Accepted: Julio 07, 2025
Available: Septiembre 16, 2025

 

How to cite / Cómo citar
M. Corrales-Álvarez, Á. M. Muñoz-Muñoz, and S. A. Cardona-Torres, “Computational Thinking in the University Context: A Literature Review of Assessment Instruments,” TecnoLógicas, vol. 28, no. 64, e3394, 2025. https://doi.org/10.22430/22565337.3394

 

Highlights


  • The need for instruments that measure beyond the cognitive domain is highlighted.
  • The dimensions most frequently assessed are algorithmic thinking, abstraction, and decomposition.
  • There is a lack of solid psychometric evidence in the instruments, affecting their validity and reliability.
  • There is a gap in the assessment of computational thinking between school and university education.
  • It is necessary to assess attitudes and practices in addition to the cognitive skills of computational thinking.
  •  


    Highlights

  • Se destaca la necesidad de instrumentos que midan más allá del dominio cognitivo.
  • Las dimensiones más evaluadas son pensamiento algorítmico, abstracción y descomposición.
  • alta evidencia psicométrica sólida en instrumentos, afectando su validez y confiabilidad.
  • Existe una brecha en la evaluación del pensamiento computacional entre educación escolar y universitaria.
  • Se requiere evaluar actitudes y prácticas, además de habilidades cognitivas del pensamiento computacional.
  •  

    Abstract

    Computational thinking (CT) is recognized as a skill that citizens of the current era must develop to perform effectively in various contexts. This highlights the need to incorporate it at all educational levels to strengthen the teaching, learning, and assessment processes. Considering assessment as one of the key processes to support CT, it is necessary to identify instruments with psychometric properties for its measurement. This systematic review aimed to identify and analyze the different instruments designed to assess CT within the university community (university students, future teachers, current teachers, and adults). The methodology involved a search from 2006 to February 2024 and the use of information sources Web of Science (WoS) and Scopus. A total of 266 articles were found, of which 27 met the inclusion criteria, and 17 instruments were identified that assess CT in the university community. The results showed that these instruments come in various formats, with scales and tests being the most common. The most frequently evaluated variables were algorithmic thinking, abstraction, and decomposition. Regarding psychometric properties, there was limited evidence on the procedures used for instrument validation. The findings revealed an open space for the development of instruments aimed at the university population that allow for the measurement of CT from dimensions beyond the cognitive one. In conclusion, it is identified that the focus of research has been on the assessment of CT skills in school populations and little evaluated in the university community.

     

    Keywords: Bibliometric analysis, computational skills, higher education, psychometric properties, skills assessment.

     

    Resumen

    El pensamiento computacional (PC) es reconocido como una habilidad que los ciudadanos de la era actual deben desarrollar para desempeñarse en diferentes contextos. Esto advierte la necesidad de incorporarlo en todos los niveles educativos para fortalecer los procesos de enseñanza, aprendizaje y evaluación. Considerando la evaluación como uno de los procesos importantes para fundamentar el PC, se hace necesario el reconocimiento de instrumentos que cuenten con características psicométricas para su medición. Esta revisión sistemática tuvo como objetivo identificar y analizar los diferentes instrumentos diseñados para la evaluación del PC en la comunidad universitaria (estudiantes universitarios, futuros docentes, docentes y adultos) El periodo de búsqueda comprendió del 2006 a febrero de 2024. La metodología empleada consistió en el uso de las fuentes de información Web of Science (WoS) y Scopus. Se encontraron 266 artículos, de los cuales se seleccionaron 27 artículos que cumplían con los criterios de inclusión y de los cuales 17 instrumentos evalúan el PC en la comunidad universitaria. Con los resultados se identificó que estos instrumentos se presentan en diferentes formatos destacándose la escala y las pruebas. Las variables evaluadas con mayor frecuencia son el pensamiento algorítmico, la abstracción y la descomposición. Con respecto a las propiedades psicométricas se identificaron escasas evidencias sobre los procedimientos aplicados para la validación de los instrumentos. Los resultados obtenidos mostraron un espacio abierto para la construcción de instrumentos orientados a población universitaria que permitan la medición del PC desde dimensiones adicionales a la cognitiva. Como conclusión se identifica que el enfoque de las investigaciones se ha centrado en la valoración de habilidades de PC en poblaciones escolares y poco evaluado en la comunidad universitaria.

    Palabras clave: Análisis bibliométrico, habilidades computacionales, educación superior, propiedades psicométricas, medición de habilidades.

     

    1. INTRODUCTION

    Computational Thinking (CT) has been proposed as an essential competence that should be addressed at different educational levels and that is indispensable for the performance of citizens in an increasingly technological world [1]. Different authors have proposed definitions for this construct, starting with [2], who established that CT is a construct linked to problem solving and system design to understand human behavior, using tools and taking advantage of fundamental concepts of computer science. In [3] was formulated a classification of CT definitions considering: generic definitions, related to the way humans think, applying computer science to understand artificial processes and systems; operational definitions, which understand CT as a problem-solving process; psychological-cognitive definitions, which refer to problem-solving skills and techniques; and educational-curricular definitions, which refer to proposals to include CT in the curriculum from models or theoretical frameworks.

    Furthermore, prominent authors [4]–[6] presented CT associated with problem solving to develop skills related to mental processes such as abstraction, decomposition, and generalization. [7]–[9] presented CT as a methodology for problem solving that uses concepts such as abstraction, iteration, and recursion, which allow data analysis and creation of tangible or virtual artifacts. Similarly, in the analysis of other definitions [3],[10]–[12], the authors highlighted approaches related to informatics, computing, technology, or programming where they stand out. The proposed definition[13] placed CT as a cognitive activity consisting of establishing a problem, conceiving it, and representing it in an algorithm to finally evaluate the quality of the solution. The definitions of CT show the diversity of conceptions about this construct, which identifies the need to support its operationalization and measurement for different populations and age groups [14].

    In parallel, CT assessment has acquired a crucial role conditioned by those diverse conceptual meanings [15] In this sense, CT assessment has been mainly oriented toward the evaluation of technical knowledge in programming [16] and skills such as logical thinking, problem solving, and decomposition [17]. Several literature reviews [18]–[23] present the use of CT measurement instruments in different populations. The scoping review of CT assessments in higher education presented in[18], analyzed 33 empirical studies on CT assessments in university students and identified that most measurement instruments that are oriented to assess CT skills combining cognitive skills, aptitudes, and attitudes. Similarly, they reviewed the psychometric properties of the instruments, and only four presented evidence of reliability and validity. The review by [19] included 39 studies with a university population. They concluded that there is little evidence of validity in the assessment instruments and the need to implement assessments aimed at improving thinking skills.

    In the systematic review proposed by [20], 96 articles were evaluated and it was concluded that CT assessment for teacher education has not been sufficiently investigated and marks the absence of evidence of the reliability and validity of the instruments used. In a review of instruments to measure CT [23], 64 studies were presented, of which 16 referred to teachers and university students. The authors highlighted the need to develop valid and reliable tools to measure CT skills at the level of infants, university students, and seniors. Another review [21] analyzed 145 investigations and concluded that it is necessary to incorporate teacher training programs related to CT. The review by [18] focused on university students and teachers in service, identifying a lack of psychometric properties of the instruments for this population.

    Considering this evidence, this systematic review of literature arises that aims to identify instruments that evaluate CT in the university community (university students, future teachers, teachers, and adults). A component of this review is related to the psychometric properties (reliability and validity) of the assessment instruments. Reliability is considered as the consistency of scores obtained on different occasions by the same evaluators [24] and a property of scores obtained by a group of people on a particular occasion [25]. This can be determined from the reliability coefficient whose most recognized methods to find it are the coefficients: 1. Products of the Pearson moments, 2. Pearson correlation, 3. Spearman-Brown prophecy formula, 4. Kuder-Richardson reliability and 5. Alpha Cronbach or standard error of measurement (SEM) [26] On the other hand, validity is traditionally considered the degree to which a test measures what it aims to measure [25] and is an estimate of the quality of the test measures in a given context [27]. It is possible to determine different types of validity: content, predictive, concurrent, and construct [26]. Some methods used to find it are factor analysis (exploratory and confirmatory), content validity index (CVI), Kappa modification, multitrait-multimethod matrix, among others [24], [28].

    Recognizing the importance of evaluating CT, this article aims to identify and analyze the different instruments that have been designed for CT evaluation in the university community, recognizing the bibliometric components, the evaluation instruments, and their psychometric properties.

     

    2. METHODOLOGY

    This systematic review of literature followed the protocol substantiated in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement [29] and is proposed from what is presented in Table 1. The method is described based on eligibility criteria, search strategies, databases selection, data extraction, bibliometric analysis, and data visualization.

    Table 1. Description of components related to the revision
    Source: own elaboration.
    Component Variables Specific objectives Research questions
    Bibliometric Source of information Year of publication Authors H-index (Google Scholar) Country and continent Magazine Quartiles Scientific Journal Rankings (SJR) Journal Citation Reports (JCR) of the journal Recognize and present the evolution of publications with respect to the variables associated with the publications, authors, and journals where the research was published. How have the publications in the sample evolved with respect to the bibliometric variables?
    CT evaluation instruments Population under study Sample size Number of items Variables evaluated Analyze the instruments that have been proposed to evaluate CT in the university community and the variables used to measure CT. What are the instruments that have been proposed to measure CT in the university community? What are the constructs or variables evaluated?
    Psychometric properties Pilot test Evidence of reliability and validity Methods used to determine them Identify the psychometric properties of the instruments and the methods used to determine them. What are the psychometric properties of the instruments? What statistical methods were used for the analysis of psychometric properties?

    2.1 Eligibility criteria

    Original research and review articles published in scientific journals from 2006 (considering that this was the year in which the concept of CT first appeared) to February 2024 were included. Research that assesses CT and uses measurement instruments such as scales, tests, surveys, questionnaires, and tools that present evidence of reliability and validity applied in the university community was considered. Documents in the formats of reviews, editorials, books, letters, abstracts, posters, and proceedings of events were not considered. Articles that addressed CT assessment from an approach other than the use of measurement instruments (interviews, observations, and interventions) were also excluded. Articles that did not correspond to the population under study were excluded: kindergarten, elementary education, primary education, and secondary education.

    2.2 Search strategy and databases

    Initially, categories were defined that allowed the construction of the chain; the main category was computational thinking, and the complementary categories corresponded to the type of tool, action, and target population. With these categories, a set of keywords was obtained that converge in the search string: Computational thinking OR “pensamiento computacional” (Topic) AND instrument* OR tool* OR survey* OR scale* OR questionnaire* OR strateg* OR test* (Topic) AND evaluat* OR assess* OR valid* OR measur* OR reliability* OR psychometric* (Topic) AND teacher* OR educator* OR “higher education” OR “university students” OR “teaching students” OR “teaching assistants” (Topic) NOT preschool* OR child* OR elementary* OR infants* OR pre-K* OR “early age” OR “Primary education” OR “k12 education” OR “primary school” OR “preschool children’s” (Topic) and Article or Review Article (Document Types). The databases used were Web of Science (WoS) and Scopus.

    2.3 Selection of studies

    A total of 266 articles were identified in both databases, of which 89 were repeated. Subsequently, the summary was analyzed, and the exclusion and inclusion criteria were applied, which determined that 132 articles were discarded. A second sample of 45 studies was obtained, and, after an in-depth analysis of their content, a definitive sample of 27 articles was determined. The 18 studies discarded did not use any instrument to assess CT in the adult population or university students. of the 27 articles, 17 report the design of instruments and the remaining 10 make use of an instrument that has already been created. Figure 1 shows the literature selection flowchart.

    Figure 1. Literature selection flowchart
    Source: adapted from [29]

    3. RESULTS AND DISCUSSION

    From the 27 articles (see the appendix) that make up the sample, the results are approached from 3 components: (1) bibliometric variables: source of information, year of publication, authors, H index (Scholar google), country, journal, and the quartiles Scientific Journal Rankings (SJR) and Journal Citation Reports (JCR) of the journal; (2) CT assessment instruments: study population, sample size, number of items, response options, and variables assessed; (3) psychometric properties: pilot test, evidence of reliability and validity, and methods used to determine them; and (4) instruments used. VOS Viewer was used for the bibliometric analysis [30]

    3.1 Bibliometric

    The development of this component is presented with the analysis of the variables: source of information, population under study, year of publication, authors, H index (Scholar google), country where the research was conducted, journal, quartiles Scientific Journal Rankings (SJR) and Journal Citation Reports (JCR) of the journal.

    3.1.1 Sources of information

    The search for information was carried out on WoS and Scopus. Nine articles appeared in WoS and seven in Scopus. The remaining eleven articles were identified in both.

    3.1.2. Target population

    The studies were classified into university students, teachers in training, and adults. Figure 2 shows the distribution of the population according to the 27 articles.

    Figure 2. Classification of the population under study
    Source: Own work

    3.1.3. Year of publication

    It is evident that the first research was initiated in 2016.[31] In 2019, four studies were found [15], [17], [32], [33]; and in 2020 the search returned two publications [34], [35], the highest productivity was identified in 2021[36][44]. For 2022, the following works were identified for [45]–[52] and finally, in 2023, 3 studies were identified [53]–[55] Figure 3 shows the number of publications per year.

    Figure 3. Number of publications per year
    Source: own elaboration.

    3.1.4. Authors and H-indexes (Scholar google)

    Eric Wiebe was found to be the only author appearing in more than one publication [39], [53]. The authors associated with the publication, and their Scholar Google H-index, are presented as a table in the Appendix. The author’s link analysis was performed with 27 documents. Figure 4 shows the relationships that exist between the co-citations of the references. The groups of nodes created in the red one are three, represented with green, blue, and red colors. Cluster 1 (red) contains 10 references, the most notable for the number of citations received: [56], [57], [58], and [59]. Cluster 2 (green) contains 9 references that stand out with greater weight: [60], and [10] and finally cluster 3 (blue) contains 7 references, where it stands out [61].

    Figure 4. Number of publications per year
    Source: own elaboration.

    On the other hand, the co-citation between the authors is presented in Figure 5, considering a minimum of ten citations per author with two clusters. Number one (red) with 11 authors include Wing (20 citations), Grover (19 citations), and Román-González (19 citations), and in cluster two (green) the authors with the highest number of citations are Yadav (27 citations) and Korkmaz (18 citations). The co-citation network reveals two main clusters led by Wing, Yadav, and Brennan (green cluster), and Grover, Román-González, and Shute (red cluster), reflecting distinct yet interconnected research trajectories. Wing and Yadav drive a conceptual orientation focused on the curricular integration and theoretical framework of CT. In contrast, Grover, Román-González, and Shute lead efforts to develop and validate CT assessment tools, emphasizing psychometric rigor. Authors like Tang, Lye, and Voogt further strengthen their respective clusters through contributions in pedagogy and skill evaluation. Intermediary figures such as Shute and Román-González connect conceptual and evaluative domains. These patterns show how the centrality of citations reflects intellectual leadership and thematic convergence in CT scholarship.

    Figure 5. Co-citations between authors
    Source: own elaboration.

    3.1.5. Country

    The country with the most publications is Türkiye with nine articles, followed by Indonesia and Thailand with three, China and Spain with two publications each. Switzerland, Iraq, Taiwan, Lithuania, Mexico, USA, Australia, and Canada have one publication, which can be seen in Figure 6.

    Figure 6. Country of origin
    Source: own elaboration.

     

    3.1.6. Journal and quartile Scientific Journal Rankings (SJR) y Journal Citation Reports (JCR)

    The journals with the highest representation in the publications are Education and Information Technologies with four articles and Journal of Educational Computing with two articles. With respect to quartiles, more than 50 % of the publications were published in journals with a Q1 and Q2 classification. This can be seen in Figure 7.

    Figure 7. Quartile Journal Citation Reports and Scientific Journal Rankings
    Source: own elaboration.

    3.2 CT assessment instruments

    The development of this component is presented with the analysis of the variables: sample size and pilot test, instrument, and variables evaluated.

    3.2.1. Sample size and pilot

    Figure 8 presents the number corresponding to the samples used in the 27 studies analyzed (to identify the studies, see Table 2). The largest sample size is represented by 1023 prospective science teachers [51], the smallest sample is represented by 36 future primary school teachers [36] and on average the samples are made up of 265 individuals. Eighty-five percent of the investigations did not present evidence of pilot testing. Two studies are presented without instrument application and without pilot testing, in the study proposed by [17] they considered a sample of 20 experts, and in the article proposed by [31] they do not present the total number of individuals.

    Figure 8. Sample sizes
    Source: own elaboration.

    3.2.2. Instrument

    A total of 17 instruments were presented and developed [15], [16], [46], [48], [49],[51], [62], [63], [17], [31],[33], [36], [37], [39]–[41]. Of the 27 articles analyzed, the remaining 10 were published in [34], [38],[42]–[45], [50], [53], [54], [64] referring to the instruments developed in one of the 17 investigations. The Bebras and T-STEM CT tasks were each used in two investigations, and the CTS Computational Thinking Scale was used in 9 investigations. Most of the instruments contain between 20 and 29 items. Table 2 presents the 17 instruments related to the variables assessed.

    3.2.3. Variables evaluated

    Considering the 17 instruments, it was found that algorithmic thinking is assessed by 9 instruments, abstraction with 7 instruments, decomposition with 6 instruments, evaluation with 5 instruments, and problem solving with 4 instruments. Table 2 presents the 17 instruments found associated with the study population, the number of items, the variables evaluated, the response format, evidence of reliability and validity. In addition, the 10 articles where the instruments T-STEM CT, Bebras Tasks, and the Computational Thinking Scale CTS were used are presented

    Table 2. Characterization of the instruments associated with the investigations where they were used
    Source: own elaboration
    Ref. Instrument used Population Items Variables Response format Evidence Apti-tudes Atti-tudes
    Reliability Validity
    [17] Hi-ACT Holistic CT assessment University students 98 Abstraction Algorithmic thinking Decomposition Debugging Evaluation, Generalization Troubleshooting Teamwork Communication Spiritual intelligence Likert 7 No evidence Content X
    [16] P-CTS Scale for development programming-oriented computational thinking University students 33 Conceptual knowledge Algorithmic thinking Assessment Unspecified Cronbach's Alpha Construct X
    [48] ATTA Algorithmic thinking test for adults Adults Bachelor's, Master’s, or Doctoral Students 20 Abstraction Decomposition Algorithmic thinking Pattern/recognition Evaluation/debugging Multiple choice Cronbach's Alpha Content X
    [39] T-STEM CT Instrument to measure self-efficacy of in-service teachers regarding teaching CT skills and practices to students Teachers 20 Personal beliefs on teaching effectiveness Beliefs about the expectation of teaching results Likert 5 Cronbach's Alpha Structural X
    [53] Indonesian teachers No evidence No evidence
    [49] Test research instrument to measure CT level University students 24 Unspecified Multiple choice Without specifying the coefficient Construct and apparent X
    [15] CT Questionnaire for students Computer education students 25 The ability to: - Analyze and manage data - Use information to solve problems - The ability to design countermeasures - The ability to create workpieces -The ability to evaluate work pieces -The ability to analytically identify the causes -The ability to work collaboratively Likert 5 Reliability coefficient Validity type is not specified X
    [62] CTS Computational thinking scale Future teachers and teachers 29 Creativity Algorithmic thinking Cooperativeness Critical thinking Problem resolution Likert 5 Cronbach's Alpha No evidence X
    [44] Mathematics teachers in Türkiye Cronbach's Alpha No evidence
    [34] Turkish trainee teachers Cronbach's Alpha No evidence
    [38] Mexican undergraduate students Cronbach's Alpha No evidence
    [50] Turkish trainee teachers Cronbach's Alpha Validity type is not specified
    [42] Teachers from Türkiye Cronbach's Alpha No evidence
    [52] University students from China Cronbach's Alpha Convergent
    [43] University students in Türkiye Reliability coefficient Validity type is not specified
    [54] Computer science and engineering students from Türkiye Cronbach's Alpha No evidence
    [36] CT test Teachers in training 10 Sequences Loops Events Parallelism Conditional Operators Data Multiple choice Kuder Richardson No evidence X
    [47] CT Content knowledge test in teachers Teachers Nivel 1: 17 Nivel 2: 25 Evidence Depuration Reuse Remix Abstraction Modularization Algorithmic thinking Multiple choice open Cohen's Kappa No evidence X
    [37] QCT CT questionnaire Teachers 22 Decomposition Abstraction Design of algorithms Automation Data collection Analysis of data Representation of data Simulation Parallelization Multiple choice Cronbach's Alpha Construct X
    [31] Bebras Tasks University students 20 Functional Thinking as a Compound of Problem Decomposition Pattern recognition Generalization Algorithmic thinking Abstraction Unspecified No evidence No evidence X
    [45] Australian trainee teachers Coefficient K Content and Criteria
    [40] CT Skills Test Future teachers 20 Decomposition Pattern recognition Abstraction Algorithm design Multiple choice Cronbach's Alpha No evidence X
    [51] CT Skills self-confidence questionnaire Future teachers 34 Self-confidence in CT Expectation Self-confidence Optimism Resilience Experience Likert 5 Cronbach's Alpha Construct X
    [41] Questionnaire to measure CT skills University students 27 Abstraction Modeling Decomposition Algorithmic thinking Representation Generalization Assessment Adjustment Unspecified Cronbach's Alpha No evidence X
    [33] CTt Computational thinking test Future teachers 32 Computational concepts: basic directions, ''repeat'', ''repeat to'', ''if'', ''if/if not'', ''while'' and simple functions Multiple choice No evidence No evidence X
    [46] CCTt Test Callysto CT Future teachers 31 Non-cognitive attitudes towards CT Technology Troubleshooting, Coding Data Concepts Practices Multiple choice Cronbach's Alpha Construct X X
    [55] CT knowledge and understanding survey Teachers 10 CT concept Experience in CT learning management Experience in participating in CT training Self-development Focus of learning activities Likert 5 No evidence No evidence X

    3.3 Psychometric properties

    To evaluate the psychometric properties of the instruments, evidence of validity, reliability, and method was analyzed. Table 2 presents each study associated with the evidence of validity and reliability. Eighty percent present reliability analysis, and the most used method was Cronbach's Alpha. With respect to validity, 50 % of the studies do not present any evidence. The most used method was the Confirmatory Factor Analysis (CFA).

    3.4 Instruments used

    As a result, the instruments associated with the studies in which they were used or created and the process carried out are presented. The Hi-ACT Holistic CT Assessment [17]: Instrument used to assess skills and attitudes in Indonesian university students. The P-CTS Programming-Oriented Scale [16]: Developed to determine programming-oriented CT skills in university students. The ATTA Algorithmic Thinking Test for Adults [48]: The development and validation process of the instrument for adults was performed. The T-STEM CT tool is designed to assess in-service teachers' confidence in their ability to teach computational thinking skills and practices to students. This instrument was used in two investigations [39], [53]. The first investigated the changes and the sources of these changes in Indonesian teachers' self-efficacy to teach science and CT [39]. The second addressed the development and validation of an instrument that measures in-service teachers' self-efficacy beliefs for teaching CT[53].

    The test research instrument to measure the level of CT [49]: Measuring CT in third-year students studying Computer Science at Baghdad University. The CT student questionnaire [15]: Assessment of CT in prospective teachers in Thai universities. CT test [36]: Research using this instrument developed a STEM-based game design project from teacher training to help prospective elementary teachers acquire CT concepts. The CT content knowledge test [47]: The objective of this study was to examine how a teacher development program changed the content knowledge and collaborative participation of elementary school teachers.

    The QCT CT Questionnaire [37]: The study aimed to explore the perspectives of STEM teachers (Science, Technology, Engineering and Mathematics) on the integration of information and communication technologies (ICT) into their teaching, with a particular focus on incorporating CT into their lessons. The Computational Thinking Test (CTt), developed by [3], was utilized for this purpose. In [33] was used this instrument to assess and illustrate the outcomes of an educational robotic intervention designed to improve the CT skills of future teachers. The CCTt [46]: This test examines the relationship between CT attitudes and the skills of preservice teachers. The CCTt Callysto CT Test [46]: With this test, the relationship between attitudes and CT skills for future teachers is examined.

    The CTS Computational Thinking Scale authored by [59] : This instrument was the most used, it was applied in nine investigations. In [32] the instrument was applied to compare the CT skills of preservice and in-service teachers. The authors in [38] identified differences in CT dimensions between disciplinary areas in undergraduate students of various disciplines at two Mexican universities. In [50] was determined the relationships between learning self-efficacy and teacher performance in training, CT skills, and metacognitive self-regulation.

    In [65] it was proposed to use the instrument to examine the acceptance, self-development, and CT skills of technology among teachers of various disciplines who participated in in-service training in basic robotics coding in primary schools. In [52] was investigated the impact of Socially Shared Regulation of Learning (SSRL) on CT, learning motivation, engagement, and academic achievement among third-year university students at a public university in China, in the context of Collaborative Learning through Teaching (CLBT). The authors in [43] explored whether collaboration and critical thinking serve as mediators between other skills of the 21st century, such as algorithmic thinking, creativity, digital literacy, and effective communication, and problem solving abilities in university students in various regions of Türkiye. Additionally,[44] used the Korkmaz scale alongside the Mental Types Scale to explore the relationship between cognitive styles and computational performance (CP) among mathematics teachers. In [34] was proposed and tested a model that explains the knowledge of STEM of preservice teachers (enrolled in the faculties of education of three state universities in different regions of Türkiye) by examining CT skills, the media and technology use and attitudes, and demographic variables, including gender, type of high school, and parental educational background. The objective of this research was to find the impact of personality and motivation on CT skills among computer science and engineering students [54]

    The Bebras Tasks: This instrument was used in two of the 27 publications analyzed. In [31] a study was presented conducted among first-year software engineering students, who are in the structured programming course looking to determine the correlation between the student's test scores and the results of the structured programming course. The authors in [45] examined the relationship between different forms of CT and two different measures of programming quality for a group of teacher trainees. The CT Skills Test [40]: This test was used to assess activities designed for online learning that developed the CT of first-year students enrolled in a Bachelor of Education Program in Educational Technology and Communications at King Mongkut's Thonburi University of Technology. The Self-Confidence Questionnaire on CT Skills [51] was applied to analyze the confidence of prospective science teachers' in CT skills from three aspects: first, self-confidence; second, differences in self-confidence; and third, differences in self-confidence according to experience. The Questionnaire to Measure CT Skills [41] focused on university students, emphasizing abstraction and its possible relationship with other factors (modeling, decomposition, algorithmic thinking, representation, generalization, evaluation and adjustment). The survey of knowledge and understanding about CT [55] consisted of studying the need and problems in learning management that promote CP of teachers at the junior high school level.

    The assessment of CT using measurement instruments has been addressed at all educational levels. The aim of this systematic review of the literature was to identify instruments that have been used in the adult population. Twenty-seven investigations were identified that assessed CT in the adult population; 17 measurement instruments were extracted from these publications. According to the results, it was identified that most of the studies focused on CT evaluation in university students with a representation of 68 % of the studies, 29 % are made up of teachers, and 3 % are adults, which demonstrates the importance of having instruments that allow objective CT evaluation in the teaching population. It is highlighted that research using instruments to evaluate CT in the adult population is on the rise with 74 % of research in the years 2021, 2022, and 2023, taking into account that now the interest of the scientific community is oriented towards aspects related to the development of new assessment tools and their relationship with learning [19] The countries that contribute the greatest amount of research are Türkiye with 32 %, followed by Indonesia with 12 %, Thailand, China, and Spain with 8 % each, and finally Switzerland, Iraq, Taiwan, Lithuania, Mexico, the United States, Australia, and Canada with 4 % of investigations each. This agrees with what was stated in the reviews of the literature proposed by [51], [18], [19], [22]. The publications have high quality standards considering the high representativeness of the publications in the Q1 and Q2 quartiles, for the selected sources. The evidence of psychometric properties for the use of the instruments shows that reliability is presented in 80 % of the studies, with respect to validity only 48 % presented this evidence. The absence of a description of the procedures to demonstrate these psychometric properties in research is highlighted. The instrument proposed by [59] stands out, which was used in 9 investigations.

    This review of the literature makes it evident that information on validity testing must be collected in future studies; which is consistent with the findings obtained by [20] and [23] Significant progress has also been shown in the past two years with respect to the growth of publications evaluating CT. With respect to the variables most assessed in the instruments, algorithmic thinking, abstraction, and decomposition stand out, which is consistent with the reviews presented by [23] and [18]. The target population for this review was the adult population, very similar to the population addressed in the proposed scoping review [18] that focuses on post-secondary education. In this review, the authors evaluated 33 articles, where most assessment tools are designed for undergraduate students in computer science and engineering or for in-service teachers in these subjects.

    In the analysis of the 27 articles, it was identified that the instrument most frequently used was the Computational Thinking Scale CTS proposed by [59]. The variables assessed most frequently agree with those found in other reviews [18], [23] unlike the review by [20], where the variables related to programming were the most frequently assessed. With respect to evidence on reliability and validity analyses, future research is required to present in more detail the procedures and specify the methods used to determine these properties.

    In relation to teaching practice in the university context, there is a highlighted need to strengthen formative assessment processes of computational thinking. Currently, there is limited evidence on the psychometric properties of instruments reported in literature, highlighting the urgent need to develop and validate tools that serve as formal resources to support the evaluation of computational thinking at this level of academic training. Having valid instruments will enable instructors to objectively identify computational thinking skills, allowing for the adjustment of learning resources and pedagogical practices. The findings reveal opportunities for future research in the Latin American context, where instruments should consider cultural and social characteristics, as well as attitudinal elements that can influence the development of computational thinking.

    4. CONCLUSIONS

    From the review protocol, three elements associated with the variables under study, and the research questions, were proposed. How have the publications in the sample evolved with respect to the bibliometric variables? According to the analysis of bibliometric variables, the distribution over time of the research has increased since 2021. Regarding the authors in Figures 4 and 5 the relationships that exist between the authors are presented, where the coupling network emerges in 2019 and the author leading the network is Ata [34]. In relation to the geographical location where the research was carried out, Europe and Asia have the highest participation, and only one research is presented in Latin America. Regarding the classification of the journals based on the Scientific Journal Rankings (SJR) and Journal Citation Reports (JCR), 15 % of the publications found in SCOPUS and 22 % of those found in WOS were not classified into any quartile. It can be concluded that research on CT assessment in the adult population or university students has had greater application in university students, presenting deficiencies in the measurement of this construct in the teaching population, who play a leading role in teaching it to students. More research is needed in Latin America according to educational and cultural contexts.

    What are the instruments that have been proposed to measure Computational Thinking in the adult population? Analysis of instruments designed to measure CT in adult populations, particularly among in-service and preservice teachers, reveals a diverse range in their design and focus. Although Bebras Tasks are used with university students, explicit evidence on their variables evaluated, reliability, and validity is lacking in the presented literature. In contrast, the Computational Thinking Scale (CTS) and the CT Questionnaire (QCT) are more consistently applied with preservice and in-service teachers, primarily assessing skills and, in some cases, attitudes such as creativity and cooperativeness. These instruments typically use Likert scales and report evidence of reliability (Cronbach's Alpha, Kuder Richardson) and validity (content, criterion, construct).

    Other instruments, such as the CTt, the CT skills test, and the CT Test, focus on evaluating aptitudes and knowledge through multiple-choice formats. It is noteworthy that self-confidence in CT skills and teacher self-efficacy are key variables assessed by instruments such as the Self-confidence questionnaire on CT skills and the T-STEM CT. This highlights the importance of attitudinal and pedagogical factors in the integration of CT. The CCTt Callysto CT Test stands out for its broader approach, which evaluates skills, concepts, and practices, in addition to technological and data literacy. Although many instruments provide evidence of reliability and validity, gaps still exist, especially in the explicit reporting of all evidence for each adaptation or use of the instruments, as observed in some cases for Bebras Tasks and the T-STEM CT.

    What are the constructs or variables evaluated? The most frequently found skill is algorithmic thinking present in 36 % of the instruments, followed by abstraction with presence in 28 %, then decomposition with a frequency of 24 %, evaluation with 20 %, and finally problem solving with 16 %. Two instruments evaluated specific programming variables (loops, conditionals, operators, basic directions, repeat, repeat until, if, if/if not, while, and simple functions). Variables related to personal and social practices were also examined, including competencies such as teamwork, communication, spiritual intelligence, personal belief systems, teaching effectiveness, beliefs regarding anticipated teaching outcomes, the ability to analyze and manage data, apply information to problem-solving, design appropriate interventions, produce and assess work artifacts, identify underlying causes analytically, collaborate effectively, as well as attributes such as self-confidence in critical thinking, expectations, general self-confidence, optimism, resilience, and professional experience. The presence of variables that value CT from a domain other than the cognitive domain is highlighted.

    What are the psychometric properties of the instruments and what statistical methods were used for the analysis of the psychometric properties? It was identified that 21 studies presented evidence of reliability, highlighting the use of Cronbach's Alpha. Regarding validity, 12 studies showed evidence, and the most used method was confirmatory factor analysis (CFA). In the information collected, there is an absence of studies that evaluate CT in teachers considering approaches other than the assessment of the cognitive component, for example, evaluations that consider competencies to propose classroom practices that strengthen the development of CT in the classroom; furthermore, no studies were found in Latin America and the evidence of validity of the studies is scarce.

    In the future, research can be proposed to evaluate CT with measurement instruments that are exclusive to the teaching population and address not only cognitive aspects, but also to propose instruments that assess attitudes and aptitudes related to programming. Finally, this review contributes to future research processes, making evident the progress in the assessment of CT from measurement instruments in the adult population. It is necessary to continue deepening proposals that allow for the assessment of CT from perspectives additional to the cognitive domains.

     

    5. ACKNOWLEDGMENTS AND FUNDING

    We acknowledge the Doctorate in Educational Sciences of the University of Quindío, which provides financial support to student Milena Corrales Álvarez to pursue her doctoral studies.

    REFERENCES

    • arrow_upward [1] M. Téllez, “Pensamiento computacional: una competencia del siglo XXI,” Educ. Super., vol. 6, no. 1, pp. 23-32, Mar. 2019. http://www.scielo.org.bo/scielo.php?pid=S2518-82832019000100007&script=sci_arttext
    • arrow_upward [2] J. M. Wing, “Computational thinking,” Communic. ACM, vol. 49, no. 3, pp. 33-35, Mar. 2006. https://doi.org/10.1145/1118178.1118215
    • arrow_upward [3] M. Román González, “Codigoalfabetización y pensamiento computacional en Educación Primaria y Secundaria: validación de un instrumento y evaluación de programas,” Tesis doctoral, Universidad Nacional de Educación a Distancia, Madrid, España, 2016. https://hdl.handle.net/20.500.14468/18800
    • arrow_upward [4] C. Selby, and J. Woollard, “Computational Thinking: The Developing Definition,” University of Southampton, 2013. [Online]. Available: https://eprints.soton.ac.uk/356481/
    • arrow_upward[5] C. C. Selby, “Relationships: Computational thinking, Pedagogy of programming, And Bloom’s taxonomy,” in ACM Int. Conf. Proceeding Ser., London, UK, Nov. 9-11, 2015, pp. 80-87. https://doi.org/10.1145/2818314.2818315
    • arrow_upward [6] A. Csizmadia et al., Computational Thinking: A Guide for Teachers. London, UK: Computing at School, 2015. https://www.researchgate.net/publication/327302966__
    • arrow_upward [7] O. Astrachan, S. Hambrusch, J. Peckham, and A. Settle, “The present and future of computational thinking,” ACM SIGCSE Bull., vol. 41, no. 1, pp. 549-550, Mar. 2009. https://doi.org/10.1145/1539024.1509053
    • arrow_upward [8] F. J. García-Peñalvo, D. Reimann, M. Tuul, A. Rees, and I. Jormanainen, Taccle 3, O5: An overview of the most relevant literature on coding and computational thinking with emphasis on the relevant issues for teachers. Belgium: Taccle3 Consortium, 2016. https://doi.org/10.5281/zenodo.165123
    • arrow_upward [9] F. Llorens Largo, F. J. García Peñalvo, X. Molero Prieto, and E. Vendrell Vidal, “La enseñanza de la informática, la programación y el pensamiento computacional en los estudios preuniversitarios,” Educ. Knowl. Soc., vol. 18, no. 2, pp. 7-17, Aug. 2017. https://doi.org/10.14201/eks2017182717
    • arrow_upward [10] M. Román-González, J. C. Pérez-González, and C. Jiménez-Fernández, “Which cognitive abilities underlie computational thinking? Criterion validity of the Computational Thinking Test,” Comput. Human Behav., vol. 72, pp. 678-691, Jul. 2017. https://doi.org/10.1016/j.chb.2016.08.047
    • arrow_upward [11] M.-J. Tsai, J.-C. Liang, and C.-Y. Hsu, “The Computational Thinking Scale for Computer Literacy Education,” J. Educ. Comput. Res., vol. 59, no. 4, pp. 579-602, Nov. 2021. https://doi.org/10.1177/0735633120972356
    • arrow_upward [12] A. V. Aho, “Computation and computational thinking,” Comput. J., vol. 55, no. 7, pp. 833-835, Jul. 2012. https://doi.org/10.1093/comjnl/bxs074
    • arrow_upward [13] A. Piatti et al., “The CT-cube: A framework for the design and the assessment of computational thinking activities,” Comput. Hum. Behav. Reports, vol. 5, p. 100166 Mar. 2022. https://doi.org/10.1016/j.chbr.2021.100166
    • arrow_upward [14] L. Sun, X. You, and D. Zhou, “Evaluation and development of STEAM teachers’ computational thinking skills: Analysis of multiple influential factors,” Educ. Inf. Technol., vol. 28, pp. 14493-14527, Apr. 2023. https://doi.org/10.1007/s10639-023-11777-7
    • arrow_upward [15] C. Aumgri, and S. Petsangsri, “Computational thinking for preservice teachers in Thailand: A confirmatory factor analysis,” Espacios, vol. 40, no. 29, pp. 6-7, Sep. 2019. https://www.revistaespacios.com/a19v40n29/a19v40n29p12.pdf
    • arrow_upward [16] S. Kılıç, S. Gökoğlu, and M. Öztürk, “A Valid and Reliable Scale for Developing Programming-Oriented Computational Thinking,” J. Educ. Comput. Res., vol. 59, no. 2, pp. 257-286, Oct. 2020. https://doi.org/10.1177/0735633120964402
    • arrow_upward [17] D. E. Sondakh, K. Osman, and S. Zainudin, “A Proposal For Holistic Assessment Of Computational Thinking for Undergraduate: Content Validity,” European Journal of Educational Research,  vol. 9, no. 1, pp. 33-50, Jan. 2020. https://doi.org/10.12973/eu-jer.9.1.33
    • arrow_upward [18] C. Lu, R. Macdonald, B. Odell, V. Kokhan, C. Demmans Epp, and M. Cutumisu, “A scoping review of computational thinking assessments in higher education,” J. Comput. High. Educ., vol. 34, no. 2, pp. 416-461, Jan. 2022. https://doi.org/10.1007/s12528-021-09305-y
    • arrow_upward [19] M. Cutumisu, C. Adams, and C. Lu, “A Scoping Review of Empirical Research on Recent Computational Thinking Assessments,” J. Sci. Educ. Technol., vol. 28, no. 6, pp. 651-676, Nov. 2019. https://doi.org/10.1007/s10956-019-09799-3
    • arrow_upward [20] X. Tang, Y. Yin, Q. Lin, R. Hadad, and X. Zhai, “Assessing computational thinking: A systematic review of empirical studies,” Comput. Educ., vol. 148, p. 103798, Apr. 2020. https://doi.org/10.1016/j.compedu.2019.103798
    • arrow_upward [21] J. Acevedo-Borrega, J. Valverde-Berrocoso, and M. del C. Garrido-Arroyo, “Computational Thinking and Educational Technology: A Scoping Review of the Literature,” Educ. Sci., vol. 12, no. 1, Jan. 2022. https://doi.org/10.3390/educsci12010039
    • arrow_upward [22] M. Tekdal, “Trends and development in research on computational thinking,” Educ. Inf. Technol., vol. 26, no. 5, pp. 6499-6529, Jun. 2021. https://doi.org/10.1007/s10639-021-10617-w
    • arrow_upward [23] H. İ. Haseskİ, and U. İlİc, “An Investigation of the Data Collection Instruments Developed to Measure Computational Thinking,” Informatics in Education, vol. 18, no. 2, pp. 297-319, Oct. 2019. https://doi.org/10.15388/infedu.2019.14
    • arrow_upward [24] A. Anastasi, and S. Urbina, Test Psicológicos. Mex.: Pearson Ed., 1998.
    • arrow_upward [25] L. R. Aiken, Tests psicológicos y evaluación. Mex.: Pearson Ed., 2003.
    • arrow_upward [26] J. Aliaga Tovar, “Psicometria: Tests Psicométricos, Confiabilidad y Validez,” Psicol. Tópicos Actual., vol. 8, pp. 85-108, 2007. https://gc.scalahed.com/recursos/files/r161r/w25645w/Psicometria_ Test_%20psicometricos_confiabilidad_validez_RS6.pdf
    • arrow_upward [27] R. J. Cohen, and M. E. Swerdlik, Pruebas y evaluación psicológicas: Introducción a las pruebas y a la medición. McGraw-Hill, 2001.
    • arrow_upward [28] E. Almanasreh, R. Moles, and T. F. Chen, “Evaluation of methods used for estimating content validity,” Res. Soc. Adm. Pharm., vol. 15, no. 2, pp. 214-221, Feb. 2019. https://doi.org/10.1016/j.sapharm.2018.03.066
    • arrow_upward [29] A. C. Tricco et al., “PRISMA extension for scoping reviews (PRISMA-ScR): Checklist and explanation,” Ann. Intern. Med., vol. 169, no. 7, pp. 467-473, Sep. 2018. https://doi.org/10.7326/M18-0850
    • arrow_upward [30] N. J. Van Eck, and L. Waltman, VOSviewer Manual. Leiden, NL: Univeristeit Leiden, 2023. https://www.vosviewer.com/documentation/Manual_VOSviewer_1.6.20.pdf
    • arrow_upward [31] V. Dolgopolovas, T. Jevsikova, V. Dagiené, and L. Savulioniené, “Exploration of computational thinking of software engineering novice students based on solving computer science tasks,” Int. J. Eng. Educ., vol. 32, no. 3, pp. 1107-1116, 2016. https://dialnet.unirioja.es/servlet/articulo?codigo=6910621
    • arrow_upward[32] M. S. Günbatar, and H. Bakırcı, “STEM teaching intention and computational thinking skills of pre-service teachers,” Educ. Inf. Technol., vol. 24, no. 2, pp. 1615-1629, Dec. 2019. https://doi.org/10.1007/s10639-018-9849-5
    • arrow_upward [33] F. M. Esteve-Mon, J. Adell-Segura, M. Á. Llopis Nebot, G. Valdeolivas Novella, and J. Pacheco Aparicio, “The development of computational thinking in student teachers through an intervention with educational robotics,” J. Inf. Technol. Educ. Innov. Pract., vol. 18, pp. 139-152, 2019. https://doi.org/10.28945/4442
    • arrow_upward [34] R. Ata, and M. Çevik, “Understanding predictor effects of computational thinking skills and media and technology use and attitudes of pre-service teachers for STEM awareness,” Kedi J. Educ. Policy, vol. 17, no. 1, pp. 99-121, 2020. https://www.researchgate.net/publication/342551812_Understanding_predictor_effects_of_computational_thinking_ skills_and_media_and_technology_use_and_attitudes_of_pre-service_teachers_for_STEM_awareness
    • arrow_upward[35] Ü. Çakıroğlu, and S. Kiliç, “Assessing teachers’ PCK to teach computational thinking via robotic programming,” Interact. Learn. Environ., vol. 31, no. 2, pp. 818-835, Sep. 2020. https://doi.org/10.1080/10494820.2020.1811734
    • arrow_upward [36] F.-H. Tsai, H.-S. Hsiao, K.-C. Yu, and K.-Y. Lin, “Development and effectiveness evaluation of a STEM-based game-design project for preservice primary teacher education,” Int. J. Technol. Des. Educ., vol. 32, no. 5, pp. 2403-2424, Nov. 2022. https://doi.org/10.1007/s10798-021-09702-5
    • arrow_upward [37] K. Bati, and M. İkbal Yetişir, “Examination of Turkish Middle School STEM Teachers’ Knowledge about Computational Thinking and Views Regarding Information and Communications Technology,” Comput. Sch., vol. 38, no. 1, pp. 57-73, Mar. 2021. https://doi.org/10.1080/07380569.2021.1882206
    • arrow_upward [38] G. Rodríguez-Abitia, M. S. Ramírez-Montoya, E. O. López-Caudana, and J. M. Romero-Rodríguez, “Factores para el desarrollo del pensamiento computacional en estudiantes de pregrado,” Campus Virt., vol. 10, no. 2, pp. 153-164, Jul. 2021. http://www.uajournals.com/campusvirtuales/journal/19/10.pdf
    • arrow_upward [39] D. C. Boulden, A. Rachmatullah, K. M. Oliver, and E. Wiebe, “Measuring in-service teacher self-efficacy for teaching computational thinking: development and validation of the T-STEM CT,” Educ. Inf. Technol., vol. 26, no. 4, pp. 4663-4689, Mar. 2021. https://doi.org/10.1007/s10639-021-10487-2
    • arrow_upward [40] V. Amnouychokanant, S. Boonlue, S. Chuathong, and K. Thamwipat, “Online Learning Using Block-based Programming to Foster Computational Thinking Abilities During the COVID-19 Pandemic,” Int. J. Emerg. Technol. Learn., vol. 16, no. 13, pp. 227-247, Jul. 2021. https://doi.org/10.3991/ijet.v16i13.22591
    • arrow_upward [41] J. Bilbao, E. Bravo, O. García, C. Rebollar, and C. Varela, “Study to find out the perception that first year students in engineering have about the Computational Thinking skills, and to identify possible factors related to the ability of Abstraction,” Heliyon, vol. 7, no. 2, p. e06135, Feb. 2021. https://doi.org/10.1016/j.heliyon.2021.e06135
    • arrow_upward [42] R. Çakır, H. Şahin, H. Balci, and M. Vergili, “The effect of basic robotic coding in-service training on teachers’ acceptance of technology, self-development, and computational thinking skills in technology use,” J. Comput. Educ., vol. 8, no. 2, pp. 237-265, Jan. 2021. https://doi.org/10.1007/s40692-020-00178-1
    • arrow_upward [43] O. Kocak, M. Coban, A. Aydin, and N. Cakmak, “The mediating role of critical thinking and cooperativity in the 21st century skills of higher education students,” Think. Ski. Creat., vol. 42, p. 100967, Dec. 2021. https://doi.org/10.1016/j.tsc.2021.100967
    • arrow_upward [44] Y. Özkan Hidiroglu, and C. Naci Hidiroglu, “The Relationship between Mathematics Teachers’ Mind Types and Computational Thinking Skills,” Pamukkale Üniv. Eğitim Fakültesi Dergisi, vol. 52, pp. 301-325, 2021. https://doi.org/10.9779/pauefd.696511
    • arrow_upward [45] K.-D. Boom, M. Bower, J. Siemon, and A. Arguel, “Relationships between computational thinking and the quality of computer programs,” Educ. Inf. Technol., vol. 27, no. 6, pp. 8289-8310, Mar. 2022. https://doi.org/10.1007/s10639-022-10921-z
    • arrow_upward [46] M. Cutumisu, C. Adams, F. Glanfield, C. Yuen, and C. Lu, “Using Structural Equation Modeling to Examine the Relationship Between Preservice Teachers’ Computational Thinking Attitudes and Skills,” IEEE Trans. Educ., vol. 65, no. 2, pp. 177-183, May. 2022. https://doi.org/10.1109/TE.2021.3105938
    • arrow_upward [47] S.-C. Kong, and M. Lai, “Effects of a teacher development program on teachers’ knowledge and collaborative engagement, and students’ achievement in computational thinking concepts,” Br. J. Educ. Technol., vol. 55, no. 2, pp. 489-512, Jul. 2022. https://doi.org/10.1111/bjet.13256
    • arrow_upward [48] M. Lafuente Martínez, O. Lévêque, I. Benítez, C. Hardebolle, and J. Dehler Zufferey, “Assessing Computational Thinking: Development and Validation of the Algorithmic Thinking Test for Adults,” J. Educ. Comput. Res., vol. 60, no. 6, pp. 1436-1463, Feb. 2022. https://doi.org/10.1177/07356331211057819
    • arrow_upward [49] B. Hassan Majeed, L. Fouad Jawad, and H. T. Salim Alikirabi, “Computational Thinking (CT) Among University Students,” Int. J. Interact. Mob. Technol., vol. 16, no. 10, pp. 244-252, May. 2022. https://doi.org/10.3991/ijim.v16i10.30043
    • arrow_upward [50] Ş. Şen, “Relations between preservice teachers’ self-efficacy, computational thinking skills and metacognitive self-regulation,” Eur. J. Psychol. Educ., vol. 38, pp. 1251-1269, Nov. 2022. https://doi.org/10.1007/s10212-022-00651-8
    • arrow_upward [51] S. Syafril, T. Rahayu, and G. Ganefri, “Prospective science teachers’ self-confidence in computational thonking skills,” J. Pendidik. IPA Indones., vol. 11, no. 1, pp. 119-128, Mar. 2022. https://doi.org/10.15294/jpii.v11i1.33125
    • arrow_upward [52] X. Zhou, and C. W. Tsai, “The Effects of Socially Shared Regulation of Learning on the Computational Thinking, Motivation, and Engagement in Collaborative Learning by Teaching,” Educ. Inf. Technol., vol. 28, pp. 8135-8152, Dec. 2022. https://doi.org/10.1007/s10639-022-11527-1
    • arrow_upward [53] A. Rachmatullah, and E. N. Wiebe, “Changes and Sources of Changes of Middle School Teachers’ Self-efficacy for Teaching Science in A Computationally Rich Environment: A Mixed-Methods Study,” J. Sci. Teacher Educ., vol. 34, no. 2, pp. 132-156, Apr. 2022. https://doi.org/10.1080/1046560X.2022.2035990
    • arrow_upward [54] A. Kaur, and K. Kaur Chahal, “Exploring Personality and Learning Motivation Influences on Students’ Computational Thinking Skills in Introductory Programming Courses,” J. Sci. Educ. Technol., vol. 32, pp. 778-792, May. 2023. https://doi.org/10.1007/s10956-023-10052-1
    • arrow_upward [55] C. Chookhampaeng, C. Kamha, and S. Chookhampaeng, “Problems and Needs Assessment to Learning Management of Computational Thinking of Teachers at the Lower Secondary Level,” J. Curric. Teach., vol. 12, no. 3, pp. 172-178, May. 2023. https://doi.org/10.5430/jct.v12n3p172
    • arrow_upward [56] V. J. Shute, C. Sun, and J. Asbell-Clarke, “Demystifying computational thinking,” Educ. Res. Rev., vol. 22, pp. 142-158, Nov. 2017. https://doi.org/10.1016/j.edurev.2017.09.003
    • arrow_upward [57] D. Weintrop et al., “Defining Computational Thinking for Mathematics and Science Classrooms,” J. Sci. Educ. Technol., vol. 25, no. 1, pp. 127-147, Feb. 2016. https://doi.org/10.1007/s10956-015-9581-5
    • arrow_upward [58] S. Grover, and R. Pea, “Computational Thinking: A Competency Whose Time Has Come,” in Computer Science Education: Perspectives on Teaching and Learning in School, S. Sentance, E. Barendsen, and C. Schulte, Eds., London: Bloomsbury Academic, 2018, pp 19-38. https://doi.org/10.5040/9781350057142.ch-003
    • arrow_upward [59] Ö. Korkmaz, R. M. Çaki, and M. Yaşar Özden, “A validity and reliability study of the computational thinking scales (CTS),” Comput. Human Behav., vol. 72, pp. 558-569, Jul. 2017. https://doi.org/10.1016/j.chb.2017.01.005
    • arrow_upward [60] J. Voogt, P. Fisser, J. Good, P. Mishra, and A. Yadav, “Computational thinking in compulsory education: Towards an agenda for research and practice,” Educ. Inf. Technol., vol. 20, no. 4, pp. 715-728, Jun. 2015. https://doi.org/10.1007/s10639-015-9412-6
    • arrow_upward [61] A. Yadav, C. Stephenson, and H. Hong, “Computational thinking for teacher education,” Commun. ACM, vol. 60, no. 4, pp. 55-62, Mar. 2017. https://doi.org/10.1145/2994591
    • arrow_upward [62] M. S. Günbatar, “Computational thinking within the context of professional life: Change in CT skill from the viewpoint of teachers,” Educ. Inf. Technol., vol. 24, no. 5, pp. 2629-2652, Apr. 2019. https://doi.org/10.1007/s10639-019-09919-x
    • arrow_upward [63] S. C. Kong, and M. Lai, “Validating a computational thinking concepts test for primary education using item response theory: An analysis of students’ responses,” Comput. Educ., vol. 187, p. 104562, Oct. 2022. https://doi.org/10.1016/j.compedu.2022.104562
    • arrow_upward[64] L. Sun, L. Hu, D. Zhou, and W. Yang, “Evaluation and developmental suggestions on undergraduates’ computational thinking: a theoretical framework guided by Marzano’s new taxonomy,” Interact. Learn. Environ., vol. 31, no. 10, pp. 6588-6610, Mar. 2022. https://doi.org/10.1080/10494820.2022.2042311
    • arrow_upward [65] R. Çakır, Ö. Korkmaz, Ö. İdil, and F. Uğur Erdoğmuş, “The effect of robotic coding education on preschoolers’ problem solving and creative thinking skills,” Think. Ski. Creat., vol. 40, p. 100812, Jun. 2021. https://doi.org/10.1016/j.tsc.2021.100812
    •  

      CONFLICTS OF INTEREST

      The authors declare no conflicts of interest that could influence the work presented in this article.

      AUTHOR CONTRIBUTIONS

      The authorship recognition of this article is based on the contributions of each author:

      Milena Corrales-Álvarez: methodology, conceptualization, research, writing, review, and editing.

      Ángela María Muñoz Muñoz: methodology, conceptualization, research, writing, review, and editing.

      Sergio Augusto Cardona-Torres: conceptualization, research, writing, review, and editing.

      APPENDIX

      Authors' H-index as of May 21, 2024
      Article Authors h-index Google Scholar
      A proposal for holistic assessment of computational thinking for undergraduate: Content validity [17] Debby Erce Sondakh 6
      Kamisah Osman 45
      Suhaila Zainudin 16
      A Valid and Reliable Scale for Developing Programming-Oriented Computational Thinking [16] Kılıç, Servet No evidence
      Seyfullah Gökoğlu 12
      Mücahit Öztürk 10
      Assessing Computational Thinking: Development and Validation of the Algorithmic Thinking Test for Adults [48] Marc Lafuente Martínez No evidence
      Olivier Leveque 21
      Isabel Benítez Baena 19
      Cécile Hardebolle No evidence
      Jessica Dehler Zufferey 14
      Changes and Sources of Changes of Middle School Teachers' Self-efficacy for Teaching Science in A Computationally Rich Environment: A Mixed-Methods Study [53] Arif Rachmatullah 17
      Eric Wiebe 48
      Computational Thinking (CT) Among University Students [49] Ban Hassan Majeed 16
      Lina Fouad Jawad 7
      Haider TH. Salim ALRikabi 38
      Computational thinking for preservice teachers in Thailand: A confirmatory factor analysis [15] Charinthorn Aumgri 3
      Sirirat Petsangsri 11
      Computational thinking within the context of professional life: Change in CT skill from the viewpoint of teachers [62] Mustafa Serkan Günbatar 13
      Development and effectiveness evaluation of a STEM-based game-design project for preservice primary teacher education [36] Fu-Hsing Tsai 13
      Hsien-Sheng Hsiao 25
      Kuang-Chao Yu 18
      Kuen-Yi Lin 22
      Effects of a teacher development program on teachers' knowledge and collaborative engagement, and students' achievement in computational thinking concepts [47] Siu-Cheung Kong 37
      Ming Lai No evidence
      Examination of Turkish Middle School STEM Teachers' Knowledge about Computational Thinking and Views Regarding Information and Communications Technology [37] Kaan BATI 12
      Mehmet İkbal Yetişir No evidence
      Exploration of Computational Thinking of Software Engineering Novice Students Based on Solving Computer Science Tasks [31] Vladimiras Dolgopolovas 9
      Tatjana Jevsikova No evidence
      Valentina Dagienė No evidence
      Loreta Savulionienė No evidence
      Factors for the development of computational thinking in undergraduate students [38] Guillermo Rodríguez Abitia 9
      María-Soledad Ramírez Montoya 60
      Edgar Omar López Caudana 16
      José-María Romero-Rodríguez 34
      Measuring in-service teacher self-efficacy for teaching computational thinking: development and validation of the T-STEM CT [39] Danielle Cadieux Boulden 12
      Arif Rachmatullah 17
      Kevin M. Oliver No evidence
      Eric Wiebe 48
      Online Learning Using Block-based Programming to Foster Computational Thinking Abilities During the COVID-19 Pandemic [40] Virawan Amnouychokanant 2
      Surapon Boonlue 6
      Saranya Chuathong No evidence
      Kuntida Thamwipat No evidence
      Prospective Science Teachers’ Self-Confidence In Computational Thinking Skills [51] Syafrimen Syafril 19
      Titik Rahayu 12
      Ganefri Ganefri 15
      Relations between preservice teachers' self-efficacy, computational thinking skills and metacognitive self-regulation [50] Şen Şenol No evidence
      Relationships between computational thinking and the quality of computer programs [45] Kay-Dennis Boom 4
      Matt Bower 42
      Jens Siemon No evidence
      Amaël Arguel 17
      Study to find out the perception that first year students in engineering have about the Computational Thinking skills, and to identify possible factors related to the ability of Abstraction [41] Javier Bilbao No evidence
      Eugenio Bravo No evidence
      Olatz García No evidence
      Carolina Rebollar No evidence
      Concepción Varela No evidence
      The development of computational thinking in student teachers through an intervention with educational robotics [33] Francesc M. Esteve Mon 34
      Jordi Adell Segura 49
      María Ángeles Llopis Nebot 10
      Gracia Valdeolivas Novella 4
      Julio Pacheco Aparicio No evidence
      The effect of basic robotic coding in-service training on teachers' acceptance of technology, self-development, and computational thinking skills in technology use [42] Recep Çakır 34
      Hüsnü Şahin No evidence
      Hasan Balci No evidence
      Mehmet Vergili 3
      The Effects of Socially Shared Regulation of Learning on the Computational Thinking, Motivation, and Engagement in Collaborative Learning by Teaching [52] Zhou, Xiaodan No evidence
      Tsai Chia-Wen 32
      The mediating role of critical thinking and cooperativity in the 21st century skills of higher education students [43] Omer Kocak 12
      Murat Coban, No evidence
      Abdulkerim Aydin 4
      Nermin Cakmak 10
      The Relationship between Mathematics Teachers' Mind Types and Computational Thinking Skills [44] Yeliz Özkan Hidiroğlu 8
      Çağlar Naci Hıdıroğlu 16
      Understanding predictor effects of computational thinking skills and media and technology use and attitudes of pre-service teachers for STEM awareness [34] Ridvan Ata 11
      Çevik, Mustafa 17
      Using Structural Equation Modeling to Examine the Relationship Between Preservice Teachers' Computational Thinking Attitudes and Skills [46] Maria Cutumisu 29
      Catherine Adams 23
      Florence Glanfield No evidence
      Levina Yuen 3
      Chang Lu 8
      Exploring Personality and Learning Motivation Influences on Students’ Computational Thinking Skills in Introductory Programming Courses [54] Amanpreet Kaur No evidence
      Kuljit Kaur Chahal 9
      Problems and Needs Assessment to Learning Management of Computational Thinking of Teachers at the Lower Secondary Level [55] Chowwalit Chookhampaeng 4
      Chantraporn Kamha No evidence
      Sumalee Chookhampaeng 10