Pedagogical practices and civic knowledge and engagement in Latin America: Multilevel analysis using ICCS data

Abstract

This study explored the relationship between pedagogical practices and the civic knowledge and engagement of students from five Latin American and Caribbean countries, using a multilevel analysis of the 2016 International Civic and Citizenship Education Study (ICCS). The results reveal the complex interplay of educational, sociodemographic, and attitudinal factors in shaping the civic knowledge and engagement of students. Extracurricular civic engagement showed a negative correlation with student civic knowledge and engagement as measured by the ICCS cognitive test. However, civic learning in school positively related to cognitive abilities, emphasizing the importance of integrating civic learning into the curriculum. Interestingly, students' educational aspirations emerged as a significant factor shaping their civic engagement, suggesting a strategy to foster high educational aspirations to enhance cognitive performance. Additionally, gender dynamics were evident in civic education, with girls consistently outperforming boys in all participating countries. The correlation between home literacy resources and test scores illuminated the significant role of home environments in academic achievement. Lastly, students’ attitudes towards political participation had a notable connection to civic knowledge outcomes, presenting an exciting avenue for future research. Collectively, these findings underscore the need for a comprehensive approach to civic education and further research to refine effective strategies.

1. Introduction

In a democratic society, the active participation of citizens in political matters is essential. A democracy requires individuals who can think critically, assume civic and citizen responsibilities, and make informed decisions regarding their representatives [1]. The primary responsibility for educating citizens and equipping them with the knowledge, skills, and abilities to actively participate in civic and political life lies with the school system [2,3]. This responsibility is especially important during the school years because it coincides with crucial developmental periods during which individuals form their civic beliefs, values, and commitments [[4], [5], [6]].

However, there are concerns about the low academic performance of students in civic knowledge and the apathy of young people toward political participation [7,8], which has led to questions about the effectiveness of schools in fostering civic and citizen competencies [9,10]. These questions emerge due to the lack of empirical evidence concerning the individual and institutional factors shaping the advancement of student civic learning and competencies [11].

This research aims to examine how pedagogical practices relate to the development of civic and citizenship competencies in Latin-American students, while considering individual characteristics of teachers, schools, and students. Data for this investigation is drawn from the 2016 cycle of the International Civic and Citizenship Education Study (ICCS) [12]. Specifically, in the study, we aim a) to assess how pedagogical practices, as a crucial component of a democratic school climate (DSC), relate to the development of civic and citizenship competencies of students, and b) to understand the intricate relationship between various educational, sociodemographic, and attitudinal factors and the civic knowledge and engagement among students.

The hypothesis that guides the research posits that the implementation of active teaching and learning practices within the school positively contributes to the development of civic and citizenship competencies in students. This hypothesis is supported by research on school effectiveness that suggests that teaching and learning practices, as well as the quality of relationships between students and teachers, and the organization of the school all play a role in fostering student outcomes [13,14]. Additionally, the hypothesis is rooted in Dewey's theory of experience-based learning [15] and Kolb's theory of experiential learning [16], which suggest that individuals tend to be more competent and engaged in their educational experiences when they are exposed to active teaching strategies within the school setting. Examples of such strategies include participating in open classroom debates or participating in service learning projects [17].

In general, this research aims to contribute to understanding of the role of schools in promoting civic education in Latin America and the importance of civic education in general. By investigating the relationship between pedagogical practices and the development of civic competencies of students, this research can be used to inform the development of effective civic education programs and initiatives in Latin America and beyond.

2. Review of the literature

2.1. The significance of civic knowledge and engagement

It is widely accepted in democratic societies that civic and citizenship education plays a pivotal role in the political sophistication of young people by equipping them with the necessary competencies and skills to participate in political and civic life [18]. Some studies have shown that students who receive high-quality civic education are more likely to understand public issues, view political engagement as a means of addressing community challenges, and participate in civic activities [19,20]. Students from urban, rural, poor, and minority backgrounds who receive high-quality civic education outperform their peers, according to research [20]. Furthermore, the efficacy of civic education has been examined in numerous studies, which concluded that civic education improves student political knowledge, political efficacy, attitudes and skills [21,22]. In civic and citizenship education, it is crucial to grasp the general conception of civic engagement of students and to identify possible understanding gaps so that teaching reaches students where they are and enables them to engage as citizens more fully and confidently [23]. Consequently, in this study, the knowledge and understanding of civic and citizenship-related issues by students is measured using their scores on the cognitive questionnaire from the 2016 cycle of the ICCS study.

2.2. The role of pedagogical practices in fostering civic and citizenship skills of students

Pedagogical practices, defined as the methods and strategies teachers use to facilitate learning [24], have been identified as pivotal factors within schools, playing a significant role in shaping students’ academic achievement and the acquisition of various competencies, including civic and citizen knowledge [15,16]. Consequently, teaching and learning practices play a crucial role in shaping the Democratic School Climate (DSC) conceptual framework [25,26], which has been developed to define the school characteristics that can help shape the development of civic and citizen competencies, as well as the attitudes and expectations of students toward political participation. These characteristics are integral to the multifaceted construct of school climate, which has implications for student learning outcomes and overall school functioning [[27], [28], [29]].

Since the 1980s, scholarly research has dedicated significant attention to exploring the connection between the school climate and student learning and academic achievement [30]. These studies have explored factors such as teacher motivation and commitment [31], attitudes toward learning [32], development of student identity and personality [33], dropout rates [34], violence and bullying [35], and performance on standardized tests [36].

According to Cohen et al. [37], the DSC is the unique character of each educational institution, encompassing teaching and learning practices, as well as “the patterns of students', parents', administrative staff's experiences, as well as the norms, goals, values, interpersonal relationships, and organizational structure” (p. 182).

Specifically, within the DSC framework, the dimension of pedagogical practices advocates primarily for the use of “active practices” that allow students to interact with their peers, teachers, and the community through activities such as community service, participation in school governance processes, and decision-making, as well as debates and discussions on political and social events [15,16,38].

Other dimensions included in the DSC conceptual framework (Fig. 1) are the following: social order, referring to the norms, values and social conventions established for harmonious and safe co-existence in the community [25]; social relationships, where teachers and students can foster the development of social competencies through tolerance [39], social justice [40], respect for the opinions of others [41], and cooperation [15]; and organizational structure, which addresses characteristics such as institution size and the modality of “segregation by ability”, playing an important role in the development of civic and citizenship competencies [13,14].

Fig. 1.

Dimension of the democratic climate of the school.

The DSC, or “democratic learning environment” [12], occupies a prominent position within the ICCS evaluation framework, as it is recognized as a determining factor in the development of civic and citizen competencies by students. This recognition allows the conceptual dimensions of the DSC described previously to be operationalized through various measurements collected via questionnaires administered to teachers and students [12].

Our decision to focus on Latin American countries that participated in the ICCS was primarily driven by the need to address the dearth of empirical research in this region. Latin America has a rich and diverse sociopolitical and educational landscape, but it remains underrepresented in global civic and citizenship research [[42], [43], [44],44]. By focusing on this region, we aim to contribute valuable empirical data to the existing body of knowledge and shed light on the unique educational challenges and opportunities within these countries.

Additionally, the shared historical, cultural, and linguistic ties among Latin American countries allow for meaningful comparative analysis. This comparative approach is particularly valuable to us as Colombian researchers. By examining diverse educational contexts, we can glean insights into the successes and challenges faced by different nations. This understanding not only allows us to identify effective practices that might be adapted to the Colombian context, but also fosters a broader perspective.

Specifically, our research aligns with and aims to inform the strategies and policies of Colombia's National Government, especially those implemented by the Ministry of National Education. These policies are designed to provide high-quality education to all students, regardless of their socio-economic backgrounds, nurturing essential competencies for life, co-existence, productivity, and lifelong learning [1].

Furthermore, our findings can also offer insights to researchers and policymakers in other countries in the region. They can utilize our research to comprehend the potential implications of various educational practices and policies, enabling them to make informed decisions that enhance their own educational landscapes.

3. Methods

3.1. Sample

This research used data from five countries in the Latin American and Caribbean (LAC) region that participated in the 2016 cycle of ICCS. The ICCS is a large-scale international study conducted by the International Association for the Evaluation of Educational Achievement (IEA) (IEA, 2020), which evaluates the civic and citizen knowledge, competencies, and attitudes of 13–14-year-old students from 24 countries. Table 1 shows the characteristics of the study sample.

Table 1.

Sample characteristics of LAC countries participating in ICCS 2016.

Country	N schools	N students	N teachers
Chile	356	5250	1452
Colombia	150	5609	1580
Perú	206	5166	2348
Dominican Republic	127	3937	754
México	213	5526	1918

3.2. Instruments

The ICCS 2016 study administered several instruments to collect data on students’ civic and citizenship knowledge, attitudes, and engagement. Considering that the conceptual framework of ICCS 2016 posits that cognitive and affective outcomes are shaped by context, the study also gathered data on individual, family, and community characteristics of students, as well as their interactions with peers and teachers, and the prevailing school climate [45]. These instruments include.

3.3. International student questionnaire (ISQ)

This questionnaire was designed to collect contextual information as well as aspects related to students' attitudes and engagement. The ISQ comprised 179 items, 22 of which were aimed at collecting student background information, and 157 were designed to measure affective-behavioral domains. An additional 26 items were optional, allowing national testing centers to decide whether to include them. The ISQ covered various areas, including personal details, home and family background, out-of-school activities, school experiences, views on citizens and society, rights and responsibilities, institutions and society, societal participation, and religious beliefs and practices. While most items were developed by the international study center, national centers also contributed new items or modifications. The questionnaire also offered international options on students’ ethnicity, household composition, and religion.

3.4. Cognitive student test

The cognitive test is the primary instrument used in the ICCS. This test measures students' knowledge and understanding of civic and citizenship concepts and students’ ability to analyze civic issues, interpret political information, and apply democratic principles to real-world scenarios. The test is comprised of 79 multiple-choice items and nine open-ended tasks. Multiple-choice items present a question or statement and ask the student to choose the correct or best answer from four options.

The open-ended tasks require students to provide a written response to a prompt. These tasks often involve real-life scenarios or issues, requiring students to express an opinion, justify a viewpoint, or propose a solution. The 88 items were divided into eight booklets, each containing three of the eight 11-item clusters, using a balanced rotated design [46]. Each student was assigned one of those booklets to be completed within a 45-min time limit.

The test followed a rigorous development and testing process, including item development, pilot testing, item analysis, and scaling to ensure the reliability and validity of the instrument. Additionally, the ICCS used advanced psychometric models, including the Rasch model and multiple matrix sampling design, to ensure fair and accurate comparisons across countries. Due to space limitations in this paper, we cannot provide a detailed account of all the psychometric properties. For a comprehensive understanding of these properties, readers are directed to the ICCS 2016 Technical Report [46] and the ICCS 2016 Assessment Framework [12].

3.5. National context survey

This online survey was directed towards National Research Coordinators, who were tasked with providing information about their country's approach to civic and citizenship education at a national level. The questionnaire, consisting of 29 items, was divided into four sections. These sections were designed to collect data about the country's education system, the inclusion of civic and citizenship education in the curriculum, information about teachers and their education, as well as assessments and quality assurance measures.

3.6. Teacher questionnaire

The teacher questionnaire consisted of 29 questions with a total of 181 items and was designed to gather a wide range of information. It collected demographic data about teachers, including their age, gender, years of teaching experience, educational qualifications, and experiences with professional development. A series of Likert scale items then probed teachers’ attitudes and perceptions about civic and citizenship education, their teaching practices, the school climate, and their professional development experiences, asking them to indicate their level of agreement with various statements.

Additionally, multiple-choice items provided insights into topics such as the pedagogical methods teachers used, their perspectives on civic and citizenship education, and their perceptions of student behavior and attitudes. To capture more nuanced and detailed information, the questionnaire also included open-ended questions, enabling teachers to describe specific experiences, challenges, or successes they had encountered in their teaching of civic and citizenship education. The teacher questionnaire played a crucial role in the ICCS, providing essential contextual information to help interpret the student assessment results.

3.7. School questionnaire

The school questionnaire was completed by school principals and aimed to gather information about the school and local community context, as well as opportunities for students to engage in civic activities beyond the school environment. The school questionnaire consisted of 21 questions with a total of 106 items, divided into five sections. These sections focused on the principal's career, the school environment, the local community, the implementation of civic and citizenship education at the school, and basic school characteristics such as size, number of teachers, and location. The questionnaire also probed aspects of the local community context, such as the resources available to students and issues of social tension within the community. Some constructs were assessed both in the school and teacher questionnaires to gather diverse perspectives on these issues.

3.8. Regional student instruments

The regional questionnaires, administered in Europe and Latin America, were designed to capture students' views on political and social issues, their individual experiences, and their potential reactions to specific situations. The Latin American questionnaire comprised nine questions with a total of 72 Likert scale items. These items probed students' attitudes towards government leadership, with statements such as “It is better that government leaders make decisions without consulting anyone,” and “The most important opinion in the country should be that of the president.” The questionnaire also delved into students’ perspectives on public service and government, with items like “It is acceptable for a public official to accept bribes when their salary is very low,” and “Good candidates give personal benefits to voters in exchange for their vote.” These questionnaires, by capturing a broad spectrum of student opinions, provide valuable insights into the political and social attitudes of the younger generation in these regions.

In conclusion, the combination of these instruments provides a rich data set to analyze the relationship between civic education practices, school climate, and students’ civic and citizenship outcomes of students. All study data and documentation are available in the IEA data repository at https://www.iea.nl/data-tools/repository/iccs.

3.9. Analysis plan

The ICCS uses a two-stage sampling procedure. In the first stage, educational institutions are randomly selected, and in the second stage, the target group of students and teachers is selected. The target group is eighth grade students (or ninth grade in countries where the average age of students in eighth grade is less than 13.5 years) [47] and the teachers who teach in these grades.

Because of the sampling methodology employed, the analysis was conducted using hierarchical linear modeling (HLM) [48]. This statistical approach allows for the simultaneous examination of individual and institutional variables within a unified model to gain insights into how they jointly contribute to student performance and attitudes. The hierarchical regression model for the analysis is defined as

$$Yij=\mathit{\alpha }j+Xijn\mathit{\beta }j+Xmj+\mathit{\beta }j+U0j+\mathit{\epsilon }ij\text{,}$$ (1)

where, $Y_{ij}$ represents the dependent variable, ${\mathit{X}}_{ijn}$ is a vector of student-level variables with their corresponding regression coefficient vector ${\mathit{\beta }}_j$; ${\mathit{X}}_{jm}$ is a vector of school-level variables with their corresponding regression coefficient vector ${\mathit{\beta }}_j$; ${\mathit{U}}_{0j}$ is the residual term at the cluster (school) level; and ${\epsilon }_{ij}$ is the residual at the student level. It is assumed that the mean is zero for both residual terms and that the variance is normally distributed at each level. In hierarchical linear models, the explained variance must be estimated separately at each level, with the estimate based on a comparison of each prediction model with the null baseline model, without predictor variables.

$$Yij=\mathit{\alpha }j+U0nullj+\mathit{\epsilon }nullij\text{.}$$ (2)

In this case, the residual term ${\mathit{U}}_{0null}$ j represents the estimated variance in ${\mathit{Y}}_{ij}$ between clusters and ${\mathit{\epsilon }}_{nullij}$ is an estimate of the variance between students $i$ within the clusters. Given the complex design of the ICCS study, the analysis was carried out using the statistical software HLM 8 [49]. This software is especially suitable for the analysis of large-scale assessment (LSA) data, where plausible values or sampling weights should be used to obtain unbiased statistics [50,51]. In addition, the R statistical language was used for some descriptive analyses and visualizations [52].

According to the guidelines for analyzing ICCS data [53], the analysis at the student level (level 1) involved applying a weight known as the ‘within-school student weight.’ This weight was calculated by multiplying the class and student level weight factors (WGTFAC2S x WGTADJ2S x WGTADJ3S). At the school level (level 2), a weight was applied known as ‘school weight’. For student data analysis, this weight was derived from the variables WGTFAC1 and WGTADJ1S, while for teacher level analysis, it was obtained by multiplying WGTFAC1 and WGTADJ1T. It is important to note that, as stated in the user guide, when conducting multilevel analysis with ICCS data, it is crucial to use software that normalizes the weights, ensuring that the sum of weights equals the sample size. This normalization process helps to maintain the integrity and representativeness of the data during analysis.

4. Variables

4.1. Dependent variable

Civic Knowledge and Engagement (SCKE). SCKE is a measure of students’ knowledge and understanding of concepts and issues related to civics and citizenship, as well as their beliefs, attitudes, and behaviors with respect to this domain [12]. The ICCS cognitive questionnaire measures four content domains of civic knowledge and engagement.

• •Civic Society and systems: This domain assesses students' knowledge of the institutions and processes of civic society, such as political parties, elections, and the media.
• •Civic principles: This domain assesses students' knowledge of the values and principles that underpin civic life, such as democracy, human rights, and the rule of law.
• •Civic participation: This domain assesses students' knowledge of ways in which citizens can participate in civic life, such as voting, volunteering, and contacting their elected representatives.
• •Civic identities: This domain assesses students' sense of civic identity, such as their feelings of belonging to a national or local community, and their sense of civic duty.

The SCKE is measured on a standardized scale with an average of 500 points and a standard deviation of 100 points. For each student in ICCS 2016, five plausible values were calculated. The five plausible values were included in the analysis. Table 2 and Fig. 2 provide a snapshot of the central tendency, variability, and precision of the data for each country. They include the number of observations (n), the mean value, standard error of the mean (SE mean), standard deviation (SD), and standard deviation of the standard error (SE SD).

Table 2.

Summary statistics of the cognitive test of the ICCS for LAC countries.

Country	n	Mean	S.E Mean	SD	SD S.E
Chile	5081	482.45	3.11	94.66	1.52
Colombia	5609	482.11	3.39	83.41	1.23
Dominican Republic	3937	381.36	3.04	81.05	1.52
Mexico	5526	467.04	2.54	84.01	1.17
Peru	5166	437.71	3.54	92.39	1.74

Fig. 2.

Mean scores on the ICCS cognitive test for Latin American and Caribbean (LAC) countries.

Compared to other participating nations in the 2016 ICCS, the five LAC countries recorded the lowest civic knowledge scores (Fig. 3).

Fig. 3.

of students at each level of civic knowledge in ICCS 2016.

4.2. Model predictors (level 2)

Pedagogical Practices. This measure consists of a set of predictors derived from the student and teacher questionnaires. Each predictor is reported on a scale with an average of 50 and a standard deviation of 10. Detailed information on the items used to create each variable can be found in Chapters 10 and 11 of the ICCS 2016 Technical Report [46], available at https://shorturl.at/lvR38.

Student Activities in the Community (T_STDCOM). This variable measures teachers’ perception of the extent to which students had taken part in activities carried out by the school in cooperation with external groups or organizations.

Openness in Classroom Discussions (OPDISC). This scale reflects the perceptions of the extent to which teachers foster independent thinking, encourage students to express their opinions, promote discussions on current political events, support students in expressing dissenting opinions, encourage dialogue with individuals with different views, and present multiple perspectives when discussing issues in class.

Civic Learning at School (S_CIVLRN). This scale assesses the perception of their learning of various topics of civic knowledge, including voting in elections, the legislative process, environmental protection, community problem solving, citizen rights protection, political issues in other countries, and economic principles.

Civic-Related Activities in Class (T_CIVCLAS). This scale measures the extent to which students engage in experiential learning by working on projects involving gathering information outside of school, collaborating in small groups on various topics, participating in role plays, discussing current issues, conducting research and analysis using online sources, and suggesting topics for future lessons.

Students' Civic Participation at School (S_SCHPART). This scale assesses the degree of students’ involvement in activities such as organized debates, voting for class representatives or school parliament, participating in decision making about school operations, participating in discussions at student assemblies, running as candidates for class representatives or school parliament, and participating in activities to promote environmental sustainability within the school.

Students’ Civic Participation in the Wider Community (S_COMPART). This scale measures student participation in various organizations, clubs, or groups, including political party or union-affiliated youth organizations, environmental action groups, human rights organizations, voluntary groups focused on community service, organizations involved in fundraising for social causes, youth campaigns for specific issues, and animal rights or animal welfare groups.

4.3. Control variables (level 1)

Additionally, to control for individual or family factors that could be related to cognitive outcomes, the analysis also included a series of measurements collected by the ICCS on students’ personal and family characteristics.

• •Student's gender (S_GENDER).
• •Highest level of education the student expects to achieve (S_ISCED.
• •Interest of the student in political and social issues (S_SINT).
• •Frequency with which the student discusses or talks about political or social issues outside of school (S_POLDISC).
• •Educational level of the mother and father (S_MISCED and S_FISCED).
• •Occupation of the mother and father (S_MISEI and S_FISEI).
• •Home literacy resources (S_HOMLIT)
• •
  Students' attitudes towards political participation. Composite scales that measure students' motivation to take an active role in political life as adults. This construct is measured through two scales:

  • o
    Students' Expected Electoral Participation (S_ELECPART). The scale assesses the expected future actions as adults, including their intentions to vote in local elections, vote in national elections, and seek information about candidates before casting their votes in elections.
  • o
    Students' Expected Active Political Participation (S_POLPART). The scale measures the future participation in political activities as adults, including their intention to help a candidate or party during an election campaign, join a political party, join a trade union, stand as a candidate in local elections and join an organization focused on political or social causes.

5. Results

In this section, we present the finding of the hierarchical linear modeling (HLM) analysis conducted using the HLM8 software. Before the analysis, a null model $\left(2\right)$ was run to assess the suitability of multilevel analysis. The intraclass correlation (ICC) value was satisfactory $\left({\chi }^2\left(119\right)=309.24,p<0.001,ICC=0.23\right)$, indicating that 23 % of the variation in civic knowledge resided at the group level. Consequently, the model outlined in $\left(1\right)$ was found suitable for our data. In the following section, we provide a comprehensive description of the statistically significant predictors of civic knowledge and engagement for each country. For interested readers, full tables with HLM results are included as supplementary material for further examination and exploration.

5.1. Chile

In Chile, student activities in the community $\left(b=-1.58,t\left(151\right)=-2.19,p=.031\right)$, civic learning $(b=4.11,t\left(151\right)=3.94,p<.001$), civic participation at school $\left(b=3.08,t\left(151\right)=2.12,p=.036\right)$, and civic participation in the wider community $b=-4.63,t\left(151\right)=-3.28,p=.001)$ significantly predicted their score in the cognitive test, with both students’ activities in the community and civic participation in the wider community indicating a statistically significant negative relationship.

Among the slopes, gender, home literacy resources, the highest level of education the student expects to achieve, the interest of the student in political and social issues, and the student's expected electoral participation $(b=2.30,t\left(1339\right)=17.50,p<.001$) showed a statistically significant positive relationship with the outcome of the cognitive test. The student's expected active political participation showed a statistically significant negative relationship $(b=-2.08,t\left(38\right)=13.31,p<.001$).

5.2. Colombia

In Colombia, openness in classroom discussions $\left(b=540,t\left(54\right)=4.46,p<.001\right)$ and civic participation in the wider community $\left(b=-6.06,t\left(114\right)=-4.068,<.001\right)$ significantly predicted their score in the cognitive test, with civic participation in the wider community indicating a statistically significant negative relationship.

Among the slopes, home literacy resources, the highest level of education the student expects to achieve, and the student's expected electoral participation $\left(b=2.53,t\left(30\right)=12.76,p<.001\right)$ showed statistically a significant positive relationship with the outcome of the cognitive test. The student's expected active political participation showed a statistically significant negative relationship $\left(b=-1.74,t\left(957\right)=-12.65,p<.001\right)$.

5.3. Dominican Republic

In the Dominican Republic, openness in classroom discussions $\left(b=2.27,t\left(109\right)=2.04,p=.04\right)$ and civic participation in the wider community $\left(b=-5.13,t\left(109\right)=-3.402,p<.001\right)$ significantly predicted their score in the cognitive test, with civic participation in the wider community indicating a statistically significant negative relationship.

Among the slopes, gender, the highest level of education the student expects to achieve and the student's expected electoral participation $\left(b=2.02,t\left(101\right)=8.22,p<.001\right)$ showed a statistically significant positive relationship with the outcome of the cognitive test. The student's expected active political participation showed a statistically significant negative relationship $\left(b=-2.14,t\left(289\right)=-10.70,p<.001\right)$.

5.4. Mexico

In Mexico, student activities in the community and civic learning at the school $\left(b=8.54,t\left(189\right)=5.61,p<.001\right)$ significantly predicted their score in the cognitive test, with civic participation in the wider community indicating a statistically significant negative relationship $\left(b=-2.40,t\left(189\right)=-2.88,p=.004\right)$.

Among the slopes, gender, home literacy resources, the highest level of education the student expects to achieve, and the student's expected electoral participation $\left(b=2.54,t\left(32\right)=12.85,p<.001\right)$ showed a statistically significant positive relationship with the outcome of the cognitive test. The student's expected active political participation showed a statistically significant negative relationship $\left(b=-2.34,t\left(378\right)=-16.29,p<.001\right)$.

5.5. Peru

In Peru, openness in classroom discussions $\left(b=6.43,t\left(183\right)=6.41,p<.001\right)$, civic learning at the school $\left(b=3.86,t\left(183\right)=3.67,p<.001\right)\text{,}$ and students’ civic participation in the wider community significantly predicted their score in the cognitive test, with civic participation in the wider community indicating a statistically significant negative relationship $\left(b=-6.63,t\left(183\right)=-5.27,p<.001\right)$.

Among the slopes, home literacy resources, the highest level of education the student expects to achieve, and the student's expected electoral participation $\left(b=2.74,t\left(23\right)=13.88,p<.001\right)$ showed statistically a significant positive relationship with the outcome of the cognitive test. The student's expected active political participation showed a statistically significant negative relationship $\left(b=-1.87,t\left(96\right)=-11.57,p<.001\right)$.

Table 3 summarizes the statistically significant predictors of civic knowledge and engagement in the participating LAC countries.

Table 3.

Summary of statistically significant predictors of civic knowledge and engagement in LAC countries.

School teaching and learning practices	COL	CHI	DOM	MEX	PER
Students' perception of openness in classroom discussions (S_OPDISC)	5.40	–	2.23	–	6.43
Student reports on civic learning at school (S_CIVLRN)	–	4.11	–	8.54	3.86
Students' participation in the wider community (S_COMPAR)	−6.06	−4.63	−5.13	–	−6.63
Students' civic participation at School (S_SCHPART)		3.08
Teachers' perceptions of student activities in the community (T_STDCOM)	–	−1.58	–	−2.40	–
Students' individual variables
Gender (Girls)	–	12.80	20.32	14.15	–
Home literacy resources (HOMLIT)	5.95	3.96	–	4.16	5.80
Students expected higher level of education (ISCED)	18.57	26.43	13.30	16.37	15.02
Student interest (S_SINT)	–	6.77	–	–	–
Students expected electoral participation (S_ELECPART)	2.53	2.30	2.02	2.55	2.75
Students' expected active political participation (S_POLPART)	−1.74	−2.08	−2.14	−2.34	−1.87

6. Discussion

In this study, we explored the relationship between pedagogical practices and the civic knowledge and engagement of young students in five countries in Latin America and the Caribbean (LAC): Chile, Colombia, the Dominican Republic, Mexico, and Peru.

The findings suggest that factors such as student activities, civic learning, openness in classroom discussions, and civic participation were all significantly associated with students’ scores on the ICCS cognitive test. However, these associations were found to be both positive and negative across different countries.

Across all surveyed countries, a significant association was observed between students’ civic participation in the wider community and their scores on the ICCS cognitive test. Interestingly, this association was negative, aligning with previous findings that suggest high levels of civic participation may detract from academic endeavors, potentially resulting in lower cognitive test scores [54]. This could imply a potential trade-off between civic engagement and academic performance in standardized assessments. It might also suggest that students who are more involved in civic or political activities may dedicate less time to their academic pursuits.

These findings contrast with research employing alternative measures of cognitive outcomes, which have indicated a potential relationship between civic participation and improved academic performance. Research has indicated that students engaged in school or community-based civic activities exhibit enhanced psychological well-being, heightened academic engagement and performance, and increased involvement in their education [[55], [56], [57], [58]]. Moreover, student engagement in community-based and civic activities has been associated with positive outcomes in six key areas, including the development of personal and social responsibility, fostering of positive mindsets and dispositions, improved graduation and retention rates, learning gains, and enhanced employability [59].

However, some studies have found that civic engagement may not produce uniform outcomes for every student. For instance, a study using panel data from the National Education Longitudinal Study of 1988 found that civically engaged high school students tend to make greater academic progress and are more likely to graduate, but the relationship between civic engagement and educational progress varies across different subjects [60]. Another study found that civic returns to higher education vary across groups of individuals classed by their pre-college social backgrounds and levels of early achievement [61]. The results of LAC suggest that the association of civic engagement and political participation with academic performance may depend on the individual's background and the specific type of civic activity involved.

Civic learning in the classroom was found to have a positive relationship with ICCS cognitive test scores in Chile, Mexico, and Peru, supporting previous findings that civic learning in the school plays a crucial role in enhancing students’ academic performance by fostering their cognitive abilities [18,62]. By integrating civic learning into the school curriculum, educators can provide students with the knowledge, skills, and values to become responsible and active citizens. This, in turn, can contribute to their overall academic success [63,64]. For example, group instruction techniques in civic education have been shown to improve student achievement in the subject, particularly among those with high and low academic abilities [64]. Furthermore, research has shown that cognitive styles, which can be shaped by the educational environment, can predict domain-specific creativity [63]. By incorporating civic learning into the curriculum, schools can help students develop their cognitive abilities, which can lead to better academic performance and well-rounded individuals.

In Colombia, the Dominican Republic, and Peru, openness in classroom discussions was associated with higher cognitive test scores. This is consistent with previous research suggesting that open classroom discussions can improve students' cognitive ability. Lin [65] reported that students' perceptions of classroom openness are substantially related to their civic knowledge scores after analyzing data from 38 countries. Another study, this one focusing on Scandinavian countries, found that a well-balanced mix of teaching methods contributes to students’ civic knowledge performance [66].

In a third study [67], researchers investigated the relationship between an open classroom environment and the socioeconomic composition of the classroom in relation to students' civic knowledge and attitudes toward voting. The findings indicated that a student's family's socioeconomic status played a significant role in explaining variations in student civic knowledge and voting attitudes. However, it was observed that the presence of an open classroom climate did not predict any significant changes in either knowledge levels or attitudes toward voting. Finally, a study in Czechia reported a significant positive relationship between civic knowledge and formal involvement but not with civil participation, while an open classroom environment promotes both types of engagement during adolescence [68]. In general, these findings imply that classroom openness and teaching approaches play an essential role in increasing civic knowledge and engagement and highlight the importance of fostering an open and inclusive classroom environment to facilitate cognitive development.

An interesting topic for discussion is the correlation between sociodemographic factors, including educational aspirations of students, gender, and home literacy resources, and their ICCS civic knowledge scores.

Our findings suggest that students' expected highest level of education is the strongest predictor of civic knowledge scores. For each point increment in the expected level of education scale, an average increase of 18 points was projected on the cognitive test. This strong correlation suggests that a student's ambition or their expectation for higher education attainment can be significantly linked to their cognitive abilities as measured by the ICCS test.

There could be multiple underlying factors driving this strong relationship. A higher education expectation could reflect the intrinsic motivation towards learning [69], which in turn contributes to better cognitive test performance [[70], [71], [72]].

Additionally, students who anticipate achieving higher levels of education might engage in learning behaviors that promote cognitive development, such as independent reading, tackling complex tasks, or actively seeking academic challenges [73,74].

Furthermore, this finding may also imply the role of external factors, such as family or societal expectations. Students from families or communities that value education highly are likely to have higher education aspirations, which could motivate them to perform better on cognitive tests [[75], [76], [77]].

This raises interesting questions about the potential ways educators could use this information. It suggests that fostering high educational aspirations in students could be an effective strategy to improve their cognitive performance. This could involve measures such as career counseling, mentorship programs, or exposure to role models who have achieved high levels of education.

Regarding gender differences, the results of this study show that girls consistently outperform boys in the cognitive component of the ICCS test, not just in the five LAC countries that participated in the study, but in all participating nations (Fig. 4). At first glance, this finding could imply that girls have a higher degree of civic knowledge and engagement than boys, or that girls are more interested or effective in absorbing civic education, or that the teaching methods used are more appealing to girls.

Fig. 4.

of students at each proficiency level by country and gender in ICCS 2016.

However, while the ICCS primarily assesses students' civics and citizenship knowledge and understanding, it also incorporates reading comprehension as a critical element, given that students must read and comprehend questions, texts, and other information during the test. As a result, the fact that girls typically outperform boys in the ICCS may indicate that they possess better reading comprehension skills, as they may be better able to assimilate and comprehend the information given in the test. This finding is supported by previous research that shows that girls typically outperform boys in reading, both in terms of comprehension and general literacy skills [[78], [79], [80]]. According to several studies [79,81,82] this gender disparity in favor of girls in reading is caused by a variety of reasons, including differences in brain development, early childhood experiences, and cultural and societal expectations. Girls are typically exposed to more ‘language rich’ environments at a younger age, which can give them an advantage in language development [83,84].

Therefore, these findings call for more research on the causes of this gender gap in civic knowledge and engagement. Differences in motivation, study habits, participation in the classroom, and even cultural expectations could all be contributing factors. To promote equitable learning results, educators, curriculum designers, and legislators can adjust educational programs to the requirements of all students, regardless of gender, by having a thorough understanding of these aspects.

Except for the Dominican Republic, where the households in the sample demonstrated similar levels of home literacy resources, implying minimal variability in this variable, the study identified a positive correlation between home literacy resources and test scores in all other countries within the Latin American and Caribbean (LAC) region. While the specific correlation between home literacy resources and test scores using ICCS data has not been extensively explored, other fields of education have examined how home resources relate to academic achievement.

Existing research indicates that a connection exists, albeit complex, shaped by several mediating factors. For example, socioeconomic status plays a considerable role in this context. Higher socioeconomic status can provide access to a wider array of educational resources, including books, educational toys, and digital tools, leading to better academic results. On the contrary, lower socioeconomic families might have limited access to such resources, thereby affecting the academic performance of the students [85,86].

Home possessions also stand out as another significant factor. Households furnished with items that stimulate learning and cognitive development can offer an environment conducive to academic success. For instance, the availability of a personal computer for studying can be strongly associated with enhanced academic performance [87,88].

The findings of this study highlight the complexities of the relationship between home literacy resources and academic achievement. A variety of sociodemographic variables can interact with this dynamic, demanding a comprehensive approach to education that considers the interplay of various elements. Future studies could try to better understand these distinctions to better help students from various socioeconomic backgrounds.

Finally, we found contrasting results regarding students' attitudes towards future political participation, which encompass both students' expected electoral participation and students' expected active political participation. Although expected electoral participation consistently showed a statistically significant association with civic knowledge and engagement outcomes in all LAC countries; the students’ expected active political participation showed a negative association with civic knowledge.

Although we could not find other empirical studies that address this relationship, we hypothesize that when students expect or intend to participate in elections, their understanding of civic matters, civic knowledge, and their involvement in civic activities tends to increase. Additionally, if students have the expectation of voting in elections in the future, it is likely that they have a personal interest in understanding the electoral process, the issues at stake, or the positions of various candidates. This interest could drive them to independently seek information about the electoral process, leading to higher levels of civic knowledge. This opens an interesting line of inquiry that should be further explored in future research. Investigating to what extent anticipation of electoral participation motivates student engagement with civic issues could yield valuable insights into how to support civic education strategies. Furthermore, understanding this relationship could also help educators and policy makers design interventions that encourage students’ active participation in democratic processes, thereby fostering a more informed and engaged citizenry. This could lead to increased civic literacy and engagement, improving the functioning of democratic societies.

The finding that students’ expected active political participation has a significant negative correlation with civic knowledge and engagement raises important questions about the various factors that could contribute to this result, especially when viewed through the lens of socioeconomic disparities in educational attainment.

We hypothesize that this negative correlation could be explained by the socioeconomic background of the students. It could be that students from lower socioeconomic backgrounds are more inclined toward active political participation, given their urgent need for social change. However, these very students may have lower levels of civic knowledge, not due to a lack of interest, but potentially due to unequal access to quality education and resources that facilitate civic learning.

These hypotheses are supported by evidence that in many Latin American contexts politics is often seen as an opportunity for socioeconomic advancement or an escape route from poverty [[89], [90], [91]]. Therefore, the attraction towards active political participation might be stronger amongst students from lower socioeconomic backgrounds. However, their level of civic knowledge might be inhibited by their economic circumstances, which could affect their access to quality education. In fact, research has consistently shown that students from lower socioeconomic backgrounds often lag in academic performance compared to their peers from higher socioeconomic backgrounds [[92], [93], [94]]. This disparity could extend to their civic knowledge, which could explain the observed negative relationship between expectations of active political participation and civic knowledge.

Additional empirical research could be useful in evaluating these hypotheses and delving into the subtle mechanisms that link socioeconomic factors to the correlation between expected active political participation and civic knowledge among students. This study could have a significant impact on educational strategies aimed at increasing civic understanding and involvement across various socioeconomic groups.

Limitations and opportunities for future research

While this study offers valuable information on the relationships between pedagogical practices and civic knowledge and engagement among young students in Latin America and the Caribbean, several limitations must be recognized. For one, only five countries in the LAC region participated in the ICCS 2016 study, limiting the generalizability of the findings to other regions with different sociocultural and educational contexts. Future research could consider expanding this scope to a more diverse set of countries and environments.

The measures of pedagogical practices used in the ICCS study, namely student activities, civic learning, civic-related activities in class, civic participation at school and in the wider community, and openness in classroom discussions, are not the only forms of civic engagement. Also, some of the measures, such as the level of openness in classroom discussions, could be perceived differently depending on the perspectives of individual students, introducing a level of subjectivity. Future research endeavors could gain value from employing more objective metrics to evaluate these variables, potentially offering a more comprehensive insight into the relationship between teaching and learning practices and civic knowledge and engagement.

Given the cross-sectional nature of the data used in this study, it is challenging to draw definitive causal inferences. The identified associations, while significant, could be further explored in longitudinal or experimental studies, which could provide more nuanced insight into the causality and dynamics of these relationships.

This study also considered several sociodemographic variables. However, other factors not included in the analyses could confound the relationship between pedagogical practices and civic knowledge and engagement. The addition of control variables such as school quality, teacher qualifications, or socioeconomic status in future studies could further refine the understanding of these relationships.

Lastly, future research would benefit from exploring the factors that predict students’ attitudes toward political participation, which may affect civic knowledge and engagement, in light of the varied findings of this study.

Conclusions

The purpose of this study was to investigate the relationship between instructional practices and civic knowledge and engagement among young students in five Latin American and Caribbean countries. The results provided important information on the intricate relationship between various educational factors and the scores of students in the cognitive tests of ICCS. They underscored the significant role of civic learning in and out of school, the openness of classroom discussions, student anticipated levels of education, gender, and home literacy resources in shaping civic knowledge and engagement.

Interestingly, while civic participation in the larger community was found to negatively affect cognitive test scores, suggesting a trade-off between civic engagement and academic performance, civic learning within the classroom environment emerged as a crucial factor in improving cognitive abilities and fostering academic success. This further emphasizes the necessity of integrating civic learning into the school curriculum.

Furthermore, the findings underscored the significant role of educational aspirations in shaping civic knowledge and engagement, suggesting that nurturing high educational aspirations could significantly improve cognitive performance. The relationship between gender and ICCS cognitive scores, with girls consistently outperforming boys, called for a deeper exploration of gender dynamics in civic education. Additionally, the positive correlation between home literacy resources and test scores, except in the Dominican Republic, shed light on the importance of the home environment in facilitating academic achievement. A plausible explanation for this discrepancy could be the lack of variability in the home literacy resources variable within the Dominican Republic data set. If most households in the Dominican Republic sample have similar levels of home literacy resources, it would be difficult to observe a significant association with test scores. In other words, if there is not much variation in the independent variable (home literacy resources), it becomes challenging to detect its relationship with the dependent variable (test scores). This could potentially explain why the relationship observed in other countries was not found in the Dominican Republic.

The relationship between students’ expected electoral participation and civic knowledge and engagement indicated a promising line of inquiry for future research. Understanding this relationship could potentially pave the way for the design of effective strategies to strengthen civic education and encourage active democratic participation.

Finally, the negative correlation between students’ expected active political participation and civic knowledge is a significant point for discussion, particularly when considering socioeconomic disparities in educational attainment. Students from lower socioeconomic backgrounds may be more inclined towards active political participation due to their need for socioeconomic advancement. However, their civic knowledge may be lower due to unequal access to quality education and resources. This disparity could extend to their academic performance, potentially affecting their civic knowledge. Further empirical research is needed to assess these hypotheses and investigate the mechanisms at play in the relationship between expected active political participation and civic knowledge among students.

In conclusion, these findings provide valuable information on the complex interplay between various educational, sociodemographic, and attitudinal factors and the civic knowledge and engagement of students. They underscore the importance of a comprehensive approach to civic education, incorporating elements such as in-class and extracurricular civic learning, fostering open classroom discussions, nurturing high educational aspirations, providing adequate home literacy resources, and promote active democratic participation. We believe that such an approach is more likely to produce more well-rounded individuals who have the knowledge, skills, and values required to be responsible and active citizens. At the same time, these findings emphasize the need for additional study to get a more nuanced understanding of these processes and to build more effective and fair civic education programs and curricula.

9. Ethics statement

The Education Research Committee at the School of Education, Universidad de Antioquia, approved this research on October 21, 2022 (Ref. No. 22060001-0275-2022). The study is based on an analysis of publicly accessible secondary data, namely the ICCS dataset. The IEA compiled and provided this dataset. There were no direct interactions with human subjects in this study, as only existing data was utilized. The IEA's website provides access to the ICCS dataset and related documentation: https://iea.nl/index.php/studies/iea/iccs.

Funding

This research received funding from the Colombian Institute for the Evaluation of Education (ICFES) under grant # 381-2022, as well as from the Office of the Vice Chancellor for Research and the School of Education at Universidad de Antioquia. The views, interpretations, and conclusions expressed in this study are solely those of the authors and do not represent the views or policies of any institution or entity affiliated with the authors.

Availability of data and materials

The datasets and scripts from this study can be accessed at the OSF repository here. For specific requests, please contact the corresponding author. Additionally, ICCS data, codebooks, manuals, and other materials are available for download from the IEA website.

CRediT authorship contribution statement

Ricardo L. Gómez: Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing. Ana María Suárez: Investigation, Methodology, Writing – original draft, Writing – review & editing.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix A. Supplementary data

The following is the Supplementary data to this article.