A method description for developing critical reading skills through a digital platform based on Barrett’s taxonomy

Abstract

The article outlines a method for developing, assessing critical reading skills using a digital learning platform based on Barrett’s taxonomy of Reading Comprehension. A taxonomy-based task design is integrated with an automated assessment. The level of questions is categorized into two types: lower-level questions, comprising literal and reorganization levels, and higher-level questions, including inferential, evaluative, and appreciative types. Lower-level comprehension is automatically evaluated with objective questions on the platform. Whereas, higher-level comprehension is assessed through a specifically designed assessment rubric, integrated into the platform. Educators can use the platform architecture for these functions: design and share reading tasks, gather student responses, and define the students' performance through integrated learning analytics. A pilot demonstration with a sample dataset shows the disparities in the students’ levels of understanding. Students face challenges in performing high level tasks. The proposed framework illustrates how teaching critical reading can be conducted with the integration of digital platform, Which allows for reproducible implementation

A Barrett’s taxonomy grounded methodological framework for developing multilevel reading comprehension tasks

A digital assessment platform, integrated with automated evaluation of objective questions and rubric-based assessment of open-ended questions

An evaluation system supported analytically, informing teachers about students’ performance across comprehension levels.

Graphical abstract

Specifications table

Subject area	Psychology
More specific subject area	Psycholinguistics; cognitive reading processes; literary comprehension Assessment, digital learning environments.
Name of your method	A digital platform-based method for developing and evaluating critical reading tasks grounded in Barrett’s taxonomy
Name and reference of original method	C. Barrett, T. (1976). Taxonomy of reading comprehension. Increasing reading comprehension using information literacy skills.
Resource availability	The method is implemented through a web-based digital platform designed by the authors with the assistance of a web developer for developing and assessing critical reading skills. Teachers can upload literary texts to the platform, make multilevel comprehension questions that follow Barrett's five levels of taxonomy, and grade students' answers using both automated scoring and a rubric. It also keeps track of what students respond and makes learning analytics to monitor their performance at different levels of understanding. The platform can be accessed at the following link: [https://english-platform-nine.vercel.app]. It can be used on any device that can connect to the internet, like a computer, tablet, or smartphone. Any special hardware is not required.

Background

In today's education, especially in language and literature, it is widely accepted that developing critical reading skills is a key goal [1]. Students need to do more than just find information in a text when they read critically. They have to also figure out what it means, judge the arguments, and come to their own conclusions based on the text's evidence [2]. Despite this, students often do better on basic comprehension tasks than on more difficult analytical and evaluative tasks in different educational settings [3]. Prior studies indicate that students frequently struggle to transcend literal comprehension to engage with texts in a more interpretive and critical manner [4]. These results show how important it is to have organized ways to teach and test that slowly help students get better at understanding.

Barrett's Taxonomy of Reading Comprehension is one of the most well-known ways to organize tasks that test reading comprehension [5]. Barrett mentions that there are five levels of understanding: literal understanding, reorganization, inferential understanding, evaluative understanding, and appreciative response [6]. A lot of reading teachers have used the taxonomy to help them come up with questions that slowly improve understanding [7]. Teachers can help students understand what they are reading and think critically about it at the same time by breaking it down into these levels [8].

Barrett's taxonomy is useful for teaching, but it can be hard to use in the classroom because it takes a long time to make tasks with different levels and grade open-ended answers [9]. Teachers also have a hard time keeping track of how well students are doing in different areas of understanding when they give them tests on paper [10]. The rapid growth of digital educational technologies creates new ways to solve these problems by bringing together learning analytics, assessment, and instructional design into one digital platform [11].

Moreover, recent studies in teaching reading emphasize the importance of proficient comprehension instruction, which comprises the systematized cognitive and methodological frameworks, that contribute to the progression of learners from basic understanding to advanced interpretative thinking [12]. These frameworks assist teachers in developing tasks and exercises that promote students’ analytical and reflective reading skills, instead of just focusing on the surface-level comprehension of the text [13].

The contemporary research in reading pedagogy underscores the importance of an instructional design integrated with technology-supported assessment tools, with the help of which both the reliability and the transparency of reading evaluation can be developed [14]. The methodology of this study delineates a digital platform-based framework for the formulation and evaluation of reading comprehension tasks grounded in Barrett's taxonomy. The proposed approach integrates task creation grounded in a taxonomy with automated scoring of multiple-choice questions and rubric-based evaluation of analytical responses. This integration makes it easy for teachers to give students reading assignments at different levels, get their answers, and see how well they understand the material at those levels. The recommended approach employs digital assessment instruments in conjunction with established theoretical concepts regarding reading instruction. This means that it can be reproducible to help them get better at reading critically.

Method details

The suggested method outlines an organized strategy for cultivating and evaluating critical reading skills via a digital learning platform grounded in Barrett’s Taxonomy of Reading Comprehension. The method combines taxonomy-based task design with a digital assessment environment that allows for both automated scoring and evaluation using a rubric. Teachers can use the platform to create reading tasks that are appropriate for different levels of understanding, give them to students, gather their answers, and use built-in learning analytics to look at how well students did.

The method is meant for language and literature classes where understanding what you read and being able to think critically about texts are two of the main goals [15]. This method can be reproduced with short stories, essays, or other types of narrative writing.

The proposed method comprises the following stages:

• 1. Selection and preparation of text. The teacher chooses a short literary text with a length of from 200 to 400 words. While selecting the literary text, the plot, theme and conceptual structure is important. A short story can be used as a model example. Such as a short story “The Last Night of the World” by Rad Bradbury.

• 2. Designing questions across Barrett’s taxonomy levels.

• Level 1 -literal level. The questions are designed to examine the knowledge of details in the literary text. Typical examples include: what happened at the beginning or end of the story? What was the name of the main character?, etc. [16].

• Level 2 - Reorganization level. The questions are designed to check the ability of students to arrange the plot of the story correctly. For instance, arranging events in chronological order, classifying concepts or events [17]

• Level 3 – Inferential level. The questions are designed to examine the students’ ability to find the implicit meaning. Examples include: Why did the character act in this way? What can be understood from the description of the setting? [18]

• Level 4- Evaluative level. The tasks at this level examine how well students can assess or judge the aspects of the text. For example, do you agree with this character's decision or not? [19]

• Level 5 – Appreciative level. These types of questions examine how well students respond emotionally, personally, and aesthetically to the literary text. For example, what is your attitude towards the central conflict in the story?(Nabela [20])

• Literal, reorganization, and inferential level questions are objective questions, and they are assessed automatically in the platform; the results are released right after performing the tasks.

• The evaluative and appreciative level questions require full, extended responses, and they are evaluated using a rubric-based assessment, but still integrated into the platform. The responses to these questions are sent to the teacher’s email address through the platform.

• 3. The integration of the method into the digital platform. The method is applied within a specific digital platform, which integrates Barrett’s taxonomy with automated and rubric-based evaluation. The platform has two interfaces: the teacher interface and the student interface. The teacher interface includes:

• −A form for text entry and uploading.
• −Question table which can be editable.
• −Fields of answer keys for objective questions.
• −Rubric upload for extended responses.

The student interface includes

• −Tasks and texts appear together.
• −Objective questions for lower levels.
• −Open-response fields for high-level tasks.
• −Text entry, submission, and feedback display.

• 4. An automated and rubric-based scoring system:

In automated scoring, when the student completes the assignments, the system instantly checks selected answers against the answer key and releases the scores automatically. There is a collection of scores in the database.

Rubric-based evaluation of extended responses:

Higher-level tasks are checked manually according to the assessment rubric, which was formed by a teacher. The submitted responses through the platform reach the teacher’s email address. Then the teacher reviews the submitted answers and marks them according to the predefined assessment rubric. Here is the assessment rubric for open-ended questions:

Criteria	Excellent (13–15 points)	Good (10–12 points)	Satisfactory (7–9 points)	Weak (0–6 points)
Interpretation and justification	Deep, original, and well-supported interpretations with clear justification.	Reasonable interpretation, some supporting evidence.	Partial understanding, limited or vague justification.	Misinterpretation or unsupported ideas.
Critical reflection	Insightful personal reflection; strong connection to author’s message.	Clear reflection; some connection to main ideas.	Minimal reflection; general comments.	Lacks reflection or personal insight.
Expression and clarity	Highly coherent, well-organised, and fluent language	Mostly clear and organised.	Somewhat disorganised or unclear at points.	Lacks reflection or personal insight.

The rubric is integrated into the platform through the task design itself. All of the tasks are formed in association with the rubric criteria, ensuring that it corresponds to the established evaluation standards. The assessment rubric is integrated into the platform’s system in a way that enables the automatic calculation and display of total scores for lower- and mid-level questions upon task completion. Meanwhile, automated scoring is not implemented for higher-order tasks, particularly at evaluative and appreciative levels, as the responses have a subjective and interpretive nature. Responses to higher-order questions are submitted via the platform and automatically forwarded to the teacher’s email address for evaluation. Then they are evaluated manually based on assessment criteria. While evaluating, the teacher reviews each response submitted by the student and assigns scores for each criterion (expression, critical reflection and interpretation) based on the descriptors of the rubric. The evaluated results are sent back to the student with feedback. The criteria for open-ended questions (higher-order tasks) are illustrated at the table above. After completing the process, students can view a summary of their scores and receive feedback on their work. Combined analytics, such as average scores, distribution patterns, and performance trends across levels are made visible to the teacher. Collectively, these stages form an accurate and reproducible methodological algorithm, including task design, assessment, and feedback.

• 5. Guidelines of implementation for teachers and students

• 5.1. Teacher implementation:

• −Register on the platform and design a course.
• −To design a new lesson, upload the selected literary text.
• −Enter questions for the literary text level by level.
• −For questions of a lower level, enter answer options and mark the correct answer.
• −For questions of a higher level, explain the task and rubric criteria.
• −Finally, to make them available for students, the lessons should be published.

• 5.2. Student Implementation

• −Log in to the platform and find the assigned lesson.
• −Read the literary text attentively.
• −After completing the lower-level questions, students receive immediate automated feedback.
• −Write extended responses for higher-level questions and submit them.
• −View the results collectively for automatically scored items and a notification that a teacher received the written responses for evaluation.

Method validation

The digital platform was piloted with 16 undergraduate students who majored in philology. This experiment was carried out to check how reproducible and applicable the proposed methodology. The short story “The Last Night of the World” by Ray Bradbury [21] was chosen to conduct the validation activity. The reason for choosing this story was its narrative clarity and suitability for developing reading comprehension tasks across a variety of levels of interpretation. Short stories are frequently implemented in the experiments of reading research, as they enable students to work with both explicit meanings and deeper interpretative meanings of the literary text [22].

The assessment rubric was designed grounded in Barrett’s taxonomy of reading comprehension, as it organizes the comprehension process into five hierarchical levels. They are as follows: literal, reorganization, inferential, evaluative, and appreciative levels [23]. The studies that were conducted previously support that Barrett’s taxonomy-aligned task design offers a structured way to assess different levels of reading comprehension and cognitive engagement with the text [24].

A systematized reading comprehension assessment, including five subsequent tasks according to their levels, was applied on the digital platform. Each task corresponds to five different levels of Barrett’s taxonomy [25]. Based on this categorization, the assessment rubric was developed, which allocates a total of 32 points across five tasks. The tasks are developed to underscore the degree of philology students’ higher-order reading skills. The assessment rubric is as follows:

Task	Comprehension level (based on Barrett's taxonomy)	Description of skills assessed	Total points
Task 1	Literal comprehension	Identifying explicitly stated facts, ideas, details and information	8
Task 2	Reorganisation	Synthesising, summarising, and sequencing information from the text	5
Task 3	Inferential comprehension	Predicting outcomes and identifying implied meanings	4
Task 4	Evaluative comprehension	Making judgements about values, attitudes and aesthetic qualities	10
Task 5	Appreciative comprehension	Expressing emotional and reflective responses	5
Total:			32 (100%)

The first two tasks were multiple-choice questions that the platform automatically graded. The other tasks required open-ended analytical responses that were graded using a rubric-based assessment system built into the platform interface. Digital platforms that combine automated and rubric-based evaluation have been shown to make assessment processes more efficient and open, while still keeping the reliability of higher-order evaluation tasks [26]

Below is a sample of pilot scores from 16 students that shows how the platform's scoring system works.

	Task 1 (8 points)	Task 2 (5 points)	Task 3 (4 points)	Task 4 (10 points)	Task 5 (5 points)	Total (32 points)
1	7	3	3	7	4	24
2	6	4	3	7	3	23
3	6	5	2	6	2	21
4	7	4	2	6	2	21
5	8	3	3	6	5	25
6	8	4	4	9	5	30
7	6	3	2	6	4	21
8	6	3	4	6	3	21
9	5	4	4	6	5	24
10	7	3	4	8	4	26
11	6	3	3	5	3	20
12	5	4	3	5	3	20
13	8	5	4	8	5	30
14	8	5	4	9	5	31
15	6	4	4	5	2	21
16	7	4	4	7	4	26

To check the validity and reliability of the platform-based method, a pilot test was conducted with 16 philology students. As mentioned above in the assessment rubric, the maximum score was 32 points. Cronbach’s alpha was chosen as the most proper statistical measure to evaluate the consistency of tasks and questions. The following standard formula was chosen to calculate the coefficient:

$$\alpha =\frac{k}{k-1}\left(1-\frac{\sum {\sigma }_i^2}{{\sigma }_T^2}\right)$$

$k$= number of tasks

${\sigma }_i^2$= variance of each task score

${\sigma }_T^2$= variance of total test scores

After the calculations, the reliability coefficient was equal to

α=0.765

The Cronbach’s alpha benchmark of reliability is α ≥ 0.70. It is immediately apparent from the results that the internal consistency of tasks and questions is acceptable [27]. It has been proven that the tasks jointly constitute a reliable instrument for assessing different levels of reading comprehension.

Descriptive statistics of the pilot test:

Task	Comprehension level	Maximum score	Mean	SD	Mean %
1	Literal	8	6.63	1.03	83%
2	Reorganization	5	3.81	0.75	76%
3	Inferential	4	3.31	0.79	83%
4	Evaluative	10	6.63	1.41	66%
5	Appreciative	5	3.69	1.08	74%

The average scores indicate that students performed better on tasks that tested their ability to understand what they read and what they meant. The evaluative task had the lowest average percentage, which shows that reading skills at a higher level require greater cognitive ability.

The reliability analysis demonstrates that the proposed taxonomy-based assessment exhibits good internal consistency (α = 0.765) and that each task contributes significantly to the overall measurement of reading comprehension. Also, the difference in average task performance shows that the test can tell the difference between tasks that require lower-level understanding and tasks that require higher-order analysis. These results show that the digital platform can be used in a way that makes sense from a methodological point of view. They also give us some early statistical evidence that the suggested framework for testing critical reading skills is reliable.

Limitations

Although the proposed method demonstrates practical applicability and acceptable reliability, it has several limitations. First, there are some higher-level open-ended tasks included in the assessment, which in turn lead to the involvement of instructors to evaluate. The digital platform-based assessment may involve a degree of subjective judgement, specifically in evaluative and appreciative tasks.

Second, the method is mostly meant for literary and narrative texts. Before applying the method in other types of reading materials, such as medical or technical texts, it is required to adapt the task design and scoring criteria.

Finally, the method can only be used if one has access to a digital platform and the internet, which may limit its use in technologically constrained educational places.

In spite of these limitations, the suggested method offers a reproducible framework for associating taxonomy-grounded reading assessment with a digital learning platform. Furthermore, it can be implemented in diverse educational contexts.

Ethics statements

The study involved human participants. All participants provided informed consent before their inclusion in the study. Participation was voluntary and did not affect students' academic grades. All procedures were followed according to the ethical rules of the institutions.

CRediT author statement

Bakhtiniso Turakulova: Conceptualization, Methodology, Investigation, Data curation, Writing – original draft, Project administration.

Barnokhon Samatova: Supervision, Writing – review & editing.

Munirabonu Raxmonova: Data curation, Formal analysis, Visualization, Writing – original draft.

Dilorom Rahmonova: Validation, Investigation, Writing – review & editing.

Gulnozakhon Ganiyeva: Resources, Formal analysis, Writing – review & editing.

Umida Jumamurodova: Software, Data curation, Visualization.

Dildora Norkuziyeva: Investigation, Validation, Writing – review & editing.

Declaration of competing interest

The authors declare that they don't have any known financial or personal interests that could have affected the work in this paper. The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Data availability

The data that has been used is confidential.