Digitalization of Learning Assessment: Design of Digital Learning Assessment_Academic Publishing_News & Cases

2021/11/17

Digitalization of Learning Assessment: Design of Digital Learning Assessment

Issue No.: HB20211117E Date: Nov. 17, 2021 Author: Global TEAM Model Education Research Institute

In the digital age, various industries have fully enjoyed the benefits of digitization, and the term is no longer unfamiliar. In the field of education, digitization has also brought many transformations and innovations, such as the digitalization of education administration, campus management, and teaching and learning. The goal of digitization is not only to optimize traditional workflows through digital technology but also to realize updates and iterations, achieving digital transformation, and even enabling a paradigm shift to create higher-quality working methods, processes, and benefits.

Echoing the global education reform trends of UNESCO and the commitment to Holistic Development and Sustainable Development Goals (SDGs), education should focus on Competency-Based curriculum and pedagogical concepts. Competency-Based Learning Assessment is particularly an important method and strategy for achieving effective teaching. It actively researches and promotes the use of learning assessment tools to precisely evaluate, feedback, and guide curriculum and instruction, giving learning assessment a new function: utilizing the learning assessment process or the assessment itself as a part of learning (Assessment As Learning). It also encourages the adoption of diverse learning assessments in the classroom, assessing not only knowledge and skills but also attitudes and behaviors, emphasizing learning outcomes, and even more so, emphasizing the learning process.

Learning assessment is a critical part of learning activities and an important indicator of the teaching team's professional capabilities. To optimize traditional learning assessment tools, improve the effectiveness of learning assessment, and preserve learning assessment history data, the digitalization of learning assessment is the most feasible optimization direction. We aim to design methods and strategies that can more effectively assess, feedback, and guide instruction through the digitalization of learning assessment design and the skillful use of Education Technology, to promote the quality and effectiveness of learning assessment, precisely grasp learning situations, facilitate learning outcomes, and assist in achieving learning goals.

📌I. Analysis of the Digital Learning Assessment Framework

The structure and design of digital learning assessment are explained below, focusing on three aspects: types of learning assessment, diverse assessment tools, and digital design case studies.

(I) Three Types of Digital Learning Assessment

Formative assessment and summative assessment are familiar assessment types. In recent years, thanks to the flourishing development of learning technology and the widespread application of smart classrooms, many international educational experts and scholars classify learning assessment types into the following three categories, based on the purpose of the assessment:
1. Assessment For Learning (AFL) is a type of formative assessment (classroom assignments, in-class quizzes, unit assignments, unit tests). Teachers use assessment methods and strategies to obtain student learning feedback, grasp students' learning situations and needs, provide appropriate feedback to students, and serve as a reference for adjusting teaching methods and strategies, thereby helping students learn and accelerating the achievement of learning goals.

2. Assessment Of Learning (AOL) is a type of summative assessment, a comprehensive assessment of learning outcomes when a teaching activity concludes. Examples include monthly exams, midterms, final exams, graduation exams, and competency tests. The assessment content covers student learning outcomes, the degree of achievement of teaching goals, the appropriateness of teaching methods, and curriculum effectiveness.

3. Assessment As Learning (AAL), also known as process assessment, involves designing assessment tasks within learning activities to guide student learning, prompting students to spontaneously learn in order to complete the assessment tasks and thereby enhance their self-capability. Furthermore, assessment tasks should be able to trigger students' metacognitive abilities, enabling students to discover their shortcomings through the assessment tasks, engage in self-reflection and introspection on their learning status, and thus motivate better learning performance. In short, Assessment As Learning is an instructional design where the assessment task is the primary learning activity. For example, a smart classroom is a teaching scenario where AAL (Assessment As Learning) is relatively easier to implement.

Based on the above learning assessment types (Ren Zonghao, 2019; Wu Bichun, 2017; Baird, Andrich, Hopfenbeck, & Stobart, 2017; Bennett, 2011), the digital design for competency-oriented learning assessment can also be planned according to this classification, as shown in Figure 1.

Figure 1: Three Types of Digital Learning Assessment

(II) Diverse Learning Assessment Tools

Diverse learning assessment emphasizes non-singularity and adopts variety, avoiding being limited to only paper-and-pencil tests. This includes the assessment content, methods, timing, scenarios, scoring methods, personnel, and process, all of which should present diversified learning assessment data.

Common learning assessment tools include test paper assessment, assignment assessment, oral examination, practical performance assessment, self-assessment, peer assessment, report assessment, presentation assessment, and portfolio assessment. The digital design of learning assessment must first be able to digitize the aforementioned learning assessment tools through appropriate design to improve assessment effectiveness and reduce the workload of assessment. Furthermore, it should develop a new generation of learning assessment tools that possess the value of digital transformation and even paradigm shift.

Digital learning assessment supports diverse assessment formats
Figure 2: Three Learning Assessment Types and Diverse Tools

(III) Digital Design Taking TEAM Model HiTeach as an Example

The TEAM Model Smart Education Support System (hereinafter referred to as the TEAM Model System) is designed based on the needs of the four major digitized learning stages: e-Teaching, e-assEssing, e-diAgnosing, and e-reMediation. It is a smart teaching system that integrates cloud computing and learning data convergence.

The main function of the HiTeach classroom terminal of the TEAM Model System is to provide digital tools to assist teachers in teaching activities, improve teaching effectiveness, assist teachers in implementing learning assessment, quickly grasp student learning status, assist teachers in generating learning status diagnostic reports, and finally assist teachers in conducting remedial teaching activities, completing the full learning activity cycle of T (Teaching), E (assEssing), A (diAgnosing), and M (reMediation).

Based on the purposes of the three types of assessment—Assessment For Learning (AFL), Assessment Of Learning (AOL), and Assessment As Learning (AAL)—and the needs for diverse assessment tools such as test paper assessment, assignment assessment, oral examination, practical performance assessment, self-assessment, peer assessment, report assessment, presentation assessment, and portfolio assessment, the learning assessment modules of the TEAM Model System can be divided into three main parts: the teacher-side, the student-side, and the cloud platform IES, to complete the design and application of digital learning assessment. The main functions are listed as follows:

1. Teacher-side HiTeach Synchronous Push Module;
2. Teacher-side HiTeach Synchronous Interaction Module;
3. Teacher-side HiTeach Synchronous Task Module;
4. Teacher-side HiTeach Synchronous Quiz Module;
5. Student-side Web IRS Instant Feedback App;
6. Student-side AClass ONE Assignments, Tasks, Quizzes;
7. Student-side AClass ONE Online Autonomous Learning;
8. Student-side AClass ONE Incorrect Question Practice, Remediation, Cloud Platform;
9. Cloud Platform IES Curriculum, Material, and Question Bank Collaborative Preparation;
10. Cloud Platform IES Management of Assignments, Tasks, Quizzes;
11. Cloud Platform IES Integrated Test Sheet and Answer Card Grading System;
12. Cloud Platform IES Score Calculation and Learning Assessment Analysis.

As shown in Figure 3: Digital Learning Assessment Tools of the TEAM Model System.

TEAM Model Smart Education Support System provides a complete solution for digital learning assessment
Figure 3: Digital Learning Assessment Tools of the TEAM Model System

📌II. Learning Assessment Module Design

The functional design of the digital learning assessment modules is explained below, taking the TEAM Model System as an example. Through actual design cases and screenshots, the application needs of digital learning assessment can be understood more clearly, as well as how to transition from traditional to digital, from digitization to digital transformation, and thereby realize a paradigm shift in learning assessment.

(I) Teacher-side HiTeach Synchronous Push Module:

The teacher-side HiTeach Synchronous Push Teaching Module is a digital material transmission function. It is responsible for instantly pushing teaching materials (resources) and diverse assessment tasks to the student-side Web IRS Instant Feedback App. It can also push differentiated assessment tasks based on student data feedback, such as implementing differentiated assessment according to groups, answer choices, correctness, etc.

(II) Teacher-side HiTeach Synchronous Interaction Module:

The teacher-side HiTeach Synchronous Interaction Learning Assessment Module is a questioning and response function, a simple learning assessment function for teacher questioning and student synchronous answering (module screen shown in Figure 4). Question types include single-choice, multiple-choice, true/false, fill-in-the-blank, and snatch authority. After students answer questions via the student-side Web IRS Instant Feedback App, the teaching module automatically collects all student answers, automatically grades them, calculates scores, and displays statistical charts. The synchronous interaction teaching module can also be called an instant learning assessment tool, which can assess learning status at any time and conduct synchronous, high-efficiency interactive classes involving questioning, feedback, follow-up questions, and explanations.

Common learning assessment tools include test paper assessment, assignment assessment, oral examination, practical performance assessment, self-assessment, peer assessment, report assessment, presentation assessment, and portfolio assessment. The digital design of learning assessment must first be able to digitize the aforementioned learning assessment tools through appropriate design to improve assessment effectiveness and reduce the workload of assessment. Furthermore, it should develop a new generation of learning assessment tools that possess the value of digital transformation and even paradigm shift.

HiTeach power tool: Synchronized Activity (Synchronous Interaction)
Figure 4: Synchronous Interaction Learning Assessment Module

(III) Teacher-side HiTeach Synchronous Task Module:

The teacher-side HiTeach Synchronous Task Learning Assessment Module is a function for digital diverse assessment tasks (module screen shown in Figure 5). Students are asked to complete learning assessment tasks according to the task requirements (e.g., a worksheet). Task types can be diversified, including images, audio, and multimedia files. Task categorization helps the teacher quickly aggregate, compare, and analyze the content of the assessment tasks completed by the entire class.

When the teacher-side initiates a learning assessment task, it is automatically synchronized to the student-side Web IRS Instant Feedback App. During the process of the entire class completing and submitting the task, the teacher-side work collection window can instantly track the completion progress of each group or individual student and select works based on time sequence, seat number, or grouping method for observation, comparison, peer review, praise, annotation, explanation, or sharing activities.

HiTeach power tool: Synchronized Assignment (Synchronous Task)
Figure 5: Synchronous Task Learning Assessment Module

(IV) Teacher-side HiTeach Synchronous Test Mode:

The teacher-side HiTeach Synchronous Test Learning Assessment Module is a quiz function in the form of a digital multimedia test paper (module screen shown in Figure 6). When the teacher-side initiates the quiz mode, the teacher simply selects the test paper to be used for the synchronous quiz from the test paper bank, and the test paper is automatically pushed to the student-side Web IRS Instant Feedback App.

During the entire class's answering process, the teacher-side progress table shows the answering progress of each student. When the teacher clicks the end quiz button, the teacher-side HiTeach assessment module automatically grades, generates summary statistics for the learning assessment, such as the correct answer rate, distribution charts, and score tables. The score table includes scores and total scores for various question types, such as single-choice, multiple-choice, true/false, fill-in-the-blank, and question sets. At the same time, the student-side also receives scores and grading results.

Furthermore, the teacher-side instantly displays the correct answer rate for each question, the student total scores, and a score analysis chart, and produces the assessment data required for interactive review for each question, enabling instant interactive assessment of each item.

HiTeach power tool: Synchronized Assessment (Synchronous Test)
Figure 6: Synchronous Quiz Learning Assessment Module

(V) Student-side Web IRS Instant Feedback App:

The student-side Web IRS Instant Feedback App is the student device module that connects to the teacher-side HiTeach smart teaching system via the internet and is a necessary module for digital learning assessment. Web IRS is a web-based app that can be used on smart terminal devices that support a browser, such as mobile phones, tablets, or laptops.

The student-side Web IRS Instant Feedback App automatically follows the interaction, task, or quiz pages based on the learning assessment type initiated by the teacher-side, guiding students through the corresponding learning assessment activity. Simultaneously, it automatically retains the class history, allowing students to browse and view learning materials, learning tasks, and class notes pushed by the teacher-side at any time.

Student's device with Web IRS via a browser
Figure 7: Student-side Web IRS Instant Feedback App

(VI) Student-side AClass ONE Assignments, Tasks, Quizzes:

The student-side AClass ONE is an asynchronous learning assessment tool for selected students after class. It is also a web-based app that can assist teachers in extending teaching activities from the classroom to post-class learning. It allows students to complete learning assessment assignments, learning assessment tasks, and quiz activities designated by the school teacher via mobile devices such as tablets, computers, or mobile phones.

AClass ONE is a smarter learning companion for students
Figure 8: Student-side AClass ONE Smart Learning Companion

(VII) Student-side AClass ONE Online Autonomous Learning:

In addition to completing learning assessment tasks, the student-side AClass ONE is also an entrance for autonomous learning. It allows students to complete school curriculum content and engage in self-learning of advanced or broadened curriculum content (e.g., watching videos, e-materials, assessment practice).

Students can review the list of activity tasks
Figure 9: Student-side AClass ONE Autonomous Learning

(VIII) Student-side AClass ONE Incorrect Question Practice, Remediation:

All data generated from the learning assessment process are automatically transmitted to AClass ONE, allowing students to view assessment results, number of correct/incorrect questions, incorrect question analysis, and remedial resources at any time, and conduct incorrect question practice, enabling students to precisely review and improve learning achievement. Figure 10 shows the student-side incorrect question practice screen.

Review of assessment responses
Figure 10: Student-side AClass ONE Incorrect Question Practice

(IX) Cloud Platform IES Curriculum, Material, and Question Bank Collaborative Preparation:

On the school-based Cloud Platform IES, resources required for teaching and learning, such as teacher accounts, student accounts, class rosters, subjects, courses, curricula, and teaching materials, can be established for the school. The Cloud Platform IES collaborative preparation of curricula, materials, and question banks is displayed in Figures 11-1 and 11-2. The curriculum function can be used to establish a systematic context and structure for courses, materials, and question banks, helping teaching teams collaboratively prepare subject materials and question banks based on the curriculum structure and share teaching resources.

IES's Personal Syllabus and School Syllabus
Figure 11-1: Cloud Platform IES Collaborative Curriculum Preparation

A hub for multimedia teaching materials for schools
Figure 11-2: Cloud Platform IES Collaborative Material Preparation

School question bank
Figure 11-3: Cloud Platform IES Collaborative Question Bank Preparation

(X) Cloud Platform IES Management of Assignments, Tasks, Quizzes:

The Cloud Platform IES is a learning assessment management platform and a learning assessment data convergence platform. When school teachers assign learning assessment assignments, learning assessment tasks, and learning assessment quiz activities on the cloud platform, students are automatically notified on the student-side AClass ONE. School teachers can also manage the progress of learning assessment activities, grade and guide students, calculate scores, and view learning assessment history data at any time on the Cloud Platform IES. Figure 12 shows the Cloud Platform IES learning assessment management page.

Figure 12: Cloud Platform IES Learning Assessment Management

(XI) Cloud Platform IES Integrated Test Sheet and Answer Card Grading System:

The Cloud Platform IES Integrated Test Sheet and Answer Card Grading System is a tool for digitizing paper-based assessment, as shown in the schematic in Figure 13. Test banks established in the Cloud Platform IES can be used with the quiz assembly function to automatically generate test papers and directly print answer sheets and answer cards, simplifying the cumbersome learning assessment workflow of question setting, assembly, preparation, and grading. Establishing multimedia test banks and assessment papers in the Cloud Platform IES has the advantage of resource accumulation and reuse. The Cloud Platform IES test bank assembly has three quiz assembly functions, including:
1. Smart Assembly: The system intelligently assembles the test paper from the question bank based on subject, difficulty, and question type.
2. Selective Assembly: Teachers manually select questions from the question bank to form a test paper.
3. Guided Assembly: Use Word directly to compile questions and import them into the cloud quiz paper.

OMR Exam Scanning and Scoring System
Figure 13: Schematic Diagram of the Cloud Platform IES Integrated Test Sheet and Answer Card Grading System

(XII) Cloud Platform IES Score Calculation and Learning Assessment Analysis:

The most important initiative in digital learning assessment design is the completion of automatic convergence and automatic analysis of all learning assessment data. The Cloud Platform IES Score Calculation and Learning Assessment Analysis include score calculation, subject assessment data analysis, grade-level assessment data analysis, class assessment data analysis, cross-class learning assessment data analysis, cross-subject learning assessment data analysis, and historical learning assessment data analysis, providing an overall, complete, and data-driven analysis of student learning outcomes. Based on this, it studies student learning effectiveness in each subject, identifies strengths and weaknesses in knowledge points, and provides differentiated teaching guidance strategies for different classes and individual students. It can also optimize the teaching process, more effectively achieve teaching goals, and improve teaching effectiveness.

Learning Status Analysis System
Figure 14: Cloud Platform IES Learning Capability Distribution and Stability

Digitalization of learning assessment is a systematic project that must cover multiple learning assessment types and diverse learning assessment needs. The preceding section, using the TEAM Model System as an example, specifically illustrated the implementation functions of digital learning assessment design, providing a more intuitive understanding of the philosophy and practice of digital learning assessment design.

#DigitalizationOfLearningAssessment #LearningProcessVisualization

Go Back