Oskar KU, Sung-Bin CHANG/HABOOK Group, Taiwan
*steven@habook.com.tw, power@habook.com

Abstract: Reflecting on one’s own teaching practice is a critical process for promoting his/her own teacher professional development. However, it is not easy for teachers to reflect on their teaching practice effectively without sufficient information. In past decades, many studies were undertaken to develop efficient methods for recording and analyzing teachers’ teaching behavior for helping teachers rethink their teaching practice. Those methods, however, were laborious or expensive. With the widespread adoption of smart classrooms, we are able to utilize their abilities of automatic data collection and thus develop an automatic teaching behavior analysis system, Sokrates Teaching Analytics System (STAS). STAS utilizes AI technology to analyze teachers’ teaching behavior which is automatically collected via HiTeach smart classroom interactive software. An integrated report of teaching analytics report is generated for teachers to reflect on their teaching practice right after the end of each class. Such a report not only helps teachers grasp their teaching practice in a class but also serves as a tool that experts can use to make concrete suggestions for teachers to promote their teacher professional development.

Keywords: Sokrates, teaching analytics, teaching behavior, smart classroom, teacher professional development

“How to improve teacher professional development? and how to make students learn effectively?” are common questions that frequently raised in teachers’ minds. Past studies indicated that reflection on teaching practice plays an essential role that may help teachers find the answers to the above questions during the process of teacher professional development (Darling-Hammond & McLaughlin, 1995; Šarić, & Šteh, 2017). More specifically, reflecting on personal teaching practice provides opportunities for teachers to re-examine what they have performed in their classes, which, in turn, helps them gain deep understanding of their teaching practice, and thus improve their teacher professional development (Loughran, 2002; Osteman, & Kottkamp, 1993; Schön, 1983). In general, reflecting on teaching involves recalling and writing down critical classroom events in detail, thinking the issues that have been written down critically, and finally making improvements to future teaching practice. 

When teachers reflecting on teaching practice, the first thing that teachers have to do is mining important information that influences their practice from their memory. More importantly, the quantity and the quality of what teachers recalled may influence later process of reflection. In other words, the more details teachers recall, the more problems can be identified; the more problems are identified, the better reflection results can be obtained. Recalling critical episodes in detail plays such an important role in teaching reflection, yet it is difficult to achieve. It may be one of the reasons why many teachers cannot obtain effective results from teaching reflection. Such a phenomenon is usually due to the fact that teachers have no enough time to recall and record these issues in time in their routine teaching practice. Furthermore, the detail of critical episodes being recorded would disappear gradually as time goes by. Thus, there is a need to help teachers record detailed information of important episodes occurred during their teaching practice.

In order to observe and record important episodes and interactive behavior during teachers’ teaching practice effectively, many studies were conducted in past decades. Those studies developed various methods to observe and record teachers’ teaching practice, which involves traditional instruments (e.g. video recorders, voice recorders, special designed sheets, and computer-aided event recorder) and modern instruments (e.g. eye tracker, portable EEG, gyroscope, and accelerometer). Compared with recalling from memory, data collected from these instruments not only provide researchers with more real-time and data, but also capture more details that teacher may not be aware of. Although these instruments present extensive details of teachers’ teaching practice, several issues, such as high human resource costs, and expensive equipment, make these approaches difficult to be comprehensively applied to routine teaching practice. Recently, the wide application of digital classroom software and hardware seems to shed light on resolving these issues. Thus, we tried to design an automatic teaching behavior analysis system, “Sokrates Teaching Analytics System (STAS)”, based on an on-shelf digital classroom system, HiTeach, aiming at facilitating teachers reflect on their teaching practice and improving their professional development consequently. 


2.    The observation and analysis of teaching behavior

The interaction behavior between teachers and students brings great impact on students’ classroom experience (Skinner & Belmont, 1993). Thus, quite a few researchers devoted plenty of time to investigating and developing effective approaches to observe and analyze teachers’ teaching behavior (e.g. Flanders, 1961; Gorham, 1988; Hughes, 1962; Liang, Huang, & Tsai, 2012; Pennings, Brekelmans, Wubbels, van der Want, Claessens, & van Tartwijk, 2014; Schempp, McCullick, Pierre, Woorons, You, & Clark, 2004). These studies consequently formed a research area: teaching analytics. More specifically, teaching analytics is an approach which records, analyzes, and evaluates how teachers deliver their lectures to students through various tools. The purpose of teaching analytics focuses on helping teachers gain deep insight into how they taught, reflect on their teaching practice, and improve their teaching skills (Gauthier, 2013; Prieto, Sharma, Dillenbourg, & Jesús, 2016; Sergis & Sampson, 2017). 

As time progresses, the tools applied to teaching analytics were advanced with new technologies. In 1960s to 1980s, researches mainly adopted paper-based approaches, such as Flanders Interaction Analysis Category System (Flanders, 1966), S-T instructional analysis (Fujita & Yoshimoto, 1980), and Seating Chart Observation Records (Gall & Acheson, 1980), to collect data. Later on, camcorder became a popular device to record classroom interaction behavior since the first consumer camcorder was introduced in the middle of 1980s. Owing to the capability of capturing almost every detail of classroom lessons, videotaping was recognized as an effective method to evaluate the quality of how teachers delivered their lessons (Ball, & Cohen, 1999) and promote professional development (Borko, Jacobs, Eiteljorg, & Pittman, 2008).

Almost at the same time, with the popularization of personal computers, more and more researchers started to use computers to analyze their experimental data thanks to the strong computing power. In the past decade, traditional paper-based approaches seemed to be replaced by newly designed computer-aided applications, such as Q-TSI (Hershkovitz, Merceron, & Shamaly, 2015), I/O Classroom Analysis Application (Ando, Aoyagi, Davis, & Kinefuchi, 2012), and the Version 3.8.1 of LessonNote (2018). These applications help researchers keep more details when observing classroom lessons. For example, LessonNote uses seating chart as the main body to record classroom interactions which is similar to the idea of SCORE, however it further provides teachers with functions to record voice notes, add photos, and keep a timestamp for every event automatically. Recently, researchers attempted to apply portable sensors which measure human physiological activities directly to record teachers’ teaching behavior (Prieto, Sharma, Dillenbourg, & Jesús, 2016). Such an attempt carved a new path for studying classroom interactions. The approaches mentioned above were developed with their own features, which focus on different perspectives of classroom interaction. The following sections give a brief review these approaches.



2.1. Flanders Interaction Analysis Category System (FIACS)
In order to analyze classroom interactions between teachers and students, Flanders (1966) developed FIACS, which records classroom verbal interaction with ten categories. An observer adds a new record into a 1010 matrix (Figure 1) every three seconds according to the content of communication between teacher and students. For example, a 4-8-2 interaction sequence is composed of “a teacher initiates a question (4)”, “a student responses to the question (8)”, “and the teacher praises the student (2)”. This sequence can be translated into two records: 4-8 and 8-2. These two records then add one to cell (4, 8) and cell (8, 2), respectively. After completing the matrix, researchers could calculate the value of interaction variables, such as percent teacher talk (PTT), percent pupil talk (PPT), teacher response ration (TRR), and discover the patterns of classroom interaction. Flanders divided the FIACS matrix into 11 areas; each area represents an educational meaning. For example, area H (shadowed area in Figure 1) represents teacher statements evoke students’ responses. In addition, high frequencies appeared in cell (4, 8) and (8, 4) (cells with thick border) might imply the teacher administer drill-and-practice in the class.

 

The FIACS matrix.

2.2    S-T Instructional Analysis
The S-T instructional analysis, which was proposed by Fujita and Yoshimoto (1980), presents teacher-student interaction via binary representation. The major difference between S-T analysis and FIACS is the recorded content of interaction. More specifically, the S-T analysis does not categorize the content of interaction; it only records the subject who is carrying out a classroom activity in each sampling period. Student activity is recorded as S while teacher activity is recorded as T. The sampling period of S-T analysis is adjustable; 30 seconds is a commonly used value for a regular class. At the end of a class, the recorded activities would form a character string which is composed of S and T and may look like “TSSSTS…STS”. 

The character string can be translated into an S-T chart (Figure 2). Furthermore, the string can also be calculated to obtain an Rt value and a Ch value. The Rt value represents the percentage of teacher activity while the Ch value represents the activity change rate. For example, the Rt value of “STTTS” is 0.6 (3/5).  The Ch value of “STTTS” is 0.5 (2/4) because there are two activity changes (S to T at the beginning and T to S at the end of the string) and two non-changed activity pairs (T to T). The Rt and Ch values can then be drawn on the Rt-Ch Chart (Figure 3). For example, the values of Rt and Ch obtained from the data in Figure 2 is shown as a diamond dot, which is located at the practice area in Figure 3 due to low teacher activity and low activity change.


S-T Chart.
2.3    Seating Chart Observation REcords (SCORE)
One of the intuitive instruments for recording teacher-students interaction is using a seating chart. Such an instrument is referred as Seating Chart Observation REcords or SCORE in abbreviation (Gall & Acheson, 1980). SCORE directly represents the classroom settings so that on the one hand observers can quickly draw the target behavior on the chart, on the other hand teachers can interpret the result with less effort. As shown in Figure 4, the star is the original place where the teacher stands; the teacher moves along the path connected to the star and finally stops at the cross mark. The question mark shown in Ann’s seat indicates Ann has asked a question. The straight marks indicate students respond to teacher’s statements. There is no a fixed form of SCORE recording sheet. This example simply demonstrates a kind of SCORE applications; a great variety of SCORE recording sheets and recording protocols could be found in other studies.


Seating Chart Observation Records.

2.4 Quantifying Teacher-Student Interactions (Q-TSI)

The interface of Q-TSI.

Q-TSI (Hershkovitz, Merceron, & Shamaly, 2015) is a classroom activity recording application (Figure 5), executed on Android tablet PCs. The design of Q-TSI is a bit similar to the combination of SCORE and FIACS. Q-TSI records a teacher’s locations in a classroom. In addition, the interaction content between teacher and students is divided into three main categories: (1) public and content related events, (2) public and non-content related events, and (3) non-public events. Each category consists of several items. By combining the collected data, detailed classroom activities can be reproduced. For example, the teacher stood by Jeremy (location #8) and give him a private instruction. Generally speaking, Q-TSI integrates the strengths of category system from FIACS and the behavior of movement from SCORE. Although the items that Q-TSI records are fewer than FIACS and SCORE, it records time as a new dimension of data, which is not available in the two traditional approaches.

2.5    Teaching Analytics with portable sensors

Recently, Prieto, Sharma, Dillenbourg, & Jesús (2016) conducted a study, which collected teachers’ eye-movement data, electrical activity of teachers’ brain, video that recording teachers’ field of view, voice, and 3-axis accelerometer readings with mobile eye-tracker goggles, a single-electrode portable electroencephalogram (EEG), and a smartphone. This study aimed at investigating the potential of applying AI technologies to extract teachers’ teaching activities automatically, including task distribution, explanation, questioning, monitoring, and repairs, during a lesson from the collected data. The preliminary result of this study demonstrated 90% accuracy on discriminating between small group and whole class activities while the reliability of detecting the five concrete teaching activities remained room for improvement.

Despite the instruments mentioned above possess their own strengths, they have some limitations unavoidably. For example, the facets of data collected with FIACS, S-T analysis, and SCORE are limited; other important features which may increase the reliability of data are missed. In addition, the results obtained through FIACS and SCORE are accumulated data which are not able to be analyzed with time dimension. Although computerized programs such as Q-TSI resolved this issue, they are still labor-intensive, especially for the situation when recording multiple independent activities which take place concurrently. Actually, the biggest challenge for either the classic or current approaches is they are labor-intensive on coding and analyzing (Prieto, Sharma, Dillenbourg, & Jesús, 2016). More importantly, the analyzed results are usually not easy to be understand, and thus needed to be explain by experts. The work conducted by Prieto and his colleagues (2016) attempted to automatically collect and analyze teachers’ teaching behavior, which in turn greatly reduced experts’ loads on coding and data analysis. However, such an approach requires teachers to wear additional devices, which may cause teacher to feel uncomfortable. In addition, the devices adopted in that study are mainly used research; they are still expensive and not widespread. In this vein, we started to think if it is possible to collect and analyze the data of teachers’ teaching behavior without allocating additional human resource and requiring teachers to wear additional devices, and meanwhile provides teachers a straightforward report letting them grasp how they deliver a 40-minute lesson in just one minute.

3.    Sokrates Teaching Analytic System

In order to achieve our goal, we designed and developed the “Sokrates Teaching Analytic System” (STAS) to collect and analyzing the data of teachers’ teaching behavior automatically in regular smart classrooms. With the support of Sokrates AI engine, teachers’ teaching behavior is translated into simple charts right after teachers finishing their lecturing so that they can reflect on their teaching practice with vivid memories. The design of STAS follows the following three principles:

(1)Collecting teachers’ teaching behavior in a more directly manner than traditional approaches. Traditional approaches require observers to observe and justify teachers’ activities, which involves more or less subjective points of view. Such manually collected data is not obtained directly from teachers’ activities and may lose its objectivity. Thus, the research team sought to design a method that records teachers’ teaching behavior directly.
(2)Collecting and analyzing data automatically and reducing the cost of human resource. There is a great variety of teaching and learning activities took place in classrooms. The best traditional way to collect these data thoroughly is allocating a large number of well-trained observers. Even we videotaped the classroom activities, it still spends experts a lot of time on coding and analyzing data. Moreover, the results of analyzed data require experts to provide a clear explanation. Therefore, we attempted to collect and analyze teachers’ teaching behavior automatically in our design to reduce the cost of human resource.
(3) Providing concise information with data visualization techniques. As mentioned in the previous paragraph, the data of classroom activities possess various features, and the analyzed results of these data require experts to give an explanation. Experts, however, cannot stay at the same classroom all the time. Thus, our design uses elements that teachers are familiar with to increase the readability of the final reports. Teachers receive a personal report right after they finish their lecturing as if there is an expert sitting in their classrooms to comment their lecturing.

3.1HiTeach Interactive Teaching Software
In order to collect teachers’ teaching behavior directly, STAS is built upon a smart classroom environment, HiTeach (Figure 7). HiTeach is an interactive teaching system with a capability to seamlessly integrate various hardware, such as interactive white boards, document cameras, smart phones, tablet computers, and an interactive response system (IRS) (Huang, Liang, & Wang, 2001; Liu, Liang, Wang, & Chan, 2003), which provide teachers with various interaction and lecturing tools to deliver their lessons. For example, Figure 7 shows a teacher is showing a multiple-choice quiz to students with IRS integrated.

A screenshot of HiTeach.

3.2    System Design
All teaching activities conducted via HiTeach are recorded and served as the source of teachers’ teaching behavior data. As shown in Figure 8, the extracted data which contains meaningful teaching and interactive behavior is sent to the Sokrates cloud service after data cleansing. Teachers will find a teaching analytics report in their account right after they finish their lecturing.



The Sokrates Teaching Analytics System
STAS is a cloud data analytics service, which evaluates teachers’ teaching behavior through statistical and AI techniques and then generates an integrated report that summarizes teachers’ lesson deliveries. The report generated by STAS (Figure 9) consists of five charts: (1) the frequencies of interactive technologies, (2) the accumulated time on using interactive technologies, (3) technological interaction index, (4) pedagogical application index, and (5) the distribution of teaching behavior.


The Sokrates teaching analytics report.

Charts one and two illustrate the frequencies of and the total time on the usage of interactive tools with bar charts. These two charts provide teachers a summative view of tool usage so that teachers can get a sense of their tendency on applying the interactive tools.

Chart three shows the technological interaction index with an appearance of speedometer. The value of this index is determined by the data of tools usage. Teachers can quickly realize the level of technological intervention in their lessons. This value does not necessarily correlate to teaching quality since each lesson has its own learning objectives and requires different models of lecturing which may involve different levels of technological intervention. 

Chart four reflects how teachers apply technology based pedagogical methods in their lessons with an appearance similar to a radar chart due to using a cyclic structure to present data. Unlike a radar chart, the pedagogical application index (PAI) chart does not connect all values of variables to form a star-like area. The PAI chart reveals the information of how a teacher applies pedagogical methods in a lesson. Each variable in the PAI chart represents a pedagogical method; the value of each variable indicates the intensity of applying the specific pedagogical method. An index is given at the center of the PAI chart, which represents a teachers’ mastery and flexibility on the application of pedagogical methods in a lesson.

Chart five illustrates the distribution of a teacher’s teaching behavior in a lesson. The idea of this chart originates from the Gantt chart and history charts. In order to illustrate the relationship and chronological order among different types of teaching activity, this chart uses time as the main axis with individual tracks for each activity. Every activity is marked on its own track at the time point or interval of its occurrence. The distribution of teaching behavior presents an overall picture of a teacher’s teaching behavior, which depicts the interaction between teacher and students via HiTeach completely in a chronological order.

4.    Applications

Smart classrooms, like the adoption of smart phones, are becoming more and more popular. In contrast to traditional classrooms, smart classrooms provide more interaction channels, and generate much more data. If we can extract useful information from the data generated in a smart classroom, we may help teachers review and improve their teaching practice. That is why we design and develop the STAS. Besides teachers, the experts of lesson debriefing are also target users of STAS. The STAS serves as a new tool helping experts grasp the integrated information of a lesson when they are giving comments in a lesson debriefing session so that experts could spend more time on observing teachers’ teaching instead of recording teachers’ behavior. The following two sections describe the application situations of the STAS.

4.1    Smart Classroom Individual Improvement Mode 

It is not easy for teachers to remember all details of their teaching behavior when delivering lectures in smart classrooms. Even in traditional classrooms, it is still not easy for teacher to do the same thing. Researchers has told us our memory is not as reliable as we think. In fact, our memory is often interwoven with episodes created by our brains instead of what really happened (Bartlett, 1932). The STAS report provides teachers with a summative view of their teaching. The teaching behavior distribution chart help teachers recall the process of their teaching practice. Teachers can compare the result of their teaching practice with their lesson plans to examine whether their lesson plans need to be revised and to reflect on how to improve their teaching practice. 

4.2    Lesson Debriefing Room Mode

In addition to self-reflection, assimilating experts’ knowledge and experience is an alternative way to improve teacher professional development. In a traditional lesson debriefing setting, experts sit at the observation area of a classroom to observe and record a teacher’s delivery of a lecture. After the teacher finish delivering his/her lecture and a short break, experts make comments to the lesson, such as the strengths and the shortcomings of the teacher’s teaching, the points that could be improved, and the actions that should be avoided. Those comments, however, may be too abstract to be understood for the teacher. For example, an expert makes a comment which says the teacher conducted too many activities of picking out students and the teacher could conduct other activities to interact with students while the teacher does not realize such a situation. Hence, such a comment may fail to bring positive effect on the teacher’s teaching practice. If the expert makes the comment with the support of the STAS report, the teacher may grasp the situation quickly since the teacher’s teaching behavior is shown on the report completely. Moreover, if the expert found the teacher conducted a special pattern of teaching behavior that forms a good example for other visitor teachers, the expert could use the teaching behavior distribution chart to address his points. In short, the STAS report provides a concrete image about the teacher’s teaching behavior so that experts could make comments with concrete data.


5.    Conclusion
Reflecting on personal teaching practice plays as a critical role in improving personal teacher professional development. In order to analyze teachers’ teaching behavior and help teachers improve their teaching practice, a great number of studies were conducted to develop efficient methods to record the details of teachers’ teaching behavior in past decades. Those methods focus on different features, such as the content of dialog between teachers and students, the frequencies of conducting specific interactive activities, the time points and the ratio of the occurrence of specific activities. Although these methods have their own strengths, they have some limitations when putting into practice. These methods, on the one hand, require a lot of human resource. On the other hand, there is a great deal of classroom interaction occurring in highly interactive smart classrooms which is different to those occurring in traditional classrooms. Therefore, such interaction data might not be able to be measured with the methods developed in traditional classroom environment. To address this issue, we develop a system, STAS, to collect and analyze the interaction data generated in smart classrooms.

In TEAM Model smarter classrooms, all behavior of operating a smart classroom system could be translate into digital data, which makes it possible to record teachers teaching behavior in smart classrooms automatically. The collected data could then be used to generate a report of teaching analytics with the support of Sokrates AI engine. The teaching analytics report provides teachers with a graphical view so that they could grasp information that facilitates their teaching reflection with ease. In additional to teachers, the teaching analytics report provides experts with an alternative view of teachers’ teaching practice. It is similar to the situation when doctors diagnosing patients’ syndromes. More specifically, in addition to face-to-face check-ups, the results of comprehensive physicals exams (e.g. chest x-rays, CT scan, and MRI exam) provide doctors with detailed information about patients’ status which cannot be observed via face-to-face check-ups. Thus, doctors can make accurate diagnosis and give correct prescriptions. In the past, experts could only make comments according to their observation in classrooms. With the additional information from the STAS report, experts can give teachers more concrete and elaborate suggestions.

STAS provides a simple way to collect and analyze teaching behavior automatically, yet it has some limitations. Firstly, some classroom interactive activities are not undertaken via HiTeach system (e.g. pick out students orally), and thus they could not be perceived by STAS so far. Secondly, gestures and movements that possess educational meanings could not be perceived either. Lastly, the dialogue between teachers and students cannot be analyzed at present. All these issues need to be addressed in future studies.

Acknowledgments

We would like to thank the reviewers for their insightful comments and all the people who participated in this project.


References

Ando, A., Aoyagi, A., Davis, D, & Kinefuchi, M. (2012). Jyugyou no souhoukousei wo hyoukasuru taburetto PC taiou apurike-syonn no kaihatu [Development of a software application for tablet mobile devices to support the evaluation of interactive classes]. Mobairu Gakkaisi, 2(2), 75-80.
Ball, D. L., & Cohen, D. K. (1999). Developing practice, developing practitioners: Toward a practice-based theory of professional education. In G. Sykes and L. Darling-Hammond (Eds.), Teaching as the learning profession: Handbook of policy and practice (pp. 3-32). San Francisco, CA: Jossey-Bass.
Bartlett, F.C. (1932). Remembering: A Study in Experimental and Social Psychology. Cambridge, UK: Cambridge University Press.
Borko, H., Jacobs, J., Eiteljorg, E., & Pittman, M. E. (2008). Video as a tool for fostering productive discussions in mathematics professional development. Teaching and Teacher Education, 24(2), 417-436.
Darling-Hammond, L., & McLaughlin, M. W. (1995). Policies that support professional development in an era of reform. Phi Delta Kappan, 76(8), 597-604.
Flanders, N. A. (1961). Analyzing teacher behavior. Educational Leadership, 19(3), 173-180, 200.
Flanders, N. A. (1966). Interaction analysis in the classroom: A manual for observers. Ann Arbor, MI: University of Michigan
Fujita, H., and Yoshimoto, H. (1980). Binary hyouji ni yoru kyousi kyouiku no tame no jyugyou bunnseki –S-T jyugyou bunnseki [Using binary representation of data in analysis of instruction for purposes of teacher training—S-T instructional analysis]. Nippon Kyouiku Kougaku Zassi, 5(3), 119-128.
Gall, M. D., & Acheson, K. A. (1980). Clinical supervision and teacher development: Preservice and inservice applications. New York, NY: Longman.
Gauthier, G. (2013). Using Teaching Analytics to Inform Assessment Practices in Technology Mediated Problem Solving Tasks. Paper presented at the Second International Workshop on Teaching, Leuven, Belgium.
Gorham, J. (1988). The relationship between verbal teacher immediacy behaviors and student learning. Communication Education, 37(1), 40-53.
Hershkovitz, A., Merceron, A., & Shamaly, A. (2015). Teacher-Student Classroom Interactions: A Computational Approach. In O. C. Santos (Ed.), Proceedings of the 8th International Conference on Educational Data Mining (pp. 558-559). Madrid, Spain: International Educational Data Mining Society.
Huang C., Liang J., & Wang H. (2001) EduClick: a computer supported formative evaluation system with wireless devices in ordinary classroom. In C. Lee, S. Lajoie, R. Mizoguchi, Y. Yoo & B. Boulay (Eds.), Proceedings of International Conference on Computers in Education 2001 (pp. 1462–1469). Seoul, Korea: AACE-APC.
Hughes, M. M. (1962). What is teaching? One viewpoint. Educational Leadership, 19(4), 251-259.
Liang, T. H., Huang, Y. M., & Tsai, C. C. (2012). An Investigation of teaching and learning interaction factors for the use of the interactive whiteboard technology. Educational Technology & Society, 15(4), 356-367.
Liu T., Liang J., Wang H., & Chan T. (2003) Embedding IRS in classrooms to enhance interaction. In K. Lee & K. Mitchell (Eds.), Proceedings of International Conference on Computers in Education 2003 (pp. 117–125). Hong Kong: AACE-APC.
LessonNote (Version 3.8.1) [Computer software]. (2018, January 12). Retrieved April 26, 2018, from https://itunes.apple.com/us/app/lessonnote/id507466065?mt=8
Loughran, J. J. (2002). Effective reflective practice: In search of meaning in learning about teaching. Journal of teacher education, 53(1), 33-43.
Osteman, K. F. & Kottkamp, R. B. (1993). Reflective practice for educators: Improving schooling through professional development. Newbury Park: Corwin Press.
Pennings, H. J., Brekelmans, M., Wubbels, T., van der Want, A. C., Claessens, L. C., & van Tartwijk, J. (2014). A nonlinear dynamical systems approach to real-time teacher behavior: Differences between teachers. Nonlinear Dynamics, Psychology, and Life Sciences, 18(1), 23-45.
Prieto, L. P., Sharma, K., Dillenbourg, P., & Jesús, M. (2016). Teaching analytics: towards automatic extraction of orchestration graphs using wearable sensors. In D. Gašević, & G. Lynch (Eds.), Proceedings of the Sixth International Conference on Learning Analytics & Knowledge (pp. 148-157). Edinburgh, United Kingdom: ACM Press.
Rosaen, C. L., Lundeberg, M., Cooper, M., Fritzen, A., & Terpstra, M. (2008). Noticing noticing: How does investigation of video records change how teachers reflect on their experiences? Journal of teacher education, 59(4), 347-360.
Šarić, M., & Šteh, B. (2017). Critical Reflection in the Professional Development of Teachers: Challenges and Possibilities. CEPS Journal: Center for Educational Policy Studies Journal, 7(3), 67-85.
Schempp, P., McCullick, B., Pierre, P. S., Woorons, S., You, J., & Clark, B. (2004). Expert golf instructors' student-teacher interaction patterns. Research Quarterly for Exercise and Sport, 75(1), 60-70.
Schön, D. A. (1983). The Reflective Practitioner: How Professionals Think in Action. New York, NY: Basic Books.
Sergis, S., & Sampson, D. G. (2017). Teaching and learning analytics to support teacher inquiry: A systematic literature review. In A. Peña-Ayala (Ed.), Learning Analytics: Fundaments, Applications, and Trends (pp. 25-63). Cham, Switzerland: Springer International Publishing.
Skinner, E. A., & Belmont, M. J. (1993). Motivation in the classroom: Reciprocal effects of teacher behavior and student engagement across the school year. Journal of Educational Psychology, 85(4), 571-581.