Partial Interval Classroom Inquiry Observation System - PDF document

1 Partial Interval Classroom Inquiry Obser
Partial Interval Classroom Inquiry Observation System (Screenshot of online coding) Partial Interval Classroom Inquiry (PICI) Observation System – Teacher and Student Forms Category Definitions for Classroom Observations TEACHER OBS ERVATION (Categories are exclusive in each period of time) Organization (Org) 1. Inquiry Instruction (Inq; I): Delivering basic instructions for any inquiry activity. This includes introduction instructions o r passing out materials for an inquiry activity. These materials can be: an advanced organizer for an inquiry activity, prepared guidelines with basic instructions for the activity, or a data - collection worksheet. 2. Non - Inquiry Instruction (N - INQ; NI): Delivering basic/introductory instructions or passing out materials for a non - inquiry activity. 3. No Instruction (No Instr; None): Any non - instructional organizational functions (e.g., roll call, collecting homework, collecting lunch money). A ctivity (Act) 4. Inquiry: Any instruction delivered within the context of a student - conducted inquiry - based activity in which it precedes the development of the concept. This includes listening to students (i.e., eye contact with students without verbalizatio ns or nodding in response to student verbalizations), monitoring the class (i.e., present in the room, walking around looking at different student groups or individual students as they work), providing clarifying information on instructions (through questi ons, clarifying statements or redirecting students attention to THEIR or class data) but not providing additional concept information , or wait - time. Any activity that Copyright © 2012 University of Nebraska - Lincoln, Nebraska Center for Resear ch on Children, Youth, Families and Schools. For use with permission . extends the concept and/or applies the new knowledge. Anything that the students do eithe r alone or in small groups. This would include an activity in which students (a) collect data or converse among themselves about (b) developing testable questions, (c) making predictions or generating hypotheses, (d) variables (e.g., independent and depend ent variables, how to measure dependent variables), (e) need for repeated trials, (f) how to design the experiment, (g) how to collect or present data, or (h) how to present their findings (e.g., developing reports). This could also be a large group or cla ss - wide activity where the assigned topic of conversation is one of these items (b - h) and precedes data collection (e.g., testable questions, hypotheses, variables, repeated trials). This would also include instances in which students are conversing about these topics (b - h) in their small groups, either with their peers in those groups or with the teacher. Students sharing about their data only in their own small groups is considered Inquiry Activity. *Coders, please note the letter (a - h) in the comments s ection to indicate the type of activity taking place. 5. Non - Inquiry : Any activity that confirms a concept that has been previously described . Any instruction delivered within the context of a non - inquiry - based activity. This may include delivering “how to” instructions (e.g., specific, step - by - step instructions for how to manipulate materials), directly providing information to the students (through lecture, “telling,” note giving, reading, or through media), listening to students, monitoring the class, pro viding clarifying instructions, providing additional concept information (directly or indirectly) , or wait - time. Identical behaviors delivered in the contex

2 t of a non - inquiry activity versus an
t of a non - inquiry activity versus an inquiry activity are coded differently. *Anything students do, either alone or in small groups but NOT large group single conversation happening. 6. No Instruction : Unavailable for student interaction as they work on either an inquiry or non - inquiry activity. Examples include being on the phone, out of the room, or talking with a classroom visitor (e.g., non - classroom student or adult). Discussion (Ds) 7. Inquiry: The teacher guides discussion following the collection, presentation, or analysis of data (including observations, which may or may not be written down). Th e inquiry discussion may follow inquiry activity and be centered on the data that the students have collected (either as individuals or groups). This includes gathering and recording student - collected data, facilitating student discussion about the data or the concept, asking questions that prompt student discussion of the data or the concept, redirecting students’ attention to their data. All students have the opportunity to benefit from a single conversation about a topic. Inquiry discussion also includes occurrences in which students report out their data to the whole class or present their findings in a report or defend their findings. *Large group discussion of data 8. Non - Inquiry : Any discussion that is not inquiry discussion. The teacher may guide a di scussion in which the teacher presents information, concepts, or ideas related to science concepts. The teacher guides a discussion that does follow an inquiry - based activity but does not include discussion of data or guiding students to the concept. Alter natively, the teacher guides a discussion after the teacher provides the concept or makes important data comparisons him or herself. Discussion may also be an opening event, before the presentation of any information. * Opportunity for every student to par ticipate in and benefit from one large discussion 9. No Instruction : Unavailable for student interaction they engage in class discussion. Examples include being on the phone, out of the room, or talking with a classroom visitor (e.g., non - classroom student or adult). Copyright © 2012 University of Nebraska - Lincoln, Nebraska Center for Resear ch on Children, Youth, Families and Schools. For use with permission . Lecture (Lc) 10. Supports Student - Developed Concept: Teacher provides background knowledge for the concept to be developed; summarizes, recaps, reflects information that the students developed; or provides a label for the concept developed using scie nce vocabulary, e.g., teacher confirms what the students learned. This constitutes greater than 66% of the interval (e.g., 10 seconds) . 11. Presents the Concept: Teacher labels and defines the concepts for the students without their development of the concepts, e.g., they tell the students what they are going to learn using the vocabulary, the definition, the predicted results, etc. Talking ab out right and wrong answers to a worksheet also is coded here. This constitutes greater than 66% of the interval (e.g., 10 seconds). 12. Concept/Science Unrelated: Teacher talks about information un - related to the concept for greater than 33% of the interv al (e.g., 6 seconds). *Lecture includes question - response patterns (Emily Van Zee) in which the teacher asks students questions in which there is a right answer in order to confirm students’ understanding. Lecture also includes students who ask a question and the teacher gives the answer. *Talking about worksheet ALWAYS stays in lecture. Worksh

3 eet (Ws) 13. Inquiry : Any student
eet (Ws) 13. Inquiry : Any student activity relating to paper work following the collection of data by the student (e.g., questions from the textbook, completing a works heet, completing a laboratory worksheet or guide, or completing a handout with numbered problems or questions). This activity may occur as independent work but may also include group work. This includes monitoring the class, providing clarifying instructi ons, wait time, redirection to student data or listening. Teacher assigns to, prepares for, or monitors students as they work on any paper work related to student data collected during the investigation. Examples of paper work include questions from the te xtbook or completing a handout with numbered problems or questions. 14. Non - inquiry : Any teacher activity relating to paper work (e.g., questions from the textbook, completing a verification laboratory worksheet following a lecture or presentation of science c oncepts by the teacher, or completing a handout with numbered problems or questions). This activity most likely occurs as independent work but could also include group work. This includes monitoring the class, providing clarifying instructions, wait time, or listening. Teacher assigns to, prepares for, or monitors students as they work on any paper work not related to an inquiry activity. Examples of paper work may include questions from a textbook, completing a laboratory handout or student guide, or comp leting any other handout with numbered problems or questions directly related to the data that the students have collected. 15. No Instruction : Unavailable for student interaction they engage in class discussion. Examples include being on the phone, out of the room, or talking with a classroom visitor (e.g., non - classroom student or adult). *Code as worksheet if students are completing a worksheet independent of any other categories. Copyright © 2012 University of Nebraska - Lincoln, Nebraska Center for Resear ch on Children, Youth, Families and Schools. For use with permission . STUDENT OBS ERVATION (Exclusive categories and when coding teacher and stud ents in the same interval, the categories should be identical) Organization (Org) 1. On - Task : Absence of off - task behavior and engaging in typically expected classroom behavior (e.g., listening to teacher during instructions, softly talking with other students when teacher is not addressing the class, assisting with organization) for at least 66% of the interval (e.g., 10 seconds). 2. Off - Task : Any incidence of typically disruptive classroom behavior (e.g., talking loudly, throwing objects, physical or ver bal aggression) or not attending to the teacher while they address the class for at least 33% of the interval (e.g., 6 seconds). 3. Inquiry Engagement : On - Task within the context of an inquiry organization activity 4. No Instruction: If teacher behavior coded a s “no instruction,” then circle No Instruction for student behavior Activity (Act) 5. On - Task : Absence of off - task behavior and engaging in typically expected classroom behavior (e.g., listening to teacher during instructions), manipulating materials, or par ticipating in small group discussion with peers or the teacher for at least 66% of the interval (e.g., 10 seconds). If some students finish their activities earlier than others and just start talking with their partners, count it as on - task. 6. Off - Task : Any incidence of typical disruptive classroom behavior (e.g., yelling, throwing objects, inappropriately manipulating materials, physical or verbal aggression); o

4 r not attending to the teacher while
r not attending to the teacher while they address the class for at least 33% of the interval (e.g., 6 seconds). If some students finish their activities earlier than others and start playing with the materials supposed to use for the activity, count it as off - task. 7. Inquiry Engagement : On - Task within the context of an inquiry activity Discussion (Ds) 8. On - Task : Engaging in typically expected classroom behavior; sharing their data or observations in response to the teacher prompts; attending to speaker (eyes orientated towards speaker with mouth closed); or asking additional questions related to the activity or concept. 9. Off - Task : Any incidence of typical disruptive classroom behavior (e.g., yelling, throwing objects, inappropriately manipulating materials, physical or verbal aggression); or not attending to the teacher while they address the class fo r at least 33% of the interval (e.g., 6 seconds). 10. Inquiry Engagement : On - Task within the context of an inquiry discussion 11. No Instruction: If teacher behavior coded as “no instruction,” then circle No Instruction for student behavior Lecture (Lc) 12. On - Task : Engaging in typically expected classroom behavior; attending to speaker (eyes orientated towards speaker with mouth closed); asking additional questions related to the activity or concept 13. Off - Task : Any incidence of typical disruptive classroom behavior (e .g., yelling, throwing objects, inappropriately manipulating materials, physical or verbal aggression); or not attending to the teacher while they address the class for at least 33% of the interval (e.g., 6 seconds). Copyright © 2012 University of Nebraska - Lincoln, Nebraska Center for Resear ch on Children, Youth, Families and Schools. For use with permission . Worksheets (Ws) 14. On - Task : Engaging in typically expected classroom behavior; attending to speaker (eyes orientated towards speaker with mouth closed); eyes and body oriented towards materials (working on the worksheet); or asking additional questions related to the activity or concept 15. Off - Tas k : Any incidence of typical disruptive classroom behavior (e.g., yelling, throwing objects, inappropriately manipulating materials, physical or verbal aggression); or not attending to the teacher while they address the class for at least 33% of the interva l (e.g., 6 seconds). 16. Inquiry Engagement : On - Task within the context of an inquiry discussion The Partial Interval Classroom Inquiry Observation System (PICI) (Kunz, et al., 2010) is a measure that was developed as a system used to conduct, record, and score direct classroom observations of science inquiry teaching practice and students inquiry engagement for classrooms in which inquiry is being taught (e.g., s cience, math). There are two observation forms used as part of the PICI: direct observation of the teacher behaviors that demonstrate inquiry teaching practice (PICI - T) and direct observation of the classroom students’ behaviors that indicate inquiry engag ement (PICI - S). These two are designed to be used together, although the PICI - T can be used without the PICI - S. The PICI uses a specific type of interval recording procedure – partial interval recording. One way to conduct a direct observation of behavio ral targets (for teachers and students) is to use a procedure called interval recording which is an approach to measuring whether a behavior occurred or did not occur during specific intervals of time (Cooper, 1987). Partial interval recording is one form of interval recording that

5 provides an estimate of the rate and dur
provides an estimate of the rate and duration of behavior and involves dividing the observation session into equal intervals (e.g., 10s) and scoring an interval as positive if the target behavior occurs at any point during the in terval (Fisher, Piazza, & Roane, 2011; Shapiro & Kratochwill, 2000). However, for behaviors of duration (e.g., on - task, off - task), it is recommended to specify a length of continuous presence of the behavior in order to determine the presence of absence of that behavior (e.g., 3 sec, more than half the interval, etc.) (Austin & Soeda, 2008; Northup, et al., 1999; Ridgway, et al., 2003). In order to minimize error and maximize accuracy of measurement, researchers suggest that small interval sizes (e.g., 10 s econds) should be employed (e.g., Rapp, Colby - Dirksen, Michalski, Carroll, and Lindenberg, 2008). Partial interval recording has been used to measure a wide - range of student behaviors (e.g., on - task, off task, inappropriate vocalizations, out of seat, fid geting, playing with objects, vocal stereotypy and disruptive behaviors such as aggression and property destruction (e.g., Haley, Heick, & Luiselli, 2010; Kern & Dunlap, 1994; Morrison, Sainato, Benchaaban, & Endo, 2002; Ridgway, Northup, Pellegrin, LaRue, & Hightshoe, 2003). The PICI was developed for middle and high school classrooms, particularly science but have also been piloted in math classes. The total observation session is intended to be the entire class period. In the PICI, intervals are divide d into 15 seconds. Pilot data that were collected during the development of the instrument indicated that a 15 - second interval was long enough to detect the teacher and student behaviors but short enough to capture one predominant behavior during intervals . In other words, there were not several different types of behaviors occurring during 15 seconds that would have resulted in missed behavioral recordings. The specific teacher behaviors coded in the PICI - T are skills or instructional practices in which the teacher engages that provide opportunities for students to engage in inquiry (labeled as “inquiry”) or that do not promote opportunities for student inquiry engagement (labeled as “non - inquiry”). There is also a behavioral code for “no instruction” (e .g., teacher is unavailable because of being on the phone, out of the room, or talking with a classroom visitor, or general classroom routines that are not related to Copyright © 2012 University of Nebraska - Lincoln, Nebraska Center for Resear ch on Children, Youth, Families and Schools. For use with permission . instructional concepts such as taking roll or collecting lunch money). These three types of behaviors are recorded in one of five categories that describe the context in which the instruction occurs: organization, (student) activity, discussion, (teacher) lecture, and worksheet. Behaviors are coded by category and instruction type combined, re sulting in 15 possible behaviors in each interval (e.g., Inquiry Activity, Non - Inquiry Worksheet, No Instruction Organization). Observers select which one of the 15 behavior options occurred during a given interval. For the “lecture” category, which is con sidered “non - inquiry,” there are three behavior options: the “supports student - developed concept”) behavior must be present for at least 10 seconds of the interval to be coded; similarly for the “presents the concept” behavior. But, if the teacher talks ab out information un - related to the concept for greater than 33% of the interval (e.g., 6 seconds), then “concept/science unrelated” is coded.

6 The PICI - S generates an estimate of
The PICI - S generates an estimate of the whole class based on rotational observation of the students. For four cons ecutive intervals (i.e., 1 minute), a single student is selected at random by the observer, and that student’s individual behavior is coded for those four intervals. For the next four intervals, another student is selected at random by the observer. This s ame process continues until the observation period ends. In general, each student is observed approximately 2 - 3 times; however, that varies by the number of students in a class and the duration of the class period. Data collected during the development of the PICI - S supported that this method of random selection of students observed for one minute each resulted in an accurate representation of the class behavior as a whole. The specific student behaviors coded in the PICI - S are “on - task” [engaging in typica lly expected classroom behavior (e.g., listening to teacher during instructions, eyes oriented to the speaker) for at least 66% of the interval (e.g., 10 seconds)]; “off - task” [any incidence of typically disruptive classroom behavior (e.g., talking loudly, throwing objects, physical or verbal aggression) or not attending to the teacher while they address the class for at least 33% of the interval (e.g., 6 seconds)]; and “inquiry engagement” (i.e., on - task within the context of a teacher inquiry instruction opportunity). In other words, students are coded for on - task or off - task, and can also be coded as inquiry engagement if they are on - task and the teacher engaged in an inquiry - type instructional practice for that same interval. Likewise, the category (e.g. , organization, activity, discussion) is coded based on the teacher behavioral category for that same interval. There is also a “no instruction” option for organization and discussion, and there is no “inquiry engagement” option for lecture. Thus, there ar e 16 possible student behavioral codes in each interval. Only one is coded per interval with the exception of inquiry engagement; therefore, it is possible for a student to be both on - task and inquiry engaged in the same interval if the teacher demonstrate d an inquiry instructional practice during that interval. Because there are 15 teacher behaviors and 16 student behaviors possible for each 15 - second interval and because it often takes some time observing to understand the context of the teaching sessio n, coding is conducted via video - recorded sessions rather than in real time. While the PICI - T can be coded by two independent observers separately for inter - rater reliability, coding for the PICI - S must be conducted with both observers watching the same vi deo at the same time so that they can know which students are to be observed for which intervals. There is a paper - and - pencil observation form available for the PICI, but we developed an electronic observation system for the PICI that we are using in a cur rent, large - scale randomized control study. Scoring is done through a computer program as is the calculation of inter - observer agreement. To calculate inter - rater agreement with partial interval recording, the total number of agreements are divided by th e total number of agreements plus disagreements and multiplied by 100%. For the PICI, we used the most conservative method of calculating agreement: the behavioral codes needed to match exactly of the 15 combinations for the PICI - T and the 16 for the PICI - S within the same interval . Inter - rater reliability with data from CSI: Coaching Science Inquiry in Rural Schools (Institute of Education C

7 opyright © 2012 University of Nebraska -
opyright © 2012 University of Nebraska - Lincoln, Nebraska Center for Resear ch on Children, Youth, Families and Schools. For use with permission . Sciences grant # R305C090022) was above 85% and indicated substantial agreement (Kappa = .89) for the PICI - T and was above 85% with substantial agreement (Kappa = .85) for the PICI - S (Landis & Koch, 1977). Use of the partial interval recording (PAR) observation system in the PICI is novel is several ways. First, PAR is much more commonly used among elementary students than any other grade level. Second, PAR is generally used to observe one target student in a classroom rather than as a way to measure class - wide behavior. Finally, of publis hed studies measuring student engagement in science and math classes, very few have indicated using partial interval recording. References Austin, J. L., & Soeda, J. M. (2008). Fixed - time teacher attention to decrease off - task behaviors of typically d eveloping third graders. Journal of Applied Behavior Analysis, 41 , 279 - 283. Cooper, J. O. (1987). Measuring and recording behavior. In J. O. Cooper, T. E. Heron, & W. L. Heward (Eds.), Applied behavior analysis . New York, NY: Prentice Hall. Fisher, W. W., Piazza, C.C., & Roane, H.S. (2011). Handbook of applied behavior analysis . New York, NY: Guilford Press. Haley, J. L., Heick, P. F., & Luiselli, J. K. (2010). Use of an antecedent intervention to decrease vocal stereotypy of a student with autism in the general education classroom. Child & Family Behavior Therapy, 32 , 311 - 321. Kern, L., & Dunlap, G. (1994). Use of a classwide self - management program to improve the behavior of students with emotional and behavioral disorders. Education & Tre atment of Children, 17 , 445 - 458. Kunz, G. M., Fluke, S., DeChenne, S. E., & Pedersen, J. (2010). Partial interval classroom inquiry observation system (PICI): teacher and student form (PICI - T, PICI - S) Unpublished measure. University of Nebraska−Lincoln. Mo rrison, R. S., Sainato, D. M., Benchaaban, D., & Endo, S. (2002). Increasing play skills of children with autism using activity schedules and correspondence training. Journal of Early Intervention, 25 , 58 - 72. doi: 10.1177/105381510202500106 Northup, J., Fu silier, I., Swanson, V., Huete, J., Bruce, T., Freeland, Edwards, S. (1999). Further analysis of the separate and interactive effects of methylphenidate and common classroom contingencies. Journal of Applied Behavior Analysis, 32 , 35 - 50. Rapp, J. T., Col by - Dirksen, A. M., Michalski, D. N., Carroll, R. A., & Lindenberg, A. M. (2008). Detecting changes in simulated events using partial - interval recording and momentary time sampling. Behavioral Interventions, 23 , 237 - 269. doi: 10.1002/bin.269 Ridgway, A., No rthup, J., Pellegrin, A., LaRue, R., & Hightshoe, A. (2003). Effects of recess on the classroom behavior of children with and without attention - deficit hyperactivity disorder. School Psychology Quarterly, 18 , 253 - 268. Shapiro, E.S. & Kratochwill, T.R. (2 000). Behavioral assessment in schools: Theory, research, and clinical foundations . New York, NY: Guilford Press. Psychometrics: PICI - Teacher Kappa = .90 PICI - Student Kappa = .85 Suggested Citation: Kunz, G., Berry, B., Fluke, S., Pedersen, J., DeChenne S. E., Nugent, G., & Houston, J. (2012). Partial Interval Classroom Inquiry (PICI) Observation System for Teacher s (PICI - T) and Student s (PICI - S) . University of Nebraska - Lincoln, Lincoln, NE: Coaching Science Inquiry in Rural Schools. Retrieved from http://r2ed.unl.edu/CSI/PI