Testim ony to the National Mathem atics Panel Steven Ritter Carnegie Learning Inc

Testim ony to the National Mathem atics Panel Steven Ritter Carnegie Learning Inc - Description

John Anderson Carnegie M llon Unive sity Nove er 6 2006 Cogni tive Tutor Tracking learning in real tim Thank you ve ry m ch for i nviting us to present in formation about how in our development of the Cognitive Tutor curricula we have ap plied basi ID: 21725 Download Pdf

177K - views

Testim ony to the National Mathem atics Panel Steven Ritter Carnegie Learning Inc

John Anderson Carnegie M llon Unive sity Nove er 6 2006 Cogni tive Tutor Tracking learning in real tim Thank you ve ry m ch for i nviting us to present in formation about how in our development of the Cognitive Tutor curricula we have ap plied basi

Similar presentations

Download Pdf

Testim ony to the National Mathem atics Panel Steven Ritter Carnegie Learning Inc

Download Pdf - The PPT/PDF document "Testim ony to the National Mathem atics ..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.

Presentation on theme: "Testim ony to the National Mathem atics Panel Steven Ritter Carnegie Learning Inc"— Presentation transcript:

Page 1
Testim ony to the National Mathem atics Panel Steven Ritter Carnegie Learning, Inc. John Anderson Carnegie M llon Unive sity Nove er 6, 2006 Cogni tive Tutor: Tracking learning in real tim Thank you ve ry m ch for i nviting us to present in formation about how, in our development of the Cognitive Tutor® curricula, we have ap plied basi c research to the im provement of mathem ati cs education. In these rem rks, we will provide som background a bout Carnegi Learning and then explai n what we consider scientifically ased resear ch to invol ve, how we have applied resear ch in

the construction of our curricula and what we view as the potenti al of tec hnolo to dramatically im prove mathematics education in the United States. The work tha led to Carnegie Learning’s Cognitive tors began i the ps hol ogy and computer science departm nts at Carnegie Mellon University. John An rson had been developi ng the ACT (later ACT-R) theor of co ition. ACT-R is a Unified Theor of ogniti on (Newell, 1973, 990) t at aims to explain the full range of hum an cognition. ACT-R was im plemented as a com puter program , which has the advantage of requiring t e theor to be precise about

all of its claims. Anderson had seen great success in using ACT-R to m odel laborator results in learning, mem ry and problem -solv ng (c.f. Ande rson, 19 83), and he was challenged to s how that the s ba sic ap proach could explain cogni tion o tside o a laborator en vironm ent. In its application to ps ho logical laborator studies, e aim of an ACT-R odel is to predict behavior. In order to pr edict behavior, the m odel needs to correctly represent hum an know ledge and also understand how that knowledge result s in particular behavior s. Applied to education, this representation of knowledge

results in predi tions about what student s can and cannot do as well as predictions about what activities and experiences will help studen ts learn to achieve curricular goals. The represent tion of know ledge inherent in this kind of m odel is called a “cognitive m odel, and the approach of using a cogniti ve m odel in a tutoring st em has co me to be called a “Cognitive T tor.” The first tutoring sy stems built in this way addressed co uter programming and m em atics (An erson, Boy e, Farrell & Reiser, 1987; Anderson , Conrad, & Co rbett, 198 Anderson, Boy e, Cor ett & Lewis, 1990). ANGLE,

a geometry pr oof tutor (Koedinger and Ande rson, 1993) w as successful in a school setting. Its success, however, appeared to be highly dependent on the teacher’s ability to integrate the tutoring software into broader classr oom goals. This, along wit Koedinger s personal experience teaching a geometry class, focused the resear ch group on the im portance of working with teachers and ad nistrator to understand schools’ curricular needs broadly. As a consequence, the research team
Page 2
for the products that became Carn egie Learning’s C ogniti ve Tutors in cluded Bill Hadley , w

ho had taught mathe atics for alm st 30 y ears and was the reci pient of the Presidential Award for Mathem at ics Teaching. This team set out to build curr icula that wer based in solid cogniti ve resear ch, were focused on erging national and state standards and that addressed the practical needs of students, teachers and ad nistrator . One decision was to include textbooks i addition to the software. The inclusion of text allow d the curricula to include som aspects ( uch as collabor ation, diagramming and writi ng abou t m them atics) that were easier to do on pa per than on th e co puter.

Th e com inati on of text and software also helped to position t e software as a regular, routin e part of th e mathe atics i struction. Instead of using the software as a “bonus” for advanced stud ents or as a review for stud ents who were lagging, t e h bri curricula set t e expectation that software could be us ed as part of the prim ary in struction. Pil t ple entati ons led to a odel whe e students used the software two day per week, with cl assroo activities stru ctured by the text the other three day each week. The curricula proved to be educationally succes sful (Koedinger, Anderson,

Hadley and Mark, 1997; Ritter and Anderson, 1995; Koedinge r, Corbett, Ritter & Shapiro, 2000) and popular with students and teachers. In our view, scientifically -based rese arc involves more than sim ly the dem onstration that a curriculum is effective. An essential co ponent is understanding why the curriculum is effective. Without a theoretical framework as a guide to unde rstanding the conditions that lead to effective thematics instruction wi thin a curriculum we have little hope of expanding and im proving instruction over time. We think of the process of building a curriculum a ha ving

four parts: Having a solid theoretical basis Apply ng the basic theory to the par ticul ar domain and objectives o interest Evaluating re sults Developing and im plementing a m thodolog y for im proving the cu rriculum based on use ACT-R (And erson, 1990, 1993; Anderson & Lebiere, 199 8; Ander on, Bothell, By rne, Dougl ass, Lebiere & Qin, 2004) forms the primary theoreti cal basis of the Cognitive T tors. The pri ary use of t e ACT-R theory has be en to reproduce i portant characte is tics of hum an behavior, inclu ing error patterns and response tim s. Most of thi work has been conducted in

the laborat or b t ACT-R has also been applied outside of the laboratory in areas r lated to hum an-com puter intera ction, training and education. This work has resulted in hundre s of publications (see http://act-r.psy.cmu.edu/publications/ nde x.php ). A full explan ation of ACT-R is bey nd the scope of th is testim ony, but som of the tenets important to education (Anderson, 2 ) include: There are two basic ty pes o knowledge: proce dural and declarative. Declarative knowledge includes facts, images and sounds. Proce dural kn owledge is an un rstanding of how to d thi ngs. All tasks

involve a co ination of the two t ypes of kn ow ledge. As we learn, we generally start ou t with declarative k nowledge, which becom proceduralized throu gh pr actice. Procedural kn owledge tends to be m re fl uent and automatic . Elements of procedural knowledge are referr ed to as “rules” or
Page 3
“productions ” because they specify t e conditions un der which they are applicable and the actions (including ch anges in mental state) that result fro appl ng them . Declarative knowledge tends to be flexible and also m broadly applicable than procedural kno wledge. We often refer to

ele ents of declarative kn owledge as “f acts. The knowledge required to acco lish com lex tasks can be descri ed as the set of declarative and procedur al knowledge com pone nts relevant to th e task. Both declarative and proce dural kn owledge beco me strengthened with use (and weakened with disuse). Strong knowledge can be re ered a nd called to attention rapidl y an d with som certainty . We ak knowledge may be slow, effortful or i possible to retrieve. Different knowledge co ponents may represent different strategies or thods for accom lishing a task (including incorrect strat gies or th

ods). The relative st rength of these co mponents helps deter ine which strategy is us ed. Learning inv lves the dev lopment and strengthe nin of correct, efficient and appropriate knowledge co ponents. It is im portan to understand that our terminology here differs so mewhat fro t e sa me ter s as they are used in an ed ucational con ext. For exa ple, a “procedure” in AC T-R is sim ly a co ponent of knowledge th at can produc e other knowl dge co ponent s and/or lead to external behavior. In mathe atics education, we m ght refer t the “pro ced ure” of solvin g a linear equation. An ACT-R odel

of that task would co nsist o many pr odu ctions and facts that are brought t bear. Even a sim le task like adding integers consist of m ny productions, including ones associated with recalling arithmetic facts, executi ng counting a tions, etc. (c.f. Lebiere, 1999). This view that e erge s from ACT-R i that lear ning is a process of encoding, strengthening and proceduralizing knowledge. This process happens graduall New knowle dge will be forgot ten (or rem in weak enough to stay unuse ) if it is not practiced, a nd elem ents of knowledge c et e to be used, based on their strength (Siegler &

Shipley, 1995). Si nce the ability to pe rform a task just relies on the indivi dual knowledge com ponents req ired for that task, educa tion will be m st efficient when it focuses students st directly on the in divi al knowledg e co ponents that have relatively l w strength. The interaction between declarative and proce dural knowledge leads to an em pha sis on active engagem nt with the conceptual underp innings of pr ocedures, so that students appropriatel generalize this knowledge (Rittle-Johnson & Siegl r, 1998; Rittle -Johnson, S egler & Alib ali, 2001; Ritt le-Johnson & Koedinger, 200

2; 20 05). Since procedural knowled e includes th e context in which it is applicable, educational activities need to be stru ctured such that students are able to practice procedures within an appropriate range of contex ts. This decom osition of co lex tasks into in divid al knowledge co ponents leads to a pedagogical odel emphasizing practice of indivi dua l com ponents, i ndependent of the larger task. At the sam time, we need to recognize that some knowledge com ponen s that are inherent to the larger task (such as integration of inform ation from different sm aller co onents), which prov

ides another rational for phasizing performance within an appropriate context. To use a sports analogy it is i portant for batters to t ke batting pr actice, because this will allow a baseball player to receive i tensive practice with st of the skills i nvolved with hitting a ball. However, it is also im rtant for the batter to pla in gam s, since so me skills (such as reading t e infield) will onl y co up in that context.
Page 4
Although the ACT-R theory indicates the basic pedagogical strategies likely t b effective in instruction, it does not specify the parti ular skills that co

rise t e abilit to, for exam ple, solve a linear equation. In order to create instruction i m thematics, we n eed to understand th e knowledge co ponents involved in co leting a particular task. It is not en ough to k now the com ponents invol ved i expert performance of the task; we a so need to know th e com onents exercised by students l earning to perform the t ask. Much of our applied research in mathematics has concerne d identif ng the particular skills and m hods that students use to com lete mathe ati cal ta sks (c.f. Corbett, McLaughlin, Scarpinna tto and Hadley, 20 00; Mark and

Koedinger, 1999; Koed inger and Anderson, 1990). Ofte n these skills do not correspond t expert beliefs (Koedinger & Nathan, 2004; Nathan & Koedinger, 2 000a; 2 000 b). One technique that we have used to un rstand how student approach mathe ati cs problems i to track their ey e m ove nts a the work through a proble (Gluck, 1999). Consider the task of a student co leting a table of values based on a word proble like that sh own in Figure 1. QuickTime™ and a Cinepak decompressor are needed to see this picture. Figure 1: Partially -completed wor probl em task u ed in ey e- tracking stud In Figure

1, t e student ha s co leted part of the tabl e corresponding to t e wor prob lem in cluding t e colum headi ngs, units of measure expression and the num ber of hours asked for in the first question. The student next needs to calculate the am unt of oney rem ining after two hours. The student might perform the t ask in at least two way . First, th ey m ght reason from the problem s cenario (perhaps im agining ha ving $20 an d then using re peated subtrac tion to calculate the oney left after spending $4 two times). T e second method would be to use the al gebraic expression and the substitute

2 for X and calculate the result. If a student has prod uced the table shown in Fi re 1 (includi ng writing th e algebraic expression for the am ount of m oney left), we ght expect that students would t en use the algebraic expression and execute the second method. In fact, Gl uck found that, about 13% of the time, when students were answering a question like question 1, th ey l ooked at t e problem scenario but n t the expression. 54% of the time, students looked at th e expression (someti es alon g with the scenario). Al st 34% of the tim e, t ey looked in neither place. As a result of this

and ot her data (c.f. Koedinge r & Anderson, 19 98) , the Cogniti Tutor curriculum treats finding the algebraic expression for si le word problem s as an induction t ask. Students are asked to solve the i ndividual que stions (“How m ch oney would be left after 2 hours?” and “When will you run ou t of m oney?”) first, and then use a generaliza tio n of their reasoning to com up with the algebraic expression. In later units of curriculum as the s ituations and algebraic expressio s become more
Page 5
co lex, we focus students on goi ng from the word problem to the expression and then

using the expression to co pute spec ific values. Bey nd the design of m th ati cal task s, the ACT-R theor guide instruction i the Cogniti ve Tutor because the software includes an activ e cognitive m odel, which is si ilar to an ACT-R odel within the software (Corbett, Koeding r & Anderso , 19 97). odel serves two purpose . First, the model follows student actions in order to deter ine the par ticular student’s strategy in s lving a prob lem The technique by which it does this is called model tracing . Second, each action that t e student takes is associat ed wi th one of m skills, which are

re ferences to knowledge com ponen s in the cognitive odel. Individual students performance on these skills is tracked over time (and display d to s udents in the “skillometer”). The Cognitive Tut r uses each student skill pr ofile to pick problems that em phasize the skills on which the student is weakest (Corbett and Anderson, 1995). In addition the skill odel is used to im plement stery learning. When all skills in a section of the curriculum are deter ed to be sufficiently m astered, the st udent m oves on to the next section of curriculum which introd uces new skills. The develop ent of

curriculum involves many decisions , and the e is often roo for disagreem ent about how learning theory should be app lied in particular ca ses. For that reason, we bel eve that care ul evaluation is an essential p rt of the process. Our development process has included many form ati e ev aluations of individua l units of instruction (e.g. Aleven & Ko edinger, 20 02 ; Corbett, Trask, Scarpi natto & Hadley , 1998; Koedin ger & Anderson, 1 998; Ritter and Anderson, 1995) . In add iti on, we have conducted several la rge evaluations of the ent re curriculum (taking text, so ftware and training

com pon ents as a single nipulation). Early evaluati ons of Cog it ive Tutors for programm and geom et ry showed great prom se, with effect sizes of approximately 1 st andard deviation (Anderson , Corbett, K edinger & Pelletier, 1995). In studies of the Algebr a I Cognitive Tutor conducted in Pitts burgh and Milw aukee (Koedi nger, Anderson, Hadle and Mark, 19 97), students were tested both o sta ndardized tests ( AT and Iowa) as w ll as performance- based problem-solving. C ogniti ve Tutor students signi ficantly out scored their peers on the standardized tests (by abo 0.3 standard

deviations), but the d fference in performance was p rticularly pr onou nced on tests of probl em -solving and m ltiple representations, where the Cognitive Tutor students outscored their peers by 85%, representing effect siz es f om 0.7 to 1.2 standard de viations. In Moore, O , a study was conducted where teacher were asked to teach so of their classes using Cognitive Tutor and som using the text book the ha d been previously using (M organ & Ritter, 2002; National Research Council, 2003). The s udy f ound that Cognitive Tutor student s scored higher on a standardized test (the ETS Algebra I

End of Cour se exam ), received higher grades, reported m confidence in their mathematical abilities and were re likely t be lieve that mat em atics will be useful to them outside of school. This stud w as recognized the US D partm nt of Education s What Works Clearinghouse as having met the highest standards of evidence. This stud showe effect sizes of approxim tely 0.4 st andard deviations. The Miam i-Dade Count s hool district studied the us e of Cogniti Tutor Algebra I in ten hi gh schools. An analy is o over 60 00 st udents taking the 200 3 FC AT (state exa ) showed that students who used

Cognitive Tutor significant y outscored t eir peers on the exam (Sarkis, 2004). T e findings were particularly am atic for special populations. Th e stud y foun d that .7% of Exceptional Stu ent Education students who us ed Cognitive Tutor passed the FCAT, as com ared to only 10.9% of such students using a different curriculum For students w ith Lim ited English Proficiency 27% of Cognitive
Page 6
Tutor student s passed the FCAT, as opposed to onl 18.9% of suc students in another curriculum ACT-R provides guideline for educational pedagogy a nd for constructing tasks that are likely

to increase learning. The theory also provides a way for us to test and im prove our curriculu over time. The Cognitive Tutor observes students. As an ob server, it sees every hing the student does, at approxim tely 10-second i tervals, for two day per week over a school ear. However, the cogniti ve odel is not a passive observer. It is continually evaluating the student and predicting what the student knows and does not know. By aggregating these predicti ons across students, we can test whet her the cognitive m odel is correctl m odeling student behavi or. Consider what an observer should

see across ti in a classroo If students are learning, the should be making fewer errors over time. However, the activ ities given to the students over time should also increase in difficulty. In a well-constructed curriculu these two forces should cancel each other out, leading to a f irly constant error rate over ti . In fact, that is what we see in the Cognitive T tor curricula. Figure 2 shows percent correct , over tim e, f r 88 stu ent using the C ogniti ve Tuto r Geo etry curriculum in one school. The percentage co rrect remains fairly constant over t . Percent Correct (all students, all

actions) 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 1 o 100 01 200 01 t 300 1 t 00 01 500 01 t 600 1 t 00 01 800 01 t 900 01 t o 1 000 100 1 t o 11 00 1101 t 120 201 t 1300 130 1 t o 14 00 1401 t 150 501 t 1600 160 1 t o 17 00 1701 t 1800 801 t 1900 1901 t o 20 00 2001 t 2100 21 01 t 2200 2201 t o 23 00 2301 t 2400 24 01 t 2500 Time (action number) Fi gure 2: Pe rce c rre over time c onsi ng l st ude nt ac ns i t Ge ome ry c rric m ACT-R makes the strong cl ai that learn ng takes place at the level of the knowledge com ponents. Thus, if we asure percent correct over tim , consideri ng onl

actions t at invol ve a particular kno wledge co ponent, we should see an increase (Anderson, C onrad & Corbett, 1989) . Fi gure 3 shows percent correct for the same group of students, c onsidering on ly those stude nt actions that the cogniti odel considers to be relevant to a single skill (calculating the area of a regular pol ygon, in an orient ation where a side is horizontal).
Page 7
Percent correct (all students, one skill) 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 1 to 2 to 4 to 6 to 8 to 10 11 to 12 13 to 14 15 to 16 17 to 18 19 to 20 Time (actions involving skill) Fi gure

3: Pe rce c rre over time for si ngl ski If ACT-R is c rrect that performance of a co lex task is determ ined b the i ndi vidual k nowl dge co ponents contribut ing t the perform ance of that task, then each skill in t e cognitive m odel should show a learning curve like this. Failure to see learning on one of the com ponent skills m st mean that the cognitive m odel im plemen ted in the tutor is not correctly representing student knowledge. In the develo pment of our algebra tutor, we di scovered that the m odel was over-predicting stu ent performance i solving equations of the form =b. An

analy is of the data reveale that the over- prediction was, in part, due to the case w ere a=-1. The explanation for this over-prediction, in retrospect, is obvious. In the case where a=-1, the stude nt needs t understand that the expression –x mea s –1 tim es and that, ot herwise, the equation can b solving us g the sam operations as would be appli d to an equation of t e form a =b. (Another way to thi nk abo this error is that some students have learned a rule equivalent to “if the equation is of t e fo rm ax=b, then divi de b t e num ber in front of t e variable.” But, when the coefficient

is –1, the st udent doesn see a num ber, just a ne gative sign, so the rule does not appl ) Once recognition as –1 times was added to the cogni tive m odel, the Cognitive T tor automatically adjusts instruction to test whether st udents have mast ered that skill and will automatically provide extra practice on such problem s t students who need it. In addition, we can define instruction specifically targeted at this skill. The process of analy ing learning curves and im proving ou r fit to the data has, to this point, been laborious. We have recentl y been exploring the possi bility of

automating the proc ess of discovering flaws in the cogni ti ve m odel (Cen, Koedinger & Junker, 2005; Junker, Koedinger and Trottini, 2000) and this is an active focus of research at the Pittsburgh Science of Learning Center ( www.le arnlab.org ). We believe that, in the near future, we will be able to greatly extend our abilit y to understand and accurately model students mathem atical cognition. I addition to i proved statistical odeli ng techniques, the expansion of Carnegie Learning cu stomer base an d the ability to aggregate student data over the inter et provi des us with the abilit to

l ook both m re deepl and m re broadl at student cognition. We have now collected data on over 3000 students us ing the Cognitive Tutor in a pre-algebra c ass. These data co rise over 8.5 m ill ion observations, which am ounts to observing an action for each student about every 9.5 seconds. With a database of this size, we e xpect to be abl to detect more subtle factors affecting lear ning, including the effectiveness of indi vidual tasks, hints and feedback patterns. We are starting to appl m crogenetic thods (Siegler and Cr owley 1991) to see if we can identify key learning experiences,

which could contribute to cognitive m odels that better m odel individual differences in prior
Page 8
knowledge or learning st es and prefer ences. We are optimistic about the potential fo r Cognitive T tors to cont nue to im prove in their abi lit to help students learn mathe atics. ev en , V W.M.M., & Ko ed r, K R. (2002 ). An ffectiv e m taco gn itiv e strateg : Learn b do ing and ex ain wit a co er-based C gn itiv e Tu r. Cogn itiv e Scien ce, 26 (2 ). An de rso J. R ( 98 3) . Th e Arch itectu e o Co gn itio . Cam rid , M : Harv ard Un iv ersity Press. on , J.R. (1 990 Th e Ad ap

tive Cha acter o Tho ught . Hillsd le, NJ: Erlb au An de rso J. R ( 99 3) . Ru les o th e Mind . Hi llsd le, N. J.: Erlb au And rson , J. R. (2 002 ). Span sev n rs of m itu : A ch allen for co gn itiv e m ling gn itive Sci nce , 26 , 85- 112 . Ande rson, J. R ., Bot ell, D ., Byrne, M. D ., Douglass, S., Le er e, C., & Q . ( 200 4) . A in tegr ated th eo of th nd. Psyc hol ogic l Review, 111 , (4 ). 036 -1 060 And rson , J. R., Bo yle, C. F., C rb ett, A., Lewis, M. W. (1 990 ) Cog itiv e m lin g an d in tellig en tu toring . Artificia l In tellig en ce, 42 , 7-49 . And rson , J. R., Bo yle,

C. F., Farrell, R., & Reiser, B. J. (198 7). Cogn itiv e p cip es in th e d sign o co er tu to rs. In P. Mo rris (Ed.), Mod lin g Cogn itio n, W iley. And rson , J. R ., Co nrad , F. G., & Co rb ett, A. T. (198 9). Skill acq sitio an th e LISP Tu tor. Cognitive Science, 13 , 67- 506 . And rson , J. R., Corb ett, A. T., Ko ed r, K. R., & Pel etier, R. (1 995 ). Cogn itiv e tu tors: Lesso s learn The Jou of e Lear ni Sci ces, 4 ) 16 7- 20 7. on , J. R & Leb ère, C. (1 998 Th e a ic compon en ts o tho ugh t. Mah ah , NJ: Erlb au Cen H., Ko edin r, K. R. an Ju nk er, B. (200 5). Learn Fact s

Analysis - A Gen ral Meth od for Co gn itiv e odel Eval uat on an d Im pro em ent In M Ike a, K As hl ey and T. C (E ds) In tellig en t Tu to ri Systems 8 th In tern tiona l Co eren ce (p p. 1 17 5) . rl ri erl g. et A. T. & An der , J. R (1 . Kn e t ng: M del g t e acq ui si on of pr oce al kn owl dge . er Mod ling an d U er- dap ted In tera ction , 253 -27 . Co rb ett, A. T., Ko ed ing r, K. R., & An derson , J. R. (1 997 ). In tellig en t tu toring syste s (Ch er 37). M. G. land er, T. Land au er , & P. Pr ab hu , ( ds.) Ha ndb oo k o Huma n-Comp er In tera ctio n, 2n ed ition . Am sterda

The Nethe lands Elsevier Science. et A ., Laug hl ., Sca pi nna , K C a d Ha dl ey , W. H. (2 . Anal g a d g ne rat math atica ls: An Al bra II co gn itiv e tu tor d si gn stud y. In G. Gau er, C. Frasson and K. Lehn (Eds.) , In telligen t Tu to ri Systems: Fifth In terna tion C eren ce (p p. 31 ). Berl in: Sp rin er erlag. et T. Tras k, H. J., Scar pi nat nd Ha dl ey , W S. (1 . A fo rm at ive al uat n of e Al ge bra II T r: S pp ort fo si le hierarc ical reasoni g. In tellig en t tu to ring systems: Pro ceed ng s th e 4 th in terna tio na l co eren ce, I S ’9 8, 38 3. Gl uc k, K ( . Ey e m

ovem nt s and al geb a t ri Un ubl hed doct ral di ss ert on. C egi M Un ersity. Ju nk er, B. W., Ko ed ing r, K. R. and Tro ttin i, M. (J y 2 000 ). Findin im ov emen ts in stud ent ls fo r in tellig en t tu toring syste s v a v riab le selectio fo r a lin ear lo stic test m l. Presen at th e A nnual N rth Am erican Meeting of the Psychom etri c Soci et Vanc uve B C nada . Ko ed ing r, K. R. & An rson , J. R. (19 93). Effectiv e e in tellig en t so ft ware i gh scho m th classroo . In Pro ceed ng s th e S xth Wo rld Con eren ce on Artificia In telli ce in Ed uca tio n, (p p. 2 41- 248 Ch arlo

ttesv ille, VA: Asso ciatio n fo r th e Ad van cem en t o C tin i Ed cation
Page 9
Koe ge r, K. R & Ande rs on , J. (1 Abst ract pl ni an d perc ept al ch un ks: El em ent of expe rt e i geom et ry . ve Sci nce 14 , 5 11- 550 . Ko ed ing r, K. R., & And rson , J. (199 8). Illustratin g in cip ed d si gn: Th e early evo tion a cog itiv e tu tor fo r al ge a sy bol zat on. Interactive Learning Envir nme ts , 5 161 -1 80. Ko ed ing r, K. R., An rso , J. R., Had ey, W. H., & Mark , M. (19 . In te llig en t tu torin go es to scho in th e g city. In terna tion Jou Artificia l In tellig en

ce i Edu tio n, , 30- 3. Ko ed ing r, K. R., Co rb ett, A. T., Ritter, S., & Sh ap iro L. J. (2 000 ). Carneg ie Learn ng 's Co gn itiv e Tu r: Su mm ar research resu lts. ite Pap r. Pittsb , PA: Carn eg Learn . [h ttp ://www.carn elearn ng .co /we s/CMU_ research _resu lts.pd ] Koe ge r, K. R & Nat an M J. ( 00 4) The real st or behi st ory pr Effect s of rep ese ons titativ e reason ing Jou nal o th e Learn Scien es, 13 (2 ), 29-164 . Leb ere, C (1 99 9). Th e d mics o cog itio An AC T-R l o cogn itiv e arith c. Kogn itio wissen ch aft., ( pp . 5- 19 . Mark , M. A., & Ko ed ing r, K. R. (1

999 ). Stra teg c Sup t Al braic Exp essi -writing Pr oce di s of e Sevent Inter ational onf er ence on User Modeling . http://www.c usa sk. ca/U 99/Pro /mar pdf Mo rg an, P., & Ritter, S. (200 2). An ex ri men al stu t e effects C itiv e Tu tor Alegb a I stud en t nowledg e an d attitu . Pit sb urgh , PA: Carneg ie Learn . ttp ://www.carn elearn ng .co /web s/ rg an _ritter_ 200 2.pdf ] Natha , M J., and Koe inger, K. R. (2 000a). An i nve stigation of teache s' belie fs of students' algebra developm ent. Co gn itio n and In stru ction 18 (2 ), 2 23 7. Natha , M. J., and Koe inge r, K. R.

(2000b). Teachers' an d resea che s' beliefs about e de velopm ent of algebraic reasoning. Journ fo r Research in Ma th ematics Ed tion 31 , 16 8-1 Nat onal R sea ch C unci (2 00 . St rat gi Ed uc at Re searc P rt ne hi . Comm ittee on a Strate gic Education Resear ch Par ner M S. Do nov an, K. gd or and C. E. Snow , ed it s. sion of Behav al an d So cial Scien ces and Ed cation Wash ing on , DC: Th e Natio l Acad em ies Press. Newel A. ( 97 . Yo u ca pl ay q ons wi nat re d wi n: Pro ect co nt s on e pa pers o sym posi In G C ase d. ), Vi su al I rm at n Pr oc essi ng (pp 28 3- 310 Yor : A ad

em ic Pr ess. Newell, A. (1 99 . Un ified Th eories o Cogn itio . Cam rid , MA: Harvard Un iv ersity Press. Ritter, S. and And rson , J. R (19 . Calcu atio n and stra teg in t e equ tio n so lv ing tu to r. In J.D. Mo ore & J.F. Lehm an (Eds .) Procee dings of the Seventee th Annual C onference of the Cogn itive Science Society . (p p. 413 18). Hillsd le, NJ: Erl au Rittle-Johns on, B. and Koedi nge r, K. R. (2002). Com ari ng structiona l strategies for integrating c ncept al and ed ur al know ledg e. Meet of t e N rt h A eri C r of t I rn at nal ou p f r t e P ych ol Ma th ema tics Edu tio

thens GA ). Oc tobe r, 2 . Rittle-Johns on, B. and Koedi nge r, K. R. (2005). Designing better learni ng environm ents: Knowle dge scaffol ing sup rt s m em at cal probl sol ng. Cogn itio an d In st ru ction 23 (3 ), 3 13- 349 . Rittle-Johns on, B. & Siegle r, R.S. (19 98). The relation between c nceptual an d proce ural knowle ge in learning them atics: A revie I C. D nlan (E ), The development of mathe atical skill . p. 75- 10). H ove , UK: Psych og y Press. Rittle-Johns on, B., Siegle r, R.S. & Alib ali, M.W (2001). Devel opi ng concept al understandi ng a d proce ural s ill in m th at ics

An iterativ e pro cess. Jou na o Edu tional Psycho og , 93 , 34 6-3 Sark is, H. (20 . Co gn itiv e Tu tor Alg bra 1 Pro ram Ev al tio Mia Dad Cou y Pub lic Sch s. Lig e Po in t, FL: Th e Reliab ilit y Grou p. [h ttp ://www.c arn elearn ng .co /web s/sark is_ pd Sieg ler, R. S., & Crow ley, K. (199 1) . Th e microg en etic m th od : A rect mean s for studyin cogn itiv e en t. Ameri Psyc hol ogi st , 46 , 60 6-6
Page 10
Sieg ler, R. S., an d Sh ip ley, C (19 . Variatio selec tio and co gn itiv e ch ang . In T. Si & G. Halfo (Ed .), Developing cognitive comp et ence: New approaches to

process modeling (pp 31 -76 . Hillsd le, NJ: Erl au 10