!"!#$%$&'!())*!+,-+!.)/012&+3!4$25-5/,$/-#!6217)5-#!821)59!:468;!!!!!8 - PDF document

1 !"!#$%$&'!())*!+,-+!.)/012&+3!4$25-5/,$/
!"!#$%$&'!())*!+,-+!.)/012&+3!4$25-5/,$/-#!6217)5-#!821)59!:468;!!!!!8-5/,!=!&#x 3 0;?@A!!A!!!!!!!!!F)5!1)52!.2+-$#3!)&!+,2!032!)G!C012&+-͛Ɛ!3)G+H-52!-&.!$&+2##2/+0-#!75)725+9=!$&/#0.$&'!+,2!$.2-3!/)&+-$&2.!$&!+,$3!())*=!322!,++7IJJ&012&+-E/)1J(03$&233K3+5-+2'9K-&.K$7JE! $88%"5$-"*'#6!@0!�*,%1!%"+0!-*!#.$/0!*,/!�*/+!$#!�0!(*6!@0!.*80!-."#!)**+!�"%%!)05*40!-.0!#-$'1$/1!/0=0/0'50!=*/!80*8%0!�.�*!$'-!-*!%0$/'!$)*,-!9:;!5*/-"5$%!-.0*/2!$'1!"-#!$88%"5$-"*'#!=*/!4$5."'0!"'-0%%"(0'50

2 6!A*"'(!=*/�$/1!�0!$'-"5"8
6!A*"'(!=*/�$/1!�0!$'-"5"8$-0!,#"'(!-�*!4$"'!=*/4#!*=!1*5,40'-$-"*'!*=!*,/!/0#0$/5.!*'0!"#!8,)%"#.01!/0#0$/5.!8$80/#!$'1!-.0!*-.0/!"#!$11"-"*'#!-*!-."#!)**+!B"*%*("5$%!$'1!;$5."'0!7'-0%%"(0'506!C,#-!$#!&#x -7 ;0!4$+0!$%%!*=!*,/!/0#0$/5.!$'1!-05.'*%*(2!$&$"%$)%0!"'!*80'!#*,/50!&#x -4 ;0!&#x -4 ;$'-!-*!)0!-/$'#8$/0'-!&#x -4 ;"-.!-."#!4$',#5/"8-!$#!&#x -4 ;0%%!0&0'!&#x -4 ;0%%D$.0$1!*=!"-#!5*48%0-"*'6!@.0'!"-!"#!="'"#.01!-."#!)**+!&#x -4 ;"%%!5*&0/!$%%!-.0!=,'1$40'-$%!5*'508-#!*=!9:;!-.0*/26!E*/!'&#x -

3 4 ;*!&#x -4 ;0!$/0!#.$/"'(!-.0!5.$8-0/#!
4 ;*!&#x -4 ;0!$/0!#.$/"'(!-.0!5.$8-0/#!-.$-!.$&0!)00'!5*48%0-01!0$5.!*=!&#x -4 ;."5.!5*'-$"'#!$!/0&"#"*'!."#-*/2!#*!2*,!5$'!#00!&#x -4 ;.$-!.$#!5.$'(01!"'!0$5.!5.$8-0/6!F&0/!-"40!&#x -4 ;0!&#x -4 ;"%%!$11!5.$8-0/#!-*!BG;76 %)=8#/$7&#x 10 ;%?%'3%)(?%@ABC?%,#*(*-#.)(%)/0%1)."#/'%2/3'((#-'/.'?%%%%;'(')$'%A?D?%E..'$$'0%)3%"336FGG/H5'/3)?.*5G4#*(*-#.)(I)/0I5)."#/'I Revision History The table the need for propulsion. Bird wings and airplane wings work on the same aerodynamic principles, and those principles had to be unders

4 tood before the Wright brothers could bu
tood before the Wright brothers could build an airplane. Indeed, they studied how birds glided and tested wing shapes in wind tunnels to learn the principles of lift. Wing flapping is different; it is a means of propulsion, and the specific method used for propulsion is less important when it comes to building flying machines. In an analogous fashion, we need to understand the principles of intelligence before we can build intelligent machines. Given that the only examples we have of intelligent systems are brains, a

5 nd the principles of intelligence are no
nd the principles of intelligence are not obvious, we must study brains to learn from them. However, like airplanes and birds, we donÕt need to do everything the brain does, nor do we need to implement the principles of intelligence in the same way as the brain. We have a vast array of resources in software and silicon to create intelligent machines in novel and exciting ways. The goal of building intelligent machines is not to replicate human behavior, nor to build a brain, nor to create machines to do what humans do

6 . The goal of building intelligent machi
. The goal of building intelligent machines is to many open issues. This approach to machine intelligence is different than that taken by classic AI and artificial neural networks. AI technologists attempt to build intelligent machines by encoding rules and knowledge in software and human-designed data structures. This AI approach has had many successes solving specific problems but has not offered a generalized approach to machine intelligence and, for the most part, has not addressed the question of how machines ca

7 n learn. Artificial neural networks (ANN
n learn. Artificial neural networks (ANNs) are learning systems built using networks of simple processing elements. In recent years ANNs, often called Òdeep learning networksÓ, have succeeded in solving many classification problems. However, despite the word ÒneuralÓ, most ANNs are based on neuron models and network architectures that are incompatible with real biological tissue. More importantly, ANNs, by deviating from known brain principles, don't provide an obvious path to building truly intelligent machines. Class

8 ic AI and ANNs generally are designed to
ic AI and ANNs generally are designed to solve specific types of problems rather than proposing a general theory of intelligence. In contrast, we know that brains use common principles for vision, hearing, touch, language, and behavior. This remarkable fact was first proposed in 1979 by Vernon Mountcastle. He said there is nothing visual about visual cortex and nothing auditory about auditory cortex. Every region of the neocortex performs the same basic operations. What makes the visual cortex visual is that it rec

9 eives input from the eyes; what makes th
eives input from the eyes; what makes the auditory cortex auditory is that it receives input from the ears. From decades of neuroscience research, we now know this remarkable conjecture understanding some of the core principles of intelligence and how the brain works that the field of machine intelligence can move forward more rapidly than in the past. The field of machine intelligence is poised to make rapid progress. Hierarchical Temporal Memory Hierarchical Temporal Memory, or HTM, is the name we use to describe t

10 he overall theory of how the neocortex f
he overall theory of how the neocortex functions. It also is the name we use to describe the technology used in machines that work on neocortical principles.HTM is therefore a theoretical framework for both biological and machine intelligence. The termHTM incorporates three prominent features of the neocortex. First, it is best to think of the neocortex as a "memory" system. The to predict the changing input stream, and it learns to play back sequences of motor commands. And finally, the regions of the neocortex are

11 connected in a logical "hierarchy". ,
connected in a logical "hierarchy". , and not just biological brains, then we would modify HTM theory to include them. There are many biological details that similarly are not part of HTM theory. Every feature included in HTM is there because we have an information-theoretical need that is met by that feature. HTM also is not a theory of an entire brain; it only covers the neocortex and its interactions with some closely related structures such as the thalamus and hippocampus. The neocortex is where most of what we

12 think of as intelligence resides but it
think of as intelligence resides but it is not in charge of emotions, homeostasis, and basic behaviors. Other, evolutionarily older, parts of the brain perform these functions. These older parts of the brain have been under evolutionary pressure for much longer time, and although they consist of neurons, they are heterogeneous in architecture and function. We are not interested in emulating entire brains or in making machines that are human intelligent machines will be of limited utility. Many will be simple, tireles

13 sly sifting through vast amounts of data
sly sifting through vast amounts of data looking for unusual patterns. Others will be fantastically fast and smart, able to explore domains that humans are not well suited for. The variety we will see in intelligent machines will be similar to the variety we see in programmable computers. Some computers are tiny and embedded in cars and appliances, and other like performance is limiting. The second reason we reject behavior-based definitions of intelligence is that they donÕt capture the incredible will change as r

14 esearch progressesSecond, the book is in
esearch progressesSecond, the book is intended for a technical but diverse audience. Neuroscientists should find the book helpful as it provides a theoretical framework to interpret many biological details and guide experiments. Computer scientists can use the material in the book to develop machine intelligence hardware, software, and applications based on neuroscience principles. Anyone with a deep interest in how brains work or machine intelligence will hopefully find the book to be the best source for these topics.

15 Finally, we hope that academics +*,#%
Finally, we hope that academics +*,#%3,#+$+&H$E!5$%"#*.H$3+$'3*(*03/&((H$/*)+%.&3)#8Kµ$&+$*==*+#8$%*$´'3*(*03/&((H$3)+=3.#8Kµ$1"3/"$3+$&$%#.,$*7%#)$6+#8$3)$,&/"3)#$(#&.)3)04$!'3*(*03/&($8#%&3(+$*7$%"#$)#*/*.%#N$,6+%$'#$/*,=&%3'(#$13%"$%"#$E!5$%"#*.HK$DQGWKHWKHRU\FDQ·WUHO\RQSULQFLSOHVWKDWFDQ·WSRVVLEO\EHLPSOHPHQWHGLQELRORJLFDOWLVVXH4$M*.$#N&,=(#/*)+38#.$%"#$=H.&,38&($)#6.*)K$%"#$,*+%$/*,,*)$%H=#$*7$)#6.*

16 )$3)$%"#$)#*/*.%#N4$H.&,38&($)#6.*)+$"&2
)$3)$%"#$)#*/*.%#N4$H.&,38&($)#6.*)+$"&2#$%.##U(3F#$#N%#)+3*)+$/&((#8$8#)8.3%#+$%"&%$/*))#/%$23&$%"*6+&)8+$*7$+H)&=+#+4$ $*7$,#,*.H4$!"#.#7*.#K$%*$6)8#.+%&)8$"*1$%"#$)#*/*.%#N$1*.F+$1#$)##8$&$%"#*.H$%"&%$&//*,,*8&%#+$%"#$#++#)%3&($7#&%6.#+$*7$)#6.*)+$&)8$+H)&=+#+4$�.%373/3&($T#6.&($T#%1*.F+$?�TT+@ +$&.#$3)%#((30#)%4$O6.$/"&((#)0#$3+$%*$+#=&.&%#$&+=#/%+$*7$'.&3)+$&)8$)#6.*)+$%"&%$&.#$#++#)%3&($7*.$3)%#((30#)/#$7.*,$%"*+#$DVSHFWVWKDWDUHDUWLIDFWVRIWKHEUDLQ·V&#x

17 0003;SDUWLFXODULPSOHPHQWDWLRQ&#x
0003;SDUWLFXODULPSOHPHQWDWLRQRILQWHOOLJHQFHSULQFLSOHVO 7.*,$3%+$'3*(*03/&($*.303)%XWZHDUHQ·WWKHUH\HW7RGD\ZHVWLOOKDYHPXFKWROHDUQIURPELRORJLFDOEUDLQV&)8$%"#.#7*.#$%*$6)8#.+%&)8$E!5$=.3)/3=(#+K$%*$&82&)/#$E!5$%"#*.HK$&)8$%*$'63(8$3)%#((30#)%$,&/"3)#+K$3%$3+$)#/#++&.H$%*$F)*1$)#6.*+/3#)/#$%#.,+$&)8$%"#$'&+3/+$*7$WKHEUDLQ· 8#+

18 /.3=%3*)+$&)8$%#/")3/&($8#%&3(+$*7$%"#$E
/.3=%3*)+$&)8$%#/")3/&($8#%&3(+$*7$%"#$E!5$=.3)/3=(#+$&.#$=.*238#8$3)$+6'+#Y6#)%$/"&=%#.+4$9+-2-:+1#2(7;,&'*#%+-.6(3"&(%'=1%='&(-&#x 3 0;(%"&(?&-1-'%&@(!"#$"6,&)$'.&3)$/*,=.3+#+$+#2#.&($/*,=*)#)%+$+6/"$&+$%"#$'.&3)$+%#,K$%"#$'&+&($0&)0(3&K$&)8$%"#$/#.#'#((6,4$!"#+#$*.0&)+$&.#$(**+#(H$+%&/F#8$*)$%*=$*7$%"#$+=3)&($/*.84$!"#$)#*/*.%#N$3+$-6+%$*)#$,*.#$/*,=*)#)%$*7$%"#$'.&3)K$'6%$3%$8*,3)&%#+$%"#$"6,&)$'.&3)K$*//6=H3)0$&'*6%$SWX$*7$3%+ !"#$,*8#.)$"6,&)$)#*/*.%#N$+#=&.&%#8$7.*,$%"*+#$*7$,*)F#H+$3)$%#.,+$*7$+3P#$&)8$/*,=(#N3%

19 H$&'*6%$IW$,3((3*)$H#&.+$&0*$?`&F3/K$IAA
H$&'*6%$IW$,3((3*)$H#&.+$&0*$?`&F3/K$IAAR@4$!"#$"6,&)$)#*/*.%#N$/*)%3)6#8$%*$#2*(2#$%*$'#$'300#.$&)8$'300#.K$.#&/"3)0$3%+$/6..#)%$+3P#$3)$"6,&)+$'#%1##)$aAAKAAA$&)8$IAAKAAA$H#&.+$&0*G4$C)$ -6+%$6)8#.$%"#$+F6(($&)8$1.&=+$&.*6)8$%"#$*%"#.$=&.%+$*7$%"#$'.&3)4$?M.*,$"#.#$*)K$%"#$ZRUG´QHRFRUWH[µZLOOUHIHUWRWKHKXPDQQHRFRUWH[5HIHUHQFHVWRWKHQHRFRUWH[RIRWKHUPDPPDOVZLOOEHH[SOLFLW

20 !"#$)#*/*.%#N$3+$"#&23(H
!"#$)#*/*.%#N$3+$"#&23(H$7*(8#8$%*$73%$3)$%"#$+FXOOEXWWKLVLVQ·WLPSRUWDQWWRKRZLWZRUNVK$+*$1#$13(($&(1&H+$.#7#.$%*$3%$&)8$3((6+%.&%#$3%$&+$&$7(&%$+"##%4$!"#$"6,&)$)#*/*.%#N$3+$(&.0#$'*%"$3)$&'+*(6%#$%#.,+$&)8$&(+*$.#(&%32#$%*$%"#$+3P#$*7$*6.$'*8H$/*,=&.#8$%*$*%"#.$,&,,&(+4$Z#$&.#$&)$3)%#((30#)%$+=#/3#+$,*+%(H$'#/&6+#$*7$%"#$+3P#$*7$*6.$)#*/*.%#N4$!"#$,*+%$.#,&.F&'(#$&+=#/%$*7$%"#$)#*/*.%#N$3+$3%+$"*,*0#)#3%H4$!"#$%H=#+$*7$/#((+$&)8$%"#3.$=&%%

21 #.)+$*7$/*))#/%323%H$ 7.*,$%"#$#H#+$&)8$
#.)+$*7$/*))#/%323%H$ 7.*,$%"#$#H#+$&)8$)*%$'#/&6+#$%"#H$"&2#$+=#/3&($23+3*)$)#6.*)+$*.$23+3*)$&(0*.3%",+$?5*6)%/&+%(#K$GRSa@4$!"3+$83+/*2#.H$3+$3)/.#83'(H$3,=*.%&)%$&)8$3+$+6==*.%#8$'H$,6(%3=(#$(3)#+$*7$#238#)/#4$^2#)$%"*60"$%"#$)#*/*.%#N$3+$(&.0#(H$"*,*0#)*6+K$+*,#$)#6.*+/3#)%3+%+$&.#Y63/F$%*$=*3)%$*6%$%"#$8377#.#)/#+$'#%1##)$)#*/*.%3/&($.#03*)+4$O)#$.#03*)$,&H$"&2#$,*.#$*7$&$/#.%&3)$/#(($%H=#K$&)*%"#.$.#03*)$"&+$#N%.&$(&H#.+K$&)8$*%"#.$.#03*)+$,&H$#N"3'3%$2&.3&%3*)+$3)$/*))#/%323%H$=&%%#.)+4$[6%$%"#.#$3+$)*$Y6#+%3*)$

22 %"&%$)#*/*.%3/&($.#03*)+$&.#$.#,&.F&'(H$
%"&%$)#*/*.%3/&($.#03*)+$&.#$.#,&.F&'(H$+3,3(&.$&)8$%"&%$%"#$2&.3&%3*)+$&.#$.#(&%32#(H$,3)*.4$!"#$8#'&%#$3+$*)(H$&'*6%$"*1$/.3%3/&($%"#$2&.3&%3*)+$&.#$3)$%#.,+$*7$76)/%3*)&(3%H4$!"#$)#*/*.%3/&($+"##%$3+$83238#8$3)%*$8*P#)+$*7$.#03*)+$+3%6&%#8$)#N%$%*$#&/"$*%"#.4$\**F3)0$&%$&$)#*/*.%#N$H*6$1*6(8$)*%$+##$&)H$.#03*)+$*.$8#,&./&%3*)+4$!"#$.#03*)+$&.#$8#73)#8$'H$/*))#/%323%H4$`#03*)+$=&++$3)7*.,&%3*)$%*$#&/"$*%"#.$'H$+#)83)0$'6)8(#+$*7$)#.2#$73'#.+$3)%*$%"#$1"3%#$,&%%#.$-6+%$'#(*1$%"#$)#*/*.%#N4$!"#$)#.2#$73'#.+$.##)%#.$&%$&

23 )*%"#.$)#*/*.%3/&($.#03*)4$!"#$/*))#/%3*
)*%"#.$)#*/*.%3/&($.#03*)4$!"#$/*))#/%3*)+$'#%1##)$.#03*)+$8#73)#$&$(*03/&($"3#.&./"H4$C)7*.,&%3*)$7.*,$&$+#)+*.H$*.0&)$3+$=.*/#++#8$'H$*)#$.#03*)1"3/"$=&++#+$3%+$*6%=6%$%*$&)*%"#.$.#03*)K$1"3/"$=&++#+$3%+$*6%=6% "*,*0#)*6+$8#+30)K$&)$&(0*.3%",$%"&%$1*.F+$13%"$&)H$%H=#$*7$+#)+*.$8&%&$&)8$&)H$%H=#$*7$'#"&23*.4$!"3+$.#=(3/&%3*)3+$%"#$'#&6%H$*7$%"#$)#*/*.%#N4$O)/#$%"#$6)32#.+&($)#*/*.%3/&($&(0*.3%",+$1#.#$#+%&'(3+"#8K$#2*(6%3*)$/*6(8$#N%#)8$%"#$'.&3)·+$"3#.&./"H$.&=38(H$'#/&6+#$3%$*)(H$"&8$%*$.#=(3/&%#$&)$#N3+%3)0$+%.6/%6

24 .#4$!"3+$#N=(&3)+"*1$"6,&)$'.&3)+$#2*(2#
.#4$!"3+$#N=(&3)+"*1$"6,&)$'.&3)+$#2*(2#8$%*$'#/*,#$(&.0#$+*$Y63/F(H4 K$1#$73.+%$%.H$%*$6)8#.+%&)8$%"#$"*,*0#)*6+$.#03*)+$%"&%$&.#$)#&.$%"#$'*%%*,$*7$%"#$)#*/*.%3/&($"3#.&./"H4$C)$*%"#.$1*.8+K$ .*'*%3/+$&+$+#=&.&%#$=.*'(#,+K$'6%$%*$6)8#.+%&)8$"*1$%"#+#$/&=&'3(3%3#+$&.#$76)8&,#)%&((H$&(($%"#$+&,#K$&)8$1"&%$+#%$*7$&(0*.3%",+$/&)$+##$�T:$"#&.$�T:$0#)#.&%#$'#"&23*.4$C)3%3&((HK$%"3+$0#)#.&($&==.*&/"$,&F#+$*6.$%&+F$+##,$"&.8#.K$'6%$6(%3,&%#(H$3%$3+$(3'#.&%3)04$Z"#)$1#$8#+/.3'#$*.$+%68H$&$=&.%3/6(&.$E!5$(#&.)3)0$

25 &(0*.3%",$1#$*7%#)$13(($+%&.%$13%" 6)8#.
&(0*.3%",$1#$*7%#)$13(($+%&.%$13%" 6)8#.+%&)8$%"#$/*,,*)$&(0*.3%",+$*7$%"#$)#*/*.%#NK$1#$/&)$&+F$"*1$#2*(6%3*)$,30"%$"&2#$%1#&F#8$%"#+#$&(0*.3%",+$%*$&/"3#2#$#2#)$'#%%#.$=#.7*.,&)/#$*)$&$=&.%3/6(&.$=.*'(#,4$[6%$*6.$7*/6+$3+$% U(#2#($+#)+*.H$&..&H+$+6/"$&+$%"#$.#%3)&$*.$/*/"(#&4$�+$3%+$)&,#$3,=(3#+K$E!5$%"#*.H$3)/*.=*.&%#+$%"#$/*)/#=%$*7$"3#.&./"H4$[#/&6+#$#&/"$.#03*)$3+$=#.7*.,3)0$%"#$+&,#$+#%$*7$,#,*.H$&)8$&(0*.3%",3/$76)/%3*)+K$%"#$/&=&'3(3%3#+$#N"3'3%#8$'H$%"#$#)%3.#$)#*/*.%3/&($"3#.&./"H$"&2#$%*$'#$=.#+#)%$3)

26 $#&/"$.#03*)4$!"6+$ +=&.+#$83+%.3'6%#8$.
$#&/"$.#03*)4$!"6+$ +=&.+#$83+%.3'6%#8$.#=.#+#)%&%3*)+4$!*$'#/*,#$7&/3(#$13%"$E!5$%"#*.HK$H*6$13(($)##8$%*$8#2#(*=$&)$3)%63%32#$+#)+#$7*.$%"#$,&%"#,&%3/&($&)8$.#=.#+#)%&%3*)&($=.*=#.%3#+$*7$Q:`+4$9+-2-:+1#2(7;,&'*#%+-.6(3"&(E.$=%,(#.C(7=%$=%,(-�(%"&(?&-1-'%&@(�+$ &'*6%$%*$,*2#K$&)8$"*1K$%"#)$&$/"&)0#$*7$=&%%#.)$*)$%"#$*=%3/$)#.2#$1*6(8$'#$=#./#32#8$&+$%"#$1*.(8$,*23)04$!"#$7&/%$%"&%$*6.$=#./#=%3*)$3+$+%&'(#$1"3(#$%"#$#H#+$,*2#$%#((+$6+$%"#$)#*/*.%#N$3+$.#(H3)0$*)$F)*1(#80#$*7$#H#$,*2#,#)%+4$Z"#)$H*6$%*6/"K

27 $"#&.K$*.$+##$+*,#%"3)0K$%"#$)#*/*.%#N$)
$"#&.K$*.$+##$+*,#%"3)0K$%"#$)#*/*.%#N$)##8+$%*$83+%3)063+"$/"&)0#+$3)$+#)+&%3*)$/&6+#8$'H$H*6.$*1)$,*2#,#)%$7.*,$/"&)0#+$/&6+#8$ '.&3)$.#03*)+$%"&%$&/%6&((H$0#)#.&%#$'#" !*$/.#&%#$&)$3)%#((30#)%$+H+%#,K$%"#$E!5$(#&.)3)0$&(0*.3%",+$)##8$'*%"$+#)+*.H$#)/*8#.+$&)8$+*,#$%H=#$*7$'#"&23*.&($7.&,#1*.F4$d*6$,30"%$+&H$%"&%$%"#$E!5$(#&.)3)0$&(0*.3%",+$)##8$&$'*8H4$[6%$%"#$'#"&23*.+$*7$%"#$+H+%#,$8*$)*%$)##8$%*$'#$&)H%"3)0$(3F#$%"#$'#"&23*.+$*7$&$"6,&)$*.$.*'*%4$$M6)8&,#)%&((H'#"&23*.$3+$&$,#&)+$*7$,*23)0$&$+#)+*.%*$+&,=(#$&$8377

28 #.#)%$=&.%$*7$%"#$1*.(84$M*.$#N&,=(#K$%"
#.#)%$=&.%$*7$%"#$1*.(84$M*.$#N&,=(#K$%"#$'#"&23*.$*7$&)$E!5$+H+%#,$/*6(8$'#$%.&2#.+3)0$(3)F+$*)$%"#$1*.(8U138#$1#'$*.$#N=(*.3)0$73(#+$*)$&$+#.2#.4$C%$3+$=*++3'(#$%*$/.#&%#$E!5$+H+%#,+$13%"*6%$'#"&23*.4$C7$%"#$+#)+*.H$8&%&$)&%6.&((H$/"&)0#+$*2#.$%3,# 3+$*7%#)$UHIHUUHGWRDV´RQUOLQHOHDUQLQJµ:LWKHDFKFKDQJHLQWKHLQSXWVWKH,#,*.H$*7$%"#$E!5$+H+%#,$3+$6=8&%#84$!"#.#$&.#$)*$'&%/"$(#&.)3)0$8&%&$+#%+$&)8$)*$'&%/"$%#+%3)0$+#%+$&

29 +$3+$%"#$)*.,$7*.$,*+%$,&/"3)#$(#&.)3)0$
+$3+$%"#$)*.,$7*.$,*+%$,&/"3)#$(#&.)3)0$&(0*.3%",+4$6RPHWLPHVSHRSOHDVN´,IWKHUHDUHQRODEHOHGWU&3)3)0$&)8$%#+%$+#%+K$"*1$8*#+$%"#$+H+%#,$F)*1$37$3%$3+$1*.F3)0$/*..#/%(H$&)8$"*1$/&)$3%$/*..#/%$3%+$'#"&23*."µ+70EXLOGVDSUHGLFWLYHPRGHORIWKH1*.(8K$1"3/"$,#&)+$%"&%$&%$#2#.H$=*3)%$3)$%3,#$%"#$E!5U'&+#8$+H+%#,$3+$=.#83/%3)0$1"&%$3%$#N=#/%+$13(($"&==#)$)#N%4$ $E#./6(&)*UE*6P#(K$Q4$?IAGI@4$!"#$.#,

30 &.F&'(#K$H#%$)*%$#N%.&*.83)&.HK$"6,&)$'.
&.F&'(#K$H#%$)*%$#N%.&*.83)&.HK$"6,&)$'.&3)$&+$&$+/&(#8U6=$=.3,&%#$'.&3)$&)8$3%+$&++*/3&%#8$/*+%4$.*/##83)0+$*7$%"#$T&%3*)&($&#x 4 0;/&8#,H$*7$Q/3#)/#+$*7$%"#$g)3%#8$$ 1$#',&(2+,%'+34%&5()&$'&,&.%#%+-.7(T)#$*7$%"#$,*+%$3)%#.#+%3)0$/"&((#)0#+$3)$�C$3+$%"#$=.*'(#,$*7$I)*1(#80#$.#=.#+#)%&%3*)4$M#=.#+#)%3)0$#2#.O8&O$7&/%+$&)8$.#(&%3*)+"3=+$3)$&$7*.,$%"&%$/*,=6%#.+$/&)$1*.I$13%"$"&+$=.*2#)$%*$'#$83773/6(%$13%"$%.&83%3*)&($/*,=6%#.$+/3#)/# &/%32#$'3%+$3)$%"#$+&,#$(*/&%3*)+E$%"#O$+"&.#$%"#$+#,&)%3/$&%%.3'6%#+$.#=.#+#)%#

31 8$'O$%"*+#$'3%+4$WO$ '&+#8$+O+%#,E$I)*1(
8$'O$%"*+#$'3%+4$WO$ '&+#8$+O+%#,E$I)*1(#80#$3+$3)"#.#)%$3)$%"#$8&%&E$)*%$3)$%"#$&(0*.3%",+4$!*$'#%%#.$6)8#.+%&)8$%"#$=.*=#.%3#+$*7$F:M+$3%$/&)$'#$"#(=76($%*$%"3)I$&'*6%$"*1$3)7*.,&%3*)$3+$%O=3/&((O$.#=.#+#)%#8$3)$/*,=6%#.+$&)8$%"#$.#(&%32#$,#.3%+$*7$F:M+$2#.+6+$%"#$.#=.#+#)%&%3*)$+/"#,#$6+#8$'O$/*,=6%#.+4$C)$/*,=6%#.+E$1#$.#=.#+#)%$3)7*.,&%3*)$13%"$'O%#+$&)8$1*.8+4$[*.$#V&,=(#E$%*$.#=.#+#)%$3)7*.,&%3*)$&'*6%$&$,#83/&($=&%3#)%E$&$/*,=6%#.$SURJUDPPLJKWXVHRQHE\WHWRVWRUH�

32 03;WKHSDWLHQW·VDJH�
03;WKHSDWLHQW·VDJHDQGDQRWKHUE\WHWRVWRUHWKHSDWLHQW·VJHQGHU:&%&$+%.6/%6.#+$+6/"$&+$(3+%+$&)8$%.##+$&.#$6+#8$%*$*.0&)3Y#$=3#/#+$*7$3)7*.,&%3*)$%"&%$&.#$.#(&%#84$!"3+$%O=#$*7$.#=.#+#)%&%3*)$1*.I+$1#(($1"#)$%"#$3)7*.,&%3*)$1#$)##8$%*$.#=.#+#)%$3+$1#(($8#73)#8$&)8$(3,3%#8$3)$#V%#)%4$H*1#2#.E$C$.#+#&./"#.+$83+/*2#.#8$%"&%$%*$,&I#$&$/*,=6%#.$3)%#((30#)%E$3%$)##8+$%*$I)*1$&$"60#$&,*6)%$*7$I)*1(#80#E$%"#$+%.6/%6.#$*7$1"3/"$3+

33 $)*%$1#(($8#73)#84$[*.$#V&,=(#E$1"&%$37$
$)*%$1#(($8#73)#84$[*.$#V&,=(#E$1"&%$37$1#$1&)%$*6.$3)%#((30#)%$/*,=6%#.$%*$I)*1$&'*6%$/&.+\$!"3)I$*7$&(($%"#$%"3)0+$O*6$I)*1$&'*6%$/&.+4$]*6$I)*1$1"&%$%"#O$8*E$"*1$%*$8.32#$%"#,E$"*1$%*$0#%$3)$&)8$*6%$*7$%"#,E$"*1$%*$/(#&)$%"#,E$1&O+$%"#O$/&)$7&3(E$1"&%$%"#$8377#.#)%$/*)%.*(+$8*E$1"&%$3+$7*6)8$6)8#.$%"#$"**8E$"*1$%*$/"&)0#$%3.#+E$#%/4$S#$I)*1$%"#$+"&=#+$*7$/&.+$&)8$%"#$+*6)8+$%"#O$,&I#4$C7$O*6$-6+%$%"3)I$&'*6%$%3.#+E$O*6$,30"%$.#/&(($8377#.#)%$%O=#+$*7$%3.#+E$8377#.#)%$'.&)8+$*7$%3.#+E$"*1$%"#O$1#&.$6)#2#)(OE$%"#$'#+%$

34 1&O$%*$.*%&%#$%"#,E$#%/4$!"#$(3+%$*7$&((
1&O$%*$.*%&%#$%"#,E$#%/4$!"#$(3+%$*7$&(($%"#$%"3)0+$O*6$I)*1$&'*6%$/&.+$0*#+$*)$&)8$*)4$X&/"$=3#/#$*7$I)*1(#80#$(#&8+$%*$*%"#.$=3#/#+$*7$I)*1(#80#$3)$&)$#2#. "*1)$3)$[306.#$KE$+ $&)$F:M$3)$,#,*.OE$1#$GRQ·WKDYHWR+%*.#$&(($%"#$'3%+$&+$O*6$1*6(8$13%"$&$8#)+#$.#=.#+#)%&%3*)4$S#$*)(O$"&2#$%*$+%*.#$%"#$(*/&%3*)+$*7$%"#$JU'3%+E$&)8$+6.=.3+3)0(OE$1#$*)(O$"&2#$%*$+%*.#$%"#$(*/&%3*)+$*7$&$+,&(($)6,'#.$*7$%"#$JU'3%+4$F&O$1#$"&2#$&)$F:M$13%"$JAEAAA$'3%+E 4$!"#$7*((*13)0$+#/%3*)+$8#+/.3'#$%"#+#$=.*=#.%3#+$&)

35 8$*=#.&%3*)+$*7$F:M+$3)$,*.#$8#%&3(4$&#x
8$*=#.&%3*)+$*7$F:M+$3)$,*.#$8#%&3(4$�%$%"#$#)8$*7$%"#$/"&=%#.$1#$83+/6++$+*,#$*7$%"#$1&O+$F:M+$&.#$6+#8$3)$%"#$'.&3)$&)8$3)$H!5+4$!8#%"&9#%+:#;(-$&'%+&,(-=(12),(C)$%"3+$+#/%3*)E$1#$83+/6++$%"#$7*((*13)0$,&%"#,&%3/&($=.*=#.%3#+$*7$F:M+$13%"$&$7*/6+$*)$8#.323)0$76)8&,#)%&($+/&(3)0$(&1+$&)8$#..*.$'*6)8+_$$x ;&=&/3%O$*7$F:M+$&)8$%"#$=.*'&'3(3%O$*7$,3+,&%/"#+$x M*'6+%)#++$*7$F:M+$&)8$%"#$=.*'&'3(3%O$*7$#..*.$13%"$)*3+#$x M#(3&'(#$/(&++373/&%3*)$*7$&$(3+%$*7$F:M$2#/%*.+$x d)3*)+$*7$F:M+$x M*'6+%)#++$*7$6)3*)+$3)$%"#$=

36 .#+#)/#$*7$)*3+#$$$!"#+#$=.*=#.%3#+$&)8$
.#+#)/#$*7$)*3+#$$$!"#+#$=.*=#.%3#+$&)8$%"#3.$&++*/3&%#8$*=#.&%3*)+$8#,*)+%.&%#$%"#$6+#76()#++$*7$F:M+$&+$&$,#,*.O$+=&/#E$1"3/"$1#$3((6+%.&%#$3)$#V&,=(#+$.#(#2&)%$%*$H!5+4$C)$*6.$&)&(O+3+$1#$(#&)$*)$%"#$3)%63%3*)+$=.*238#8$'O$e&)#.2&$?e&)#.2&E$Jf^^ &)8$Ü¡$"&2#$%"#$+&,#$/&.83)&(3%O$ݓE$1#$/&)$8#%#.,3)#$&)$#V&/%$,&%/"$'O$+#%%3)0$ߠൌݓ4$C)$%"3+$/&+#E$37$ß $3+$(#++$%"&)$,E$%"#$*2#.(&=$+/*.#$13(($3)83/&%#$&)$+.&D#:%(9#%:"4$;*)+38#.$&)$#V&,=(#$*7$%1*$F:M$2#/%*.+_$ܠൌሾͲͳͲͲͲͲͲͲͲͲͲͲͲͲͲͲͲͲͲͳͲͲͲͲͲͲÍ

37 ²Í²Í²Í²Í²Í³Í³Í²Í²Í²Í²Í²Í²Í²áˆ¿$ܡൌሾÍ
²Í²Í²Í²Í²Í³Í³Í²Í²Í²Í²Í²Í²Í²áˆ¿$ܡൌሾͳͲͲͲͲͲͲͲͲͲͲͲͲͲͲͲͲͲͲͳͲͲͲͲͲͲͲͲͲͲͲͳͳͲͲͲͲͲͲͲሿ$W*%"$2#/%*.+$"&2#$+3Y#$݊ൌͶͲE$ݏൌͲǤͳE$&)8$ݓൌͶ4$!"#$*2#.(&=$'#%1##)$ݔ$&)8$ݕ$3+$PZ$34#4$%"#.#$&.#$%".##$T`$'3%+$3)$/*,,*)$=*+3%3*)+$*7$'*%"$2#/%*.+4$!"6+$%"#$%1*$2#/%*.+$,&%/"$1"#)$%"#$%".#+"*(8$3+$+#%$&%$Ʌൌ͵E$'6%$%"#O$&.#$)*%$&)$H[DFWPDWFK1RWHWKDWDWKUHVKROGODUJHUWKDQHLWKHUYHFWRU·VFDUGLQDOLW\&#x

38 0003;²$34#4E$ߠ൐ݓ$²$3,=(3#+$&$,&%/"
0003;²$34#4E$ߠ൐ݓ$²$3,=(3#+$&$,&%/"$3+$)*%$=*++3'(#4$$$!#$#:+%0(-=(12),(#.5(%"&(-3#3+;+%0(-=(8+,9#%:"&,(!*$'#$6+#76($3)$=.&/%3/#E$F:M+$+"*6(8$"&2#$&$(&.0#$/&=&/3%O4$h32#)$&$2#/%*.$13%"$73V#8$+3Y#$ %*$"#(=$&)&(OY#$%"#$#77#/%+$*7$,&%/"3)0$6)8#.$2&.O3)0$/*)83%3*)+4$D#%$Ü $'#$&)$F:M$#)/*83)0$*7$+3Y#$ h32#)$&)$F:M$#)/*83)0$Ü $&)8$&)*%"#.$.&)8*,$F:M$Ü¡E$'*%"$13%"$+3Y#$-$&)8$/&.83)&(3%O$ݓE$1"&%$3+$%"#$=.*'&'3(3%O$*7$&$7&(+#$,&%/"E$34#4$%"#$/"&)/#$%"#$݋ݒ݁ݎ݈ܽ݌ �)$3)%#.#+%3)0$=.*=#.%O$*7$F:M+$3+$%"#$&'3(3%O$%*

39 $.#(3&'(O$/*,=&.#$&0&3)+%$&$+6'+&,=(#8$2
$.#(3&'(O$/*,=&.#$&0&3)+%$&$+6'+&,=(#8$2#.+3*)$*7$&$2#/%*.4$!"&%$3+E$.#/*0)3Y3)0$&$(&.0#$83+%.3'6%#8$=&%%#.)$'O$,&%/"3)0$&$+,&(($+6'+#%$*7$%"#$&/%32#$'3%+$3)$%"#$(&.0#$=&%%#.)4$D#%$Ü $'#$&)$F:M$&)8$(#%$Ü Ô¢$'#$&$+6'+&,=(#8$2#.+3*)$*7$Ü E$+6/"$%"&%$ݓ௫ᇱ൑ݓ௫,W·VVH(7U#238#)%$%"#$+6'+&,=(#8$2#/%*.$࢞Ԣ$13(($&(1&O+$,&%/"$Ü E$=.*238#8$ߠ൑ݓ௫ᇱ$%"&%$3+4$H*1#2#.E$&+$O*6$3)/.#&+#$%"#$+6'+&,=(3)0$%"#$/"&)/#$*7$&$7&(+#$=*+3%32#$3)/.#&+#+4$$S"&%$3+$%"#$=.*'&'3(3%O$*7$&$7&(+#$,&%/"$'#%1##)�

40 03;Ü Ô¢$&)8 +3,3(&.$F:M+4$S#$/*)+38#.$&$
03;ܠԢ$&)8 +3,3(&.$F:M+4$S#$/*)+38#.$&$7*.,$*7$/(&++373/&%3*)$+3,3(&.$%*$)#&.#+%$)#30"'*.$/(&++373/&%3*)4$D#%$ܺ$'#$&$+#%$*7$ܯ$2#/%*.+E$ܺൌሼܠଵǡǥǡܠெሽE$1"#.#$#&/"$2#/%*.$ *7$7&(+#$=*+3%32#+$3+$&'*6%$*)#$3)$KK4$C)/.#&+3)0$ݓ$%*$JK$&)8$ߠ$%*$#30"%E$,&3)%&3)3)0$%"#$.&%3*$ఏ௪ൌଶଷE$%"#$/"&)/#$*7$&$7&(+#$=*+3%32#$8.*=+$%*$&'*6%$*)#$3)$KPLP4$`*1$3)/.#&+#$%"#$=&.&,#%#.+$%*$,*.#$.#&(3+%3/$2&(6#+_$݊ൌͳͲʹͶE$ݓൌʹͳE$&)8$ߠൌͳͶ$?34#4$%1* ͲͲͲͲͲͲͲͲͲͲͲͲͲͲͳͳͲͲͲͲ 4$S#$+##$%"#$#..*.$3

41 )/.#&+#+$,*)*%*)3/&((O$13%"$%"#$)6,'#.$*
)/.#&+#+$,*)*%*)3/&((O$13%"$%"#$)6,'#.$*7$=&%%#.)+$+%*.#84$5*.#$3,=*.%&)%(O$%"#$=(*%$+"*1+$%"&%$%"#$+3Y#$*7$%"#$F:M$3+$&$/.3%3/&($7&/%*._$&$+,&(($)6,'#.$*7$'3%+$?JAAA@$(#&8+$%*$.#(&%32#(O$"30"$#..*.$.&%#+E$1"3(#$(&.0#.$2#/%*.+$?JAEAAAj@$&.#$,6/"$,*.#$.*'6+%4$d+#$%"3+$=(*%$3)%#.&/%32#(O$*)(3)#_!6;='&#x 11 ;5;?'&!@5&!6;54/;A!;#-B &)8$3)2*(2#$7(*&%3)0$=*3)%$/&(/6(&%3*)+ I#$%3,#$#)/*8#+$3) %"&%$ "&.81&.#$3,=(#,#)%&%3*)+$,6/"$+3,=(#.4$H!5$+O+%#,+$.#R63.#$&(,*+%$)*$7(*&%3)0$=*3)%$*=#.&%3*)+E$1"3/"$&.#$)##8#8$3)$&)O$+O+%#,$13%

42 "$.&%#$#)/*83)04$H&.81&.#$7*.$3,=(#,#)%3
"$.&%#$#)/*83)04$H&.81&.#$7*.$3,=(#,#)%3)0$H!5$13(($'#$7&.$+3,=(#.$13%"*6%$%"#$)##8$7*.$7(*&%3)0$=*3)%$,&%"4$!"#.#$3+$&)$&)&(*0O$%*$=.*0.&,,&'(#$/*,=6%#.+4$S"#)$=#*=(#$73.+%$+%&.%#8$'63(83)0$=.*0.&,,&'(#$/*,=6%#.+E$+*,#$8#+30)#.+$&82*/&%#8$7*.$8#/3,&($(*03/4$W3)&.O$(*03/$1*)$*6%$'#/&6+#$3%$3+$,6/"$+3,=(#.$%*$'63(84$�(%"*XJK+70QHXURQVGRQ·WKDYHYDULDEOHUDWHRXWSXWVE$WKH\GRLQFRUSRUDWHWZRQHZVWDWHVWKDWGRQ·W&

43 #x0003;H[LVWLQRWKHU�
#x0003;H[LVWLQRWKHU%"#*.3#+4$S"#)$&$)#6.*)$.#/*0)3Y#+$&$=&%%#.)$*)$3%+$83+%&($*.$&=3/&($8#)8.3%#+E$%"#$8#)8.3%#$0#)#.&%#+$&$(*/&($`5:$+=3I#E$1"3/"$8#=*(&.3Y#+$%"#$/#(($'*8O$13%"*6%$0#)#.&%3)0$&$+*,&%3/$+=3I#4$C)$H!5$%"#*.OE$%"3+$3)%#.)&($8#=*(&.3Y#8$+%&%#$*7$%"#$/#(($.#=.#+#)%+$&$=.#83/%3*)$*7$76%6.#$&/%323%O$&)8$=(&O+$&$/.3%3/&($.*(#$3)$+#R6#)/#$,#,*.O4$)8$73)&((OE$6)8#.$+*,#$/*)83%3*)+$&$)#6.*)$13(($+%&.%$73.3)0$13%"$&$,3)3U'6.+%$*7$+=3I#+4$T)#$/*)83%3*)$%"&%$/&)$/&6+#$&$,3)3 SR

44 SXODWLRQVRIFHOOV
SXODWLRQVRIFHOOVWKHQSDWWHUQ´$µZLOOFRQFXUUHQWO\DFWLY&%#$SDWWHUQ´%4µ,IWKHQHXURQVLQSDWWHUQ´$µFRQQHFWWRWKHGLVWDOV\QDSVHVRIQHXURQVLQSDWWHUQ´%EµWKHQWKH´%µSDWWHUQZLOOEHSUHGLFWHG,IWKHQHXURQVLQSDWWHUQ´$µ

45 ;FRQQHFWWRWKHSUR
;FRQQHFWWRWKHSUR[LPDOV\QDSVHVRIQHXURQVLQSDWWHUQ´%EµWKHQWKH´%µSDWWHUQZLOOFRQVLVWRIDFWLYHQHXURQV$(($&++*/3&%32#$,#,*.O$*=#.&%3*)+$6+#$%"#$+&,#$'&+3/$,#,*.O$,#/"&)3+,E$%"#$7*.,&%3*)$*7$)#1$+O)&=+#+$*)$%"#$8#)8.3%#$+#0,#)%+$*7$)#6.*)+4$W#/&6+#$&(($)#6.*)+$"&2#$8*Y#)+$*7$8#)8.3%#$+#0,#)%+$&)8$%"*6+&)8+$*7$+O)&=+#+E$#&/"$)#6.*)$8*#+$)*%$-6+%$.#/*0)3Y#$*)#$=&%%#.)$'

46 6%$8*Y#)+$*7$3)8#=#)8#)%$=&%%#.)+4$$X&/"
6%$8*Y#)+$*7$3)8#=#)8#)%$=&%%#.)+4$$X&/"$)#6.*)$=&.%3/3=&%#+$3)$,&)O$8377#.#)%$F:M+4$!-.:;4,+-.(F:M+$&.#$%"#$(&)06&0#$*7$%"#$'.&3)E$&)8$H!5$%"#*.O$8#73)#+$"*1$%*$/.#&%#E$+%*.#E$&)8$.#/&(($F:M+$&)8$+#R6#)/#+$*7$F:M+4$C)$%"3+$/"&=%#.$1#$(#&.)#8$&'*6%$=*1#.76($,&%"#,&%3/&($=.*=#.%3#+$*7$F:M+E$&)8$"*1$%"#+#$=.*=#.%3#+$#)&'(#$%"#$'.&3)E$&)8$H!5E$%*$(#&.)$+#R6#)/#+$&)8$%*$0#)#.&(3Y#4$)&=&'&.:&,(e&)#.2&E$4$?Jf^^@4$F=&.+#$:3+%.3'6%#8$5#,*.O4$W.&87*.8$W**I+$*7$5C!$.#++4$$e&)#.2&E$4$?JffQ@4$[6((O$83+%.3'6%#8$.#=.#+#)%&%3*)4$.*/##

47 83)0+$*7$JffQ$M#&($S*.(8$;*,=6%3)0$FO,=*
83)0+$*7$JffQ$M#&($S*.(8$;*,=6%3)0$FO,=*+36,!5:&·!*IO*E$G&)4$JffQ@E$==4$PN^²PLN4$!+6I6'&U/3%OE$G&=&)_$M#&($S*.(8$;*,=6%3)0$&.%)#.+"3=4$$W(**,E$W4H4$?JfQA@4$F=&/#b$!3,#$!.&8#*77+$3)$H&+"$;*83)0$13%"$&#x -22;&#x 000;((*1&'(#$X..*.+4$;*,,6)3/&%3*)+$*7$%"#$&#x -22;&#x 000;;5E$9*(6,#$JPE$`6,'#.$QE$G6(O$JfQAE$==4$BKKUBKL4$W.*8#.E$&#x -22;&#x 000;4E$g$53%Y#),&/"#.E$54$?KAAB@4$`#%1*.I$&#x -22;&#x 000;==(3/&%3*)+$*7$W(**,$[3(%#.+E$&#x -22;&#x 000;$F6.2#O4$C)%#.)#%$5&%"#,&%3/+E$9*

48 (6,#$JE$`*4$BE$==4$B^NUNAf4 for this pro
(6,#$JE$`*4$BE$==4$B^NUNAf4 for this process (Fig. 1) comprises a set of inner hair cells organized in a row that are sensitive to different frequencies. When an appropriate frequency of sound occurs The output of an encoder must always produce the same number of bits for each of its inputs. SDRs are compared and operated on using a bit-by-bit assumption such that a bit with a certain ÒmeaningÓ is always in the same position. If the encoders produced varying bit lengths for the SDRs, comparisons and other operations wou

49 ld not be possible.4) The output should
ld not be possible.4) The output should have similar sparsity for all inputs and have enough one-bits to handle noise )/1! With this design, each bit in an SDR can represent multiple ranges of values. If these ranges are assigned via a hash function then the SDRs for two values that are far apart may overlap by one or two bits, but this small overlap will not cause a problem for the HTM. We will now show how this works in a numeric encoder, but the same principle is used in the geospatial encoder described later in the

50 chapter.If we look at step 6 in the pre
chapter.If we look at step 6 in the previous section, we see that each bucket is identified by a specific bit. We then select bits for the following w-1 buckets to complete the representation. Each bucket has overlapping bits with its neighbors. For a more flexible numeric encoder, we can do the same steps 1-6 but change how we select the bits in the representation. Specifically, we can use a hashing function to deterministically select one of the output bits from the bucket index. We do this selection separately for

51 each bit, so rather than w consecutive b
each bit, so rather than w consecutive bits being active, there are w bits active based on the hashing function. The hashing operation looks like this: The advantage of this method is that you donÕt need to restrict the values seen before. This encoder is different from the previous encoders since it uses both the current and the previous inputs to determine the output. The implementation of this encoder is the same as one of the other scalar encoders but you apply the encoder to the difference between the current inpu

52 t value and the previous. Example 2 Ð En
t value and the previous. Example 2 Ð Encoding Categories Many datasets contain categorical information. In some cases the data consists of discrete, cyclic cases, the encoding must Òwrap.Ó For this example, we will use a small number of bits in the encoding. In a real implementation you would want more bits active and would need more total bits as a result. Figure meaning, we first have to determine the resolution that we want to encode. For this example, we geospatial encoding requires that the distance of two posi

53 tions that you want to have overlap dete
tions that you want to have overlap determine the number of active bits. You may not always want that many bits active. We can subsample from the range but need to determine which bits to subsample in a way that preserves the desired encod ... 8 9 10 11 12 13 14 15 ... 2 3 4 5 6 7 8 9 ... .3)/3=&($&#x 4 0;6%"*.?+@$A4B$5&./"$CADE$F)3%3&($.#(#&+#$;4$G&.&)+H3$12$-'%#.%(3-%&%-()&',5(!"3+$8*/6,#)%I$&$=(&/#"*(8#.$7*.$&$76(($/"&=%#.$*)$%"#$J=&

54 %3&($**(3)0$&(0*.3%",I$3+$&$(3+%$*7$/6..
%3&($**(3)0$&(0*.3%",I$3+$&$(3+%$*7$/6..#)%(K$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$

55 $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$&2&3(&'(#$.#+*6./#+$*)$J=&%3&($**(3)046$#%+#7(8--7+.9(!"#$)#*/*.%#L$3+$/*,=*+#8$*7$&$"3#.&./"K$*7$.#03*)+4$!"#$7##87*.1&.8$3)=6%$%*$#&/"$.#03*)$/*)+3+%+$*7$

56 ,3((3*)+$*7$)#.2#$&L*)+$1"3/"$/*,#$7.*,$
,3((3*)+$*7$)#.2#$&L*)+$1"3/"$/*,#$7.*,$+#)+*.K$*.0&)+$&)8$*%"#.$.#03*)+4$5*+%$.#03*)+$3)$%"#$)#*/*.%#L$.#/#32#$3)=6%$7.*,$,6(%3=(#$+*6./#+M$3%$3+$)*%$6)/*,,*)$7*.$&$.#03*)$%*$.#/#32#$3)=6%$7.*,$+3L$*.$,*.#$*%"#.$.#03*)+4$!"#+#$3)=6%+$&.#$0#)#.&((K$,L[HGWRJHWKHUVRDUHJLRQGRHVQ·W´NQRZµZKHUHWKHLQSXWVRULJLQDWHGRUZKDWWKH\UHSUHVHQW&RQIXVLQJWKHSLFWXUHWKH�

57 03;QXPEHURILQSXW
03;QXPEHURILQSXWD[RQVGRHVQ·WFRUUHODWHFORVHO\ZLWKWKHVL]HRIWKHUHJLRQ+RZFDQDUHJLRQSURFHVV3)=6%+$7.*,$,&)K$8377#.#)%$+*6./#+$13%"*6%$&)K$=.3*.$H)*1(#80#$*7$1"&%$%"#+#$3)=6%+$.#=.#+#)%I$"*1$,&)K$3)=6%$'3%+$%"#.#$13(($'#I$&)8$1"&%$+=&%3&($=&%%#.)+$,&K$#L3+%$3)$%"#$3)=6%N$J=&%3&($**(3)0I$&$(#&.)3)0$,#/"&)3+,$76)8&,#)%&($%*$'*%"$%"#$)#*/*.%#L$&)8$O3#.&./"3/&($!#,=*.&($5#,*.K$?O

58 !5@I$3+$%"#$&)+1#.$%*$%"3+$=.*'(#,4$P"3(
!5@I$3+$%"#$&)+1#.$%*$%"3+$=.*'(#,4$P"3(#$1#$&.#$1*.H3)0$*)$&$+%&)8&(*)#$/"&=%#.$7*.$G&#x 4 0;5F$%"&%$13(($8#%&3($%"#$J=&%3&($**(3)0$&(0*.3%",I$7*.$)*1$K*6$/&)$#L=(*.#$%"#$7*((*13)0$.#+*6./#+$%*$(#&.)$,*.#Q$!x /:;(6"--7(&$+,-4&,(#=-&#x 6 0;%(6$#%+#7(8--7+.9(‡O!5$J/"**($3+$&$238#*$%6%*.3&($+#.3#+$%"&%$#L=(&3)+$/*)/#=%+$*7$O!5$13%"$&)$#,="&+3+$*)$23+6&(3R&%3*)+$&)8$#L&,=(#+4$$!"#+#$#=3+*8#+$%&(H$&'*6%$J=&%3&($**(3)0$&)8$.#(&%#8$76)/%3*)+4$! J=&%3&($**(3)0Q$F)=6%$J=&/#$S$;*))#/%3*)+$?T=3+*8#$E@$! J=&%3&($**(

59 3)0Q$U#&.)3)0$?T=3+*8#$V@$! G**+%3)0$?T=
3)0Q$U#&.)3)0$?T=3+*8#$V@$! G**+%3)0$?T=3+*8#$W@$$x 8&&'?'&*+&@&4($#$&'(‡$!"3+$8*/6,#)%I$&6%"*.#8$'K$X61#3$;63I$J6'6%&3$‡",&8$&)8$Y#77$O&1H3)+I$3+$&$=.#Z=.3)%$*7$&$=&=#.$+6',3%%#8$%*$&$=##.Z.#23#1#8$-*6.)&(4$!"#$=&=#.$/*)%&3)+$&$8#%&3(#8$8#+/.3=%3*)$*7$%"#$J=&%3&($**(3)0$&(0*.3%",I$3)/(683)0$,&%"#,&%3/&($8#.32&%3*)+$&)8$#L=#.3,#)%&($.#+6(%+4$! !"#$O!5$J=&%3&($**(#.Q$&$)#*/*.%3/&($&(0*.3%",$7*.$*)(3)#$+=&.+#$83+%.3'6%#8$/*83)0$$$x 6$#%+#7(8--7+.9($'&,&.%#%+-.(‡$F)$%"3+$.#/*.8#8$=.#+#)%&%3*)I$X61#3$;6

60 3$0*#+$3)%*$8#%&3(+$*7$%"#$J=&%3&($**(3)
3$0*#+$3)%*$8#%&3(+$*7$%"#$J=&%3&($**(3)0$&(0*.3%", Feb 2017 Update to Inhibition radius: The size of a columnÕs local neighborhood, within which columns may inhibit each other from becoming active. function, broken down into four phases: th or higher among the columns within its inhibition radius. .3)/3=&($&#x 4 0;6%"*.?+@$A4B$5&./"$CADE$F)3%3&($.#(#&+#$;4$G&.&)+H3$12$-'%#.%(3-%&%-()&',5(!"3+$8*/6,#)%I$&$=(&/#"*(8#.$7*.$&$76(($/"&=%#.$*)$%"#$!#,=*.&($5#,*.J$&(0*.3%",I$3+$&$(3+%$*7$/6..#)%(J$$$$$$$$$$$$$$$$$$$$$$$

61 $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$

62 $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$&2&3(&'(#$.#+*6./#+$*)$!#,=*.&($5#,*.J4$$6&2$-'#7(8&2-'0(F)$K!5I$+#L6#)/#$,#,*.J$3+$3,=(#,#)%#8$'J$%"#!#,=*.&($5#,*.J$&(0*.3%",4$!"3+$&(0*.3%",$(#&.)+$+#L6#)/#+$*7$M=&.+#$:3+%.3'6%#8$N#=.#+#)%&%3*)+$?M:N+@$7*.,#8$'J$%"#$M=&%

63 3&($**(3)0$&(0*.3%",I$&)8$,&H#+$=.#83/%3
3&($**(3)0$&(0*.3%",I$&)8$,&H#+$=.#83/%3*)+$*7$1"&%$%"#$)#O%$3)=6%$M:N$13(($'#4$P"3(#$1#$&.#$1*.H3)0$*)$&$+%&)8&(*)#$/"&=%#.$7*.$G&#x 4 0;5F$%"&%$13(($8#%&3($%"#$ '&*+&?&4($#$&',(‡$! P"J$T#6.*)+$K&2#$!"*6+&)8+$*7$MJ)&=+#+I$&$!"#*.J$*7$M#L6#)/#$5#,*.J$3)$T#*/*.%#O$$!"3+$=&=#.I$&6%"*.#8$'J$U#77$K&1H3)+$&)8$M6'6%&3$‡",&8I$1&+$=6'(3+"#8$*)$5&./"$VAI$CADW$3)$X.*)%3#.+$3)$T#6.&($;3./63%+4$!"#$=&=#.I$1.3%%#)$7.*,$&$)#6.*+/3#)/#$=#.+=#/%32#I$8#+/.3'#+$%"#$/*.#$K!5$%"#*.J$7*.$ $5#,*.J$DOJRULWKPFDOOH

64 G´+70VHTXHQFHPH
G´+70VHTXHQFHPHPRU\µLQWKHSDSHUDQGFRPSDUHVLWWRVWDWLVWLFDODQG'HHSZ#&.)3)0$%#/")3L6#+4$$$x 6&2$-'#7(8&2-'0(#7@-'+%"2(4&%#+7,(‡$+#68*/*8#$3,=(#,#)%3)0$%"#$2#.+3*)$*7$%"#$!#,=*.&($5#,*.J$ forward connectio patterns to recognize what it is. Vision is usually like that, you usually process a stream of visual images. But under certain conditions you can recognize an image with a single exposure

65 . Temporal and static recognition might
. Temporal and static recognition might appear to require different inference mechanisms. One requires recognizing sequences of patterns and making predictions based on variable length context. The other requires recognizing a static spatial pattern without using temporal context. An HTM layer with multiple cells per column is ideally suited for recognizing time-based sequences, and an HTM layer with one cell per column is ideally suited to recognizing spatial patterns. Temporal Memory Algorithm Steps 1) Form a repre

66 sentation of the input in the context of
sentation of the input in the context of previous inputs 2). Figure 2 By activating a subset of cells in each column, an HTM layer can represent the same input in many different contexts. Columns only activate predicted cells. Columns with no predicted cells activate all the cells in the column. The figure shows some columns with one cell active and some columns with all cells active. Consider hearing two spoken sentences, ÒI ate a pearÓ and ÒI have eight pearsÓ. The words ÒateÓ and ÒeightÓ are homonyms; they sound

67 identical. We can be certain that at so
identical. We can be certain that at some point in the brain there are neurons that respond identically to the spoken words ÒateÓ and ÒeightÓ. After all, identical sounds are entering the ear. However, we also can be certain that at another point in the brain the neurons that respond to this input are different, in different contexts. The representations for the sound ÒateÓ will be different when you hear ÒI ateÓ vs. ÒI have eightÓ. Imagine that you have memorized the two sentences ÒI ate a pearÓ and ÒI have eight pear

68 sÓ. Hearing ÒI ate...Ó leads to a differ
sÓ. Hearing ÒI ate...Ó leads to a different prediction than ÒI !,&-'%"&!,$-./,-�070?.$-0?1%&'()2$%&'()3 Select a winner cell and a learning segment for the column (lines 29-35). If any cells have a matching segment, select the best matching segment and its cell (lines 30-31). Otherwise select the least used cell and grow a new segment on it connected and potential synapses that are active (lines 58 #)',#&$$E%%*0L,)?%(2&0,*-[*0?10&&*3! The least used cell is the cell with the fewest segments. Find all the cells

69 with the fewest segments (lines 74 When
with the fewest segments (lines 74 When growing synapses, grow to a random subset of the previous time stepÕs winner cells. A segment can only co March 2017 ,&-.+.#/(0.+*#"1(!"#$%&'(#$)*%#+$,&-*.$/"&)0#+$'#%1##)$.#23+3*)+4$53)*.$/"&)0#+$+6/"$&+$+,&(($/(&.373/&%3*)+$*.$7*.,&%%3)0$/"&)0#+$&.#$)*%$)*%#84$$9#.+3*)$:&%#$;"&)0#+$3)/3=&($&#x. -1; 00;6%"*.?+@$A4B %.&3)#8$KP$E4 S*1$8*#+$*)(3)#$(#&.)3)0$"&==#)$3)$%"#$KP$W4 ;&)$F*6$8*$+=&%3&($=**(3)0$13%"$+,&(($)6,'#.+P$U*.$#Q&,=(#M$3+$3%$.#&+*)&'(#$%*$"&2#$&)$K$13%"$HA$/*(6,)