/
DISTRIBUTION A. Approved for public release: distribution unlimited. DISTRIBUTION A. Approved for public release: distribution unlimited.

DISTRIBUTION A. Approved for public release: distribution unlimited. - PowerPoint Presentation

natalia-silvester
natalia-silvester . @natalia-silvester
Follow
342 views
Uploaded On 2019-11-23

DISTRIBUTION A. Approved for public release: distribution unlimited. - PPT Presentation

DISTRIBUTION A Approved for public release distribution unlimited Lifelong Learning Machines L2M Hava T Siegelmann US Defense Advanced Research Projects Agency MTO L2M Proposers Day April 26 2017 ID: 767314

approved distribution release public distribution approved public release unlimited learning turing www l2m program http systems machines driving https

Share:

Link:

Embed:

Download Presentation from below link

Download Presentation The PPT/PDF document "DISTRIBUTION A. Approved for public rele..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.


Presentation Transcript

DISTRIBUTION A. Approved for public release: distribution unlimited. Lifelong Learning Machines (L2M) Hava T. SiegelmannU.S. Defense Advanced Research Projects Agency MTO L2M Proposers Day April 26, 2017

DISTRIBUTION A. Approved for public release: distribution unlimited. 1 Purpose of this briefing Discuss program objectives and structure BAA takes precedence These slides are meant to provide background and clarification only Please consult published BAA for final program specificsUntil the deadline for receipt of proposalsWe encourage open communication between proposers and the program managerInformation given to one proposer must be available to all proposers The best way to get a question answered is to submit it by email and retrieve the response from the Frequently Asked Questions (FAQ) list via the DARPA Opportunities webpage (see “Useful links” at the end of this presentation)Questions that contain distribution restrictions, such as “company proprietary,” will not be answered Ground rules Questions: HR0011-17-S-0016@darpa.mil

What is the state of Artificial Intelligence (AI) today? 2 DISTRIBUTION A. Approved for public release: distribution unlimited.

DISTRIBUTION A. Approved for public release: distribution unlimited. 3 GamingIn 2017, Google DeepMind’s AlphaGo AI compiled a 60-0 record against premier Go players AI successes http ://fortune.com/2017/01/07/google-alphago-ai/ Image recognitionIn 2015, Microsoft outperformed humans on ImageNet Large Scale Visual Recognition Challenge  Other researchers subsequently created AI systems that outperformed humans in image recognition Other researchers previously created AI systems that outperformed humans in chess, poker, Jeopardy, and Atari https ://www.microsoft.com/en-us/research/blog/microsoft-researchers-algorithm-sets-imagenet-challenge-milestone/

AI limitations Google’s self-driving carThe 2016 collision was due to a human driver running a red light—not the self-driving car’s fault Uber’s self-driving car The 2017 collision was due to a human driver failing to yield— not the self-driving car’s fault AI systems only compute with what they’ve been programmed or trained for in advance 4 DISTRIBUTION A. Approved for public release: distribution unlimited. http ://www.abc15.com/news/region-southeast-valley/tempe/tempe-police-self-driving-uber-vehicle-involved-in-car-accident-no-injuries https ://www.theguardian.com/technology/2016/sep/26/google-self-driving-car-in-broadside-collision-after-other-car-jumps-red-light-lexus-suv

DISTRIBUTION A. Approved for public release: distribution unlimited. 5 Tesla self-driving electric car cleared in deadly collision http :// www.businessinsider.com/what-went-wrong-in-the-tesla-autopilot-crash-2016-7 “ The regulators warned, however, that advanced driver-assistance systems like the one in Tesla’s cars could be relied upon to react properly in only some situations that arise on roadways. And the officials said that all automakers needed to be clear about how the system should be used.” [emphasis added]https://www.nytimes.com/2017/01/19/business/tesla-model-s-autopilot-fatal-crash.html?_r=0 The collision was not the self-driving car’s fault

DISTRIBUTION A. Approved for public release: distribution unlimited. 6 AI is successful but brittle We want the rigor of automation with the flexibility of a human Catastrophic forgetting Retraining is expensive and time consuming Is there an easy fix? No. But Programming is frozen during execution Malfunctions in circumstances beyond preparation AI systems operating in real-world environments are bound to fail at some point Rigor of automation = free from human errors (e.g., no texting while driving)

DISTRIBUTION A. Approved for public release: distribution unlimited. 7 What is our plan to improve AI?

8 L2M will develop fundamentally new machine learning mechanisms that will enable systems to improve their performance over their lifetimes Lifelong Learning Machines (L2M) program objective Performance Time Time Continuously Improve Performance Training Fielded Training Fielded Current AI L2M Adapt to New Conditions Adapts to changing environment Can’t adapt to new mission Improves at the task Performance Current machine learning is based on large datasets; DoD data may be scarce Situation may change after training and fielding (external, internal) Surprise DISTRIBUTION A. Approved for public release: distribution unlimited.

9 In 1936, A lan Turing modeled “human -calculators” as theoretical a utomatic machines Today’s computational foundation: Turing MachinesCurrent AI has two pre-execution parts Program and rules Parameter learning (e.g., in ML) Loadable program Memory tape I nput Output http ://2.bp.blogspot.com/-2_iIEWHGey0/VKLLvir2OuI/AAAAAAAALPY/62FByqqVViY/s1600/alan-turing.jpg DISTRIBUTION A. Approved for public release: distribution unlimited.

10 Natural systems don’t freeze at execution DISTRIBUTION A. Approved for public release: distribution unlimited. “…it is not the strongest that survives; but…the one that is able best to adapt…to the changing environment….”L.C. Megginson, re “On the Origin of Species” http://onedio.co/content/get-to-know-charles-darwin-better-with-these-13-facts-14574

11 Definition Storage process for restoring retrieved memories through which memories can be reinforced , faded , and modified toward new experiences EffectsTemporal changes to memory leading to adaptive behavior regulationDefinitionDynamic alterations in cell’s transcription—external to DNA strand—that affect how cells express genes based on external/environmental factors EffectsChanges in the way of interpreting DNA, leading to adaptive organisms http ://ind3.ccio.co/w3/B9/r1/b54462b246c38979b0288dffa00a186b.jpg?iw=300 http :// 2.bp.blogspot.com/EoTDSv8D_tc/U_eYScuZB_I/AAAAAAAAE58/08xglTNvLh8/s1600/Mechanisms-of-epigenetics.jpg DISTRIBUTION A. Approved for public release: distribution unlimited. Epigenetics Brain Reconsolidation Nature’s mechanisms change beyond preloaded programs

12 DISTRIBUTION A. Approved for public release: distribution unlimited. Theoretical computer science beyond pre-programming Possible Ingredients Analog value/rich information Randomness/asynchronous Evolving, plasticity Interactive paradigm: from to (out of ) Property of using only as much precision and energy as needed Siegelmann, “Computation Beyond the Turing Limit” (Science, Vol. 268, 28 April 1995 )   Continuum of computational hierarchy f rom Turing Machines (fixed programs) to Super-Turing Computation (modifiable programs)

13 Turing foresaw the feasibility of a superior model “ Electronic computers are intended to carry out any definite rule of thumb process … working in a disciplined but unintelligent manner .” (Turing, ’50)“My contention is that machines can be constructed that will simulate the behavior of the human mind.” (Turing, ’51) Turing argued against his peers that machines would someday be more powerful than what we now call Turing machines. He suggested various types of “unorganized machines.” DISTRIBUTION A. Approved for public release: distribution unlimited. http ://www.cs4fn.org/turing/alanturingslife.php

14 DISTRIBUTION A. Approved for public release: distribution unlimited. Natural systems run fixed (Turing-like) programs They adapt as needed, changing their Turing programs They store revised Turing programs as components for future use Some properties of brains as lifelong learning machines Plastic nodal network structures are not homogeneous or fixed Signaling combines discrete and continuous (e.g., chemical vs electrical)Signaling may change modes of behavior (e.g., exposure to drugs, stress)Ability to select which inputs drive adaptation Nature combines Turing and Super-Turing Computation http ://www.webmd.com/brain/picture-of-the-brain#1

Continuous adaptation echanisms Flexible model L2M vision Today Execution follows completed training cycle Execution is fixed In four years Continues learning during execution Can adapt to new situations Prepared code, training Rigid model Input Output Input Output Training Fielded The situation changed, and now my machine keeps making the same mistakes over and over! Wow! This machine gets better with time. Prepared code, training DISTRIBUTION A. Approved for public release: distribution unlimited. 15

16 TA 2: Biological p rinciples Learn from nature Transfer to machine learning TA 1: Lifelong learning systems Software, architectures, and algorithms Theory , including combining supervised, unsupervised, and reward-based learning Error on Set 1 Time old algorithm UTK algorithm Introduction of Set 2 Continuous adaptation m echanisms Flexible m odel Input Output L2M DISTRIBUTION A. Approved for public release: distribution unlimited. L2M program structure

Core capabilities of an L2M system Continual learning – systems capable of learning during execution Adaptation to new tasks and circumstances – applying previously learned skills to novel situations without forgetting previously learned tasks Goal-driven perception – understanding input signals from mission viewSelective plasticity – balancing stability and plasticitySafety and monitoringDISTRIBUTION A. Approved for public release: distribution unlimited.17

DISTRIBUTION A. Approved for public release: distribution unlimited. 18 Current efforts to advance AI are limited Progressive neural networks grow a new fixed-structure network for each new game and utilize information from previously trained ones Limitations Adds full network for each new task (e.g., Atari game )—not scalable System must be told when task is new Only works for some combinations DeepMind , 2016 Input Trained game Next New game

19 Continual learning Adapting to new tasks Goal-driven perception Selective plasticity Safety & monitoring Proof of concept Quantifiably demonstrate working prototype Phase 1 Final Core capabilities Generalized lifelong learning solutions Quantifiably demonstrate final system We seek a new chapter for AI: new structures that can modify their own rules—possibly a new type of neural net—in this four-year, two-phase 6.1 program Technical area 1 milestones DISTRIBUTION A. Approved for public release: distribution unlimited.

20 Technical area 1 examples of demonstration applications http:// www.jabil.com /technologies/heavy-fuel-engines-for-unmanned-aerial-vehicles/ https:// www.lorextechnology.com/hd-dvr-security-system/complete-16-camera-security-system-with-monitor/MPX16124MVDW-1-p https://www.youtube.com / watch?v =76XXev8R6YY https://lh3.googleusercoAtent.com/XuSIt4CPzTb9FcOJqYe4i31Mgd3TyzC5JJIash37fZtWPjr7mUNW1WEmAeHebIWDZ_OM=s147 DISTRIBUTION A. Approved for public release: distribution unlimited. Any domain area is welcome as long as the performer shows all 5 core capabilities

Technical area 2: transfer natural mechanisms to AI DISTRIBUTION A. Approved for public release: distribution unlimited. 21 6.1 program—wet lab research OK, modeling and literature study OK Study Study Transfer Demonstrate mechanisms from biology or other physical domains that support lifelong learning and have not yet been applied to machine learning m echanisms into lifelong learning algorithms that algorithms quantifiably improve performance of at least one of the five core capabilities

22 Technical area 2: new mechanisms for core capabilities Salamanders Planaria Regeneration local global background Updating frequencies for relevance F rom H. Markram F rom L. Kay F rom M. Levine Playground for computational biologists DISTRIBUTION A. Approved for public release: distribution unlimited. Neuronal varieties

23 Center Group: develop future L2M community leaders DISTRIBUTION A. Approved for public release: distribution unlimited. Goal. Encourage cross-pollination of emerging approaches to share expertise in biology and computation —unique training for members Approach. Share program-level experience and insights to develop collective understanding ; collaboration is not meant to include propriety data Outputs. Cutting edge results, new evaluation paradigms and metrics, common test problems Members. Government experts and one technical representative from each performer

24 Outside the scope of our project The point of the L2M program is to study structures that change with live circumstances and cause changes in behavior We are not looking for…Incremental improvements to the current state of neural networksMethods that add updates to current neural networks but do not scale (in growing number of tasks, resources, etc.)Philosophical approachesDISTRIBUTION A. Approved for public release: distribution unlimited.

25 L2M Broad Agency Announcement (BAA) https://www.fbo.gov/spg/ODA/DARPA/CMO/HR001117S0016/listing.htmlMore information, including a Frequently Asked Questions (FAQ) document, will be posted and updated regularly on the DARPA Opportunities webpagehttp:// www.darpa.mil/work-with-us/opportunities?tFilter=85&oFilter =& sort=name Useful linksDISTRIBUTION A. Approved for public release: distribution unlimited.

26 Thank you DISTRIBUTION A. Approved for public release: distribution unlimited. “Once you stop learning, you start dying.” Albert Einstein “…it is not the strongest that survives; but…the one that is able best to adapt…to the changing environment….”L.C. Megginson, re “On the Origin of Species” https://www.izlesene.com/iz/memcn3342

27 DISTRIBUTION A. Approved for public release: distribution unlimited.