Optimal Control via Neural Networks:A Convex ApproachYize Chen*, Yuany - PDF document

Optimal Control via Neural Networks:A Co
Optimal Control via Neural Networks:A Convex ApproachYize Chen*, Yuanyuan Shi*, Baosen Zhang Motivation and IntroductionHow to employ data for optimal control?PlantDisturbanceInputControllerCostsConstraints State•Model-Free RL simultaneously parameterize -Poor data efficiency-Dynamics is easier to parameterize, sample efficientOur work:Learn to Control OptimallyInput Convex NN Predictive ModelInput Convex NN Input OptimizerPlantDisturbanceInputStateAction*ActionCostsConstraints Training stageDesign NN Dynamics for Optimal ControlControl Task I: Building Energy ManagementFuture DirectionsControl Task II: MuJoCo Locomotion Task-Probabilistic system dynamics and uncertain rewards[1] Amos, B., Xu, L., & Kolter, J. Z. Input convex neural networks. ICML’17[2] Nagabandi, A., Kahn, G., Fearing, R. S., & Levine, S. deep reinforcement learning with model-free fine-tuning. ICRA’18Efficiency and Representation PowerAdvantages coming from such construction of ICNNNetwork Design•Direct passthrough layers •All weights and nonnegative •Expanded inputs •Nonnegative sum of •Composition of convex and convex, non-decreasing convex functionsModel Predictive Control (MPC) Convex Objective, Affine Constraints A NN with non-negative weights and ReLU activationVariableNormal WeightsAffine functionsConvexFunctions affine function can be represented by Learn an ICNN is much more efficient than -layer ICNN needs pieces of affine functionsICNN representation powerfor convex functionsICRNNTradeoffTraining: �raining: Networks: 2-layer ICNNGoal: Minimize building energy consumpion/electricity billControl: zone temperature setpoint (every 15mins)States: zone temperature, occupancy, appliances scheduleTraining:�raining: Cumulative RewardsComputation Time vs Random Shooting [2]Computation efficient than previousmodel-based RL algorithmsControl: zone temperature setpoint