Lecture given at MIT May 6, 2014 (shorter version given at ITA UCSD on Valentines Day 2014). Based on joint research with Ana Busic, Prabir Barooah, Jordan Erhan, and Yue Chen, contained in three papers at http://www.meyn.ece.ufl.edu/pp Renewable energy sources such as wind and solar power have a high degree of unpredictability and time-variation, which makes balancing demand and supply challenging. One possible way to address this challenge is to harness the inherent flexibility in demand of many types of loads. At the grid-level, ancillary services may be seen as actuators in a large disturbance rejection problem. It is argued that a randomized control architecture for an individual load can be designed to meet a number of objectives: The need to protect consumer privacy, the value of simple control of the aggregate at the grid level, and the need to avoid synchronization of loads that can lead to detrimental spikes in demand. I will describe new design techniques for randomized control that lend themselves to control design and analysis. It is based on the following sequence of steps: 1. A parameterized family of average-reward MDP models is introduced whose solution defines the local randomized policy. The balancing authority broadcasts a common real-time control signal to the loads; at each time, each load changes state based on its own current state and the value of the common control signal. 2. The mean field limit defines an aggregate model for grid-level control. Special structure of the Markov model leads to a simple linear time-invariant (LTI) approximation. The LTI model is passive when the nominal Markov model is reversible. 3. Additional local control is used to put strict bounds on individual quality of service of each load, without impacting the quality of grid-level ancillary service. Examples of application include chillers, flexible manufacturing, and even residential pool pumps. It is shown through simulation how pool pumps in Florida can supply a substantial amount of the ancillary service needs of the Eastern U.S.

Published on: **Mar 4, 2016**

- 1. Distributed Randomized Control for Ancillary Service to the Power Grid With application to rational pools LIDS Seminar May 6, 2014 Ana Buˇsi´c and Sean Meyn Prabir Barooah, Yue Chen, and Jordan Ehren TREC & D´epartement d’Informatique INRIA & ENS Electrical and Computer Engineering University of Florida Thanks to NSF, AFOSR, ANR, and DOE / TCIPG
- 2. Outline 1 Distributed Control of Loads 2 Design for Intelligent Loads 3 Linearized Dynamics and Passivity 4 One Million Pools 5 Conclusions and Extensions 6 References 0 / 28
- 3. Power GridControl WaterPump Batteries Coal GasTurbine BP BP BP C BP BP Voltage Frequency Phase HC Σ − Actuator feedback loop A LOAD Architecture for Distributed Control
- 4. Distributed Control of Loads Power grid as a feedback control system Massive disturbance rejection problem One day in Gloucester: 1 / 28
- 5. Distributed Control of Loads Power grid as a feedback control system Massive disturbance rejection problem Two typical weeks in the Paciﬁc Northwest (BPA): 0 2 4 6 8 2 4 6 8 0 0.8 -0.8 1 -1 0 0.8 -0.8 1 -1 0 Sun Mon Tue October 20-25 October 27 - November 1 Hydro Wed Thur FriSun Mon Tue Wed Thur Fri GenerationandLaodGW GWGW RegulationGW Thermal Wind Load Generation Regulation 1 / 28
- 6. Distributed Control of Loads Control of Deferrable Loads Control Goals and Architecture Power GridControl WaterPump Batteries Coal GasTurbine BP BP BP C BP BP Voltage Frequency Phase HC Σ − Actuator feedback loop A LOAD Context: Consider population of similar loads that are deferrable. Two examples of ﬂexible energy hogs: Chillers in HVAC systems and residential pool pumps 2 / 28
- 7. Distributed Control of Loads Control of Deferrable Loads Control Goals and Architecture Power GridControl WaterPump Batteries Coal GasTurbine BP BP BP C BP BP Voltage Frequency Phase HC Σ − Actuator feedback loop A LOAD Context: Consider population of similar loads that are deferrable. Two examples of ﬂexible energy hogs: Chillers in HVAC systems and residential pool pumps Control Constraints: Grid operator demands reliable ancillary service; Consumers demand reliable service (and more) 2 / 28
- 8. Distributed Control of Loads One Million Pools in Florida Pools Service the Grid Today On Call1: Utility controls residential pool pumps and other loads 1 Florida Power and Light, Florida’s largest utility. www.fpl.com/residential/energysaving/programs/oncall.shtml 3 / 28
- 9. Distributed Control of Loads One Million Pools in Florida Pools Service the Grid Today On Call1: Utility controls residential pool pumps and other loads Contract for services: no price signals involved Used only in times of emergency — Activated only 3-4 times a year 1 Florida Power and Light, Florida’s largest utility. www.fpl.com/residential/energysaving/programs/oncall.shtml 3 / 28
- 10. Distributed Control of Loads One Million Pools in Florida Pools Service the Grid Today On Call1: Utility controls residential pool pumps and other loads Contract for services: no price signals involved Used only in times of emergency — Activated only 3-4 times a year Opportunity: FP&L already has their hand on the switch of nearly one million pools! 1 Florida Power and Light, Florida’s largest utility. www.fpl.com/residential/energysaving/programs/oncall.shtml 3 / 28
- 11. Distributed Control of Loads One Million Pools in Florida Pools Service the Grid Today On Call1: Utility controls residential pool pumps and other loads Contract for services: no price signals involved Used only in times of emergency — Activated only 3-4 times a year Opportunity: FP&L already has their hand on the switch of nearly one million pools! Surely pools can provide much more service to the grid 1 Florida Power and Light, Florida’s largest utility. www.fpl.com/residential/energysaving/programs/oncall.shtml 3 / 28
- 12. Distributed Control of Loads One Million Pools in Florida Pools Service the Grid Today On Call1: Utility controls residential pool pumps and other loads Contract for services: no price signals involved Used only in times of emergency — Activated only 3-4 times a year Opportunity: FP&L already has their hand on the switch of nearly one million pools! Surely pools can provide much more service to the grid Summary (beyond pools): For many household loads, contracts with consumers are already in place 1 Florida Power and Light, Florida’s largest utility. www.fpl.com/residential/energysaving/programs/oncall.shtml 3 / 28
- 13. Distributed Control of Loads One Million Pools in Florida Pools Service the Grid Today On Call1: Utility controls residential pool pumps and other loads Contract for services: no price signals involved Used only in times of emergency — Activated only 3-4 times a year Opportunity: FP&L already has their hand on the switch of nearly one million pools! Surely pools can provide much more service to the grid Summary (beyond pools): For many household loads, contracts with consumers are already in place For other loads, such as plug-in electric vehicles (PEVs), consumer risk may be too great. 1 Florida Power and Light, Florida’s largest utility. www.fpl.com/residential/energysaving/programs/oncall.shtml 3 / 28
- 14. Distributed Control of Loads Control of Deferrable Loads Randomized Control Architecture Context: Consider population of similar loads that are deferrable. Constraints: Grid operator demands reliable ancillary service; Consumer demands reliable service Control strategy Requirements: 4 / 28
- 15. Distributed Control of Loads Control of Deferrable Loads Randomized Control Architecture Context: Consider population of similar loads that are deferrable. Constraints: Grid operator demands reliable ancillary service; Consumer demands reliable service Control strategy Requirements: 1. Minimal communication: Each load should know the needs of the grid, and the status of the service it is intended to provide. 4 / 28
- 16. Distributed Control of Loads Control of Deferrable Loads Randomized Control Architecture Context: Consider population of similar loads that are deferrable. Constraints: Grid operator demands reliable ancillary service; Consumer demands reliable service Control strategy Requirements: 2. Smooth behavior of the aggregate =⇒ Randomization 4 / 28
- 17. Distributed Control of Loads Control of Deferrable Loads Randomized Control Architecture Control strategy Requirements: 1. Minimal communication: Each load should know the needs of the grid, and the status of the service it is intended to provide. 2. Smooth behavior of the aggregate =⇒ Randomization Need: A practical theory for distributed control based on this architecture For recent references see thesis of J. Mathieu 4 / 28
- 18. Distributed Control of Loads Example: One Million Pools in Florida How Pools Can Help Regulate The Grid 1,5KW 400V Needs of a single pool Filtration system circulates and cleans: Average pool pump uses 1.3kW and runs 6-12 hours per day, 7 days per week 5 / 28
- 19. Distributed Control of Loads Example: One Million Pools in Florida How Pools Can Help Regulate The Grid 1,5KW 400V Needs of a single pool Filtration system circulates and cleans: Average pool pump uses 1.3kW and runs 6-12 hours per day, 7 days per week Pool owners are oblivious, until they see frogs and algae 5 / 28
- 20. Distributed Control of Loads Example: One Million Pools in Florida How Pools Can Help Regulate The Grid 1,5KW 400V Needs of a single pool Filtration system circulates and cleans: Average pool pump uses 1.3kW and runs 6-12 hours per day, 7 days per week Pool owners are oblivious, until they see frogs and algae Pool owners do not trust anyone: Privacy is a big concern 5 / 28
- 21. Distributed Control of Loads Example: One Million Pools in Florida How Pools Can Help Regulate The Grid 1,5KW 400V Needs of a single pool Filtration system circulates and cleans: Average pool pump uses 1.3kW and runs 6-12 hours per day, 7 days per week Pool owners are oblivious, until they see frogs and algae Pool owners do not trust anyone: Privacy is a big concern Randomized control strategy is needed. 5 / 28
- 22. yt Control @ Utility Gain One Million Pools Disturbance to be rejected Proportion of pools on desired ζt d dt µt = µtDζt yt = µt, U Intelligent Appliances
- 23. Distributed Control of Loads General Model Controlled Markovian Dynamics Assumptions 1 Continuous-time model on ﬁnite state space X 2 Controlled Markovian generator Dζ(x, y) : x, y ∈ X, ζ ∈ R. 6 / 28
- 24. Distributed Control of Loads General Model Controlled Markovian Dynamics Assumptions 1 Continuous-time model on ﬁnite state space X 2 Controlled Markovian generator Dζ(x, y) : x, y ∈ X, ζ ∈ R. D = D0 models nominal behavior of load. 6 / 28
- 25. Distributed Control of Loads General Model Controlled Markovian Dynamics Assumptions 1 Continuous-time model on ﬁnite state space X 2 Controlled Markovian generator Dζ(x, y) : x, y ∈ X, ζ ∈ R. D = D0 models nominal behavior of load. 3 Smooth and Lipschitz: For a constant ¯b, d dζ Dζ(x, y) ≤ ¯b, for x, y ∈ X, ζ ∈ R. 6 / 28
- 26. Distributed Control of Loads General Model Controlled Markovian Dynamics Assumptions 1 Continuous-time model on ﬁnite state space X 2 Controlled Markovian generator Dζ(x, y) : x, y ∈ X, ζ ∈ R. D = D0 models nominal behavior of load. 3 Smooth and Lipschitz: For a constant ¯b, d dζ Dζ(x, y) ≤ ¯b, for x, y ∈ X, ζ ∈ R. 4 Utility function U : X → R represents state-dependent power consumption 6 / 28
- 27. Distributed Control of Loads General Model Controlled Markovian Dynamics Assumptions 1 Continuous-time model on ﬁnite state space X 2 Controlled Markovian generator Dζ(x, y) : x, y ∈ X, ζ ∈ R. D = D0 models nominal behavior of load. Each load is subject to common controlled Markovian dynamics. For the ith load, P{Xi (t + s) = x+ | Xi (t) = x− } ≈ sDζt (x− , x+ ) + O(s2 ) 6 / 28
- 28. Distributed Control of Loads General Model Mean Field Model Aggregate model: N loads running independently, each under the command ζ. Empirical Distributions: µN t (x) = 1 N N i=1 I{Xi (t) = x}, x ∈ X 7 / 28
- 29. Distributed Control of Loads General Model Mean Field Model Aggregate model: N loads running independently, each under the command ζ. Empirical Distributions: µN t (x) = 1 N N i=1 I{Xi (t) = x}, x ∈ X Limiting model: d dt µt(x ) = x∈X µt(x)Dζt (x, x ), yt := x µt(x)U(x) via Law of Large Numbers for martingales 7 / 28
- 30. Distributed Control of Loads General Model Mean Field Model Aggregate model: N loads running independently, each under the command ζ. Empirical Distributions: µN t (x) = 1 N N i=1 I{Xi (t) = x}, x ∈ X Mean-ﬁeld model: d dt µt = µtDζt , yt = µt(U) ζt = ft(µ0, . . . , µt) by design 7 / 28
- 31. Distributed Control of Loads General Model Mean Field Model Mean-ﬁeld model: d dt µt = µtDζt , yt = µt(U) ζt = ft(µ0, . . . , µt) by design yt Control @ Utility Gain N Loads Disturbance to be rejected desired ζt µt+1 = µtPζt yt = µt, U Questions: 1. How to control this complex nonlinear system? CDC 2013 8 / 28
- 32. Distributed Control of Loads General Model Mean Field Model Mean-ﬁeld model: d dt µt = µtDζt , yt = µt(U) ζt = ft(µ0, . . . , µt) by design yt Control @ Utility Gain N Loads Disturbance to be rejected desired ζt µt+1 = µtPζt yt = µt, U Questions: 1. How to control this complex nonlinear system? CDC 2013 2. How to design {Dζ} so that control is easy? 8 / 28
- 33. Distributed Control of Loads General Model Mean Field Model Mean-ﬁeld model: d dt µt = µtDζt , yt = µt(U) ζt = ft(µ0, . . . , µt) by design yt Control @ Utility Gain N Loads Disturbance to be rejected desired ζt µt+1 = µtPζt yt = µt, U Questions: 1. How to control this complex nonlinear system? CDC 2013 2. How to design {Dζ} so that control is easy? focus here 8 / 28
- 34. Design
- 35. Design for Intelligent Loads Mean Field Model Design via “Bellman-Shannon” Optimal Control Goal: Construct a family of generators {Dζ : ζ ∈ R}. 9 / 28
- 36. Design for Intelligent Loads Mean Field Model Design via “Bellman-Shannon” Optimal Control Goal: Construct a family of generators {Dζ : ζ ∈ R}. Design: Consider ﬁrst the ﬁnite-horizon control problem: Choose distribution pζ to maximize 1 T ζEpζ T t=0 U(Xt) − D(pζ p0) D denotes relative entropy. p0 denotes nominal Markovian model. 9 / 28
- 37. Design for Intelligent Loads Mean Field Model Design via “Bellman-Shannon” Optimal Control Goal: Construct a family of generators {Dζ : ζ ∈ R}. Design: Consider ﬁrst the ﬁnite-horizon control problem: Choose distribution pζ to maximize 1 T ζEpζ T t=0 U(Xt) − D(pζ p0) D denotes relative entropy. p0 denotes nominal Markovian model. Explicit solution for ﬁnite T: p∗ ζ(xT 0 ) ∝ exp ζ T t=0 U(xt) dt p0(xT 0 ) 9 / 28
- 38. Design for Intelligent Loads Mean Field Model Design via “Bellman-Shannon” Optimal Control Explicit solution for ﬁnite T: p∗ ζ(xT 0 ) ∝ exp ζ T t=0 U(xt) dt p0(xT 0 ) 9 / 28
- 39. Design for Intelligent Loads Mean Field Model Design via “Bellman-Shannon” Optimal Control Explicit solution for ﬁnite T: p∗ ζ(xT 0 ) ∝ exp ζ T t=0 U(xt) dt p0(xT 0 ) Markovian, but not time-homogeneous. As T → ∞, we obtain generator Dζ 9 / 28
- 40. Design for Intelligent Loads Mean Field Model Design via “Bellman-Shannon” Optimal Control Explicit solution for ﬁnite T: p∗ ζ(xT 0 ) ∝ exp ζ T t=0 U(xt) dt p0(xT 0 ) As T → ∞, we obtain generator Dζ Simple construction via eigenvector problem: 9 / 28
- 41. Design for Intelligent Loads Mean Field Model Design via “Bellman-Shannon” Optimal Control Explicit solution for ﬁnite T: p∗ ζ(xT 0 ) ∝ exp ζ T t=0 U(xt) dt p0(xT 0 ) As T → ∞, we obtain generator Dζ Simple construction via eigenvector problem: Dζ(x, y) = v(y) v(x) ζU(x) − Λ + D(x, y) 9 / 28
- 42. Design for Intelligent Loads Mean Field Model Design via “Bellman-Shannon” Optimal Control Explicit solution for ﬁnite T: p∗ ζ(xT 0 ) ∝ exp ζ T t=0 U(xt) dt p0(xT 0 ) As T → ∞, we obtain generator Dζ Simple construction via eigenvector problem: Dζ(x, y) = v(y) v(x) ζU(x) − Λ + D(x, y) where Dv = Λv, D(x, y) = ζU(x) + D(x, y) Extension/reinterpretation of [Todorov 2007] ⊕ Kontoyiannis & M 200X 9 / 28
- 43. d dt µt = µtDζt yt = µt, U d dt Φt = AΦt + Bζt γt = CΦt Linearized Dynamics
- 44. Linearized Dynamics and Passivity Mean Field Model Linearized Dynamics Mean-ﬁeld model: d dt µt = µtDζt , yt = µt(U) Linear state space model: Φt+1 = AΦt + Bζt γt = CΦt 10 / 28
- 45. Linearized Dynamics and Passivity Mean Field Model Linearized Dynamics Mean-ﬁeld model: d dt µt = µtDζt , yt = µt(U) Linear state space model: Φt+1 = AΦt + Bζt γt = CΦt Interpretations: |ζt| is small, and π denotes invariant measure for D. 10 / 28
- 46. Linearized Dynamics and Passivity Mean Field Model Linearized Dynamics Mean-ﬁeld model: d dt µt = µtDζt , yt = µt(U) Linear state space model: Φt+1 = AΦt + Bζt γt = CΦt Interpretations: |ζt| is small, and π denotes invariant measure for D. • Φt ∈ R|X|, a column vector with Φt(x) ≈ µt(x) − π(x), x ∈ X 10 / 28
- 47. Linearized Dynamics and Passivity Mean Field Model Linearized Dynamics Mean-ﬁeld model: d dt µt = µtDζt , yt = µt(U) Linear state space model: Φt+1 = AΦt + Bζt γt = CΦt Interpretations: |ζt| is small, and π denotes invariant measure for D. • Φt ∈ R|X|, a column vector with Φt(x) ≈ µt(x) − π(x), x ∈ X • γt ≈ yt − y0; deviation from nominal steady-state 10 / 28
- 48. Linearized Dynamics and Passivity Mean Field Model Linearized Dynamics Mean-ﬁeld model: d dt µt = µtDζt , yt = µt(U) Linear state space model: Φt+1 = AΦt + Bζt γt = CΦt Interpretations: |ζt| is small, and π denotes invariant measure for D. • Φt ∈ R|X|, a column vector with Φt(x) ≈ µt(x) − π(x), x ∈ X • γt ≈ yt − y0; deviation from nominal steady-state • A = DT , Ci = U(xi), and input dynamics linearized: 10 / 28
- 49. Linearized Dynamics and Passivity Mean Field Model Linearized Dynamics Mean-ﬁeld model: d dt µt = µtDζt , yt = µt(U) Linear state space model: Φt+1 = AΦt + Bζt γt = CΦt Interpretations: |ζt| is small, and π denotes invariant measure for D. • Φt ∈ R|X|, a column vector with Φt(x) ≈ µt(x) − π(x), x ∈ X • γt ≈ yt − y0; deviation from nominal steady-state • A = DT , Ci = U(xi), and input dynamics linearized: BT = d dζ πDζ ζ=0 10 / 28
- 50. Linearized Dynamics and Passivity Linearized Dynamics Transfer Function Linear state space model: Φt+1 = AΦt + Bζt γt = CΦt A = DT , Ci = U(xi ), BT = d dζ πDζ ζ=0 11 / 28
- 51. Linearized Dynamics and Passivity Linearized Dynamics Transfer Function Linear state space model: Φt+1 = AΦt + Bζt γt = CΦt A = DT , Ci = U(xi ), BT = d dζ πDζ ζ=0 Transfer Function: G(s) = C[Is − A]−1 B = C[Is − DT ]−1 B hmmmm... 11 / 28
- 52. Linearized Dynamics and Passivity Linearized Dynamics Transfer Function Linear state space model: Φt+1 = AΦt + Bζt γt = CΦt A = DT , Ci = U(xi ), BT = d dζ πDζ ζ=0 Transfer Function: G(s) = C[Is − A]−1 B = C[Is − DT ]−1 B hmmmm... Resolvent Matrix: Rs = ∞ 0 e−st etD dt = [Is − D]−1 11 / 28
- 53. Linearized Dynamics and Passivity Linearized Dynamics Transfer Function Linear state space model: Φt+1 = AΦt + Bζt γt = CΦt A = DT , Ci = U(xi ), BT = d dζ πDζ ζ=0 Transfer Function: G(s) = C[Is − A]−1 B = C[Is − DT ]−1 B hmmmm... = CRT sB TF for L-MFM Resolvent for one load Resolvent Matrix: Rs = ∞ 0 e−st etD dt = [Is − D]−1 11 / 28
- 54. Linearized Dynamics and Passivity Linearized Dynamics Passive Pools Theorem: Reversibility =⇒ Passivity 12 / 28
- 55. Linearized Dynamics and Passivity Linearized Dynamics Passive Pools Theorem: Reversibility =⇒ Passivity Suppose that the nominal model is reversible. Then its linearization satisﬁes, Re G(jω) = PSDY (ω), ω ∈ R , where G(s) = C[Is − A]−1 B for s ∈ C. PSDY (ω) = ∞ −∞ e−jω Eπ[U(X0)U(Xt)] dt 12 / 28
- 56. Linearized Dynamics and Passivity Linearized Dynamics Passive Pools Theorem: Reversibility =⇒ Passivity Suppose that the nominal model is reversible. Then its linearization satisﬁes, Re G(jω) = PSDY (ω), ω ∈ R , where G(s) = C[Is − A]−1 B for s ∈ C. PSDY (ω) = ∞ −∞ e−jω Eπ[U(X0)U(Xt)] dt Implication for control: G(s) is positive real 12 / 28
- 57. Linearized Dynamics and Passivity Linearized Dynamics Example Without Passivity Example: Eight state model Utility function U(xi) = i. 1 2 3 4 5 6 7 8 a c c a a a a a a a b b b b Not reversible 13 / 28
- 58. Linearized Dynamics and Passivity Linearized Dynamics Example Without Passivity Example: Eight state model Utility function U(xi) = i. Generator is the 8 × 8 matrix, D = −a a 0 0 0 0 0 0 a −(a + b + c) 0 b c 0 0 0 b 0 −(a + b) a 0 0 0 0 0 0 a −a 0 0 0 0 0 0 0 0 −a a 0 0 0 0 0 0 a −(a + b) 0 b 0 0 0 c b 0 −(a + b + c) a 0 0 0 0 0 0 a −a 13 / 28
- 59. Linearized Dynamics and Passivity Linearized Dynamics Example Without Passivity 1 2 3 4 5 6 7 8 a c c a a a a a a a b b b b Example: Eight state model a = c = 10, b = 1 −2 −1 0 1 2 Real Axis −1 0 1 2 ImaginaryAxis −25 −20 −15 −10 −5 0 5 10 −15 −10 −5 0 5 10 15 Real Part ImaginaryPart Pole-zero Plot G(jω), ω >0 Non-minimum phase zero Nyquist Plot G(s) = C[Is − A]−1B not positive real 13 / 28
- 60. yt Control @ Utility Gain One Million Pools Disturbance to be rejected Proportion of pools on desired ζt µt+1 = µtPζt yt = µt, U One Million Pools
- 61. One Million Pools A Single Pool Control Architecture Transition diagram for a single pool: 1 2 T−1 T . . . T On Off 12T−1 ... Numerics using discrete-time model 14 / 28
- 62. One Million Pools A Single Pool Control Architecture 1 2 T−1 T . . . T On Off 12T−1 ... Utility: U(x) = I Pool is on x ∝ (Power consumption) x 14 / 28
- 63. One Million Pools A Single Pool Control Architecture 1 2 T−1 T . . . T On Off 12T−1 ... Utility: U(x) = I Pool is on x ∝ (Power consumption) x Controlled dynamics: As ζ increases, probability of turning on increases: 0 2412 0 0.5 1 zζ =-4 ζ=-2 ζ = 4 ζ = 2 ζ = 0 (ζ) 14 / 28
- 64. One Million Pools Mean Field Pool Model Linearization: Minimum Phase Pole-Zero Plot −1 0 1 −1 0 1 Real PartImaginaryPart Transfer function Normalized Frequency (×π rad/sample)1/24 hrs 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 -180 0 20 Phase(degrees)Magnitude(dB) 40 -90 0 15 / 28
- 65. One Million Pools Mean Field Pool Model Stochastic simulation — Filtered regulation signal −400 −200 0 200 400 MW July 18 July 20July 19 July 21 July 23July 22 July 24 BPA Balancing Reserves Deployed (July 2013) ∗transmission.bpa.gov/Business/Operations/Wind/reserves.aspx 16 / 28
- 66. One Million Pools Mean Field Pool Model Stochastic simulation — Filtered regulation signal −400 −200 0 200 400 MW July 18 July 20July 19 July 21 July 23July 22 July 24 Filtered BPA reserve signal that can be tracked by pools BPA Balancing Reserves Deployed (July 2013) ∗transmission.bpa.gov/Business/Operations/Wind/reserves.aspx 16 / 28
- 67. One Million Pools Mean Field Pool Model Stochastic simulation using N = 105 pools PI control Reference (from Bonneville Power Authority) −300 −200 −100 0 100 200 300 TrackingBPARegulationSignal (MW) 17 / 28
- 68. One Million Pools Mean Field Pool Model Stochastic simulation using N = 105 pools PI control Reference (from Bonneville Power Authority) Output deviation y −300 −200 −100 0 100 200 300 TrackingBPARegulationSignal (MW) 17 / 28
- 69. One Million Pools Mean Field Pool Model Stochastic simulation using N = 105 pools PI control Reference (from Bonneville Power Authority) Output deviation y −300 −200 −100 0 100 200 300 -2 -1 -3 2 3 0 1 Input ζ ζ TrackingBPARegulationSignal (MW) 17 / 28
- 70. One Million Pools Mean Field Pool Model Stochastic simulation using N = 105 pools PI control Two scenarios, using two diﬀerent reference signals: 0 20 40 60 80 100 120 140 160 t/hour 0 20 40 60 80 100 120 140 160 0 20 40 60 80 100 120 140 160 t/hour 0 20 40 60 80 100 120 140 160 ReferenceOutputdeviation(MW) ReferenceOutputdeviation(MW) ReferenceOutputdeviation(MW) ReferenceOutputdeviation(MW) −300 −200 −100 0 100 200 300 −300 −200 −100 0 100 200 300 −600 −400 −200 0 200 400 600 −400 −200 0 200 400 600 800 -2 -1 -3 2 3 0 1 −3 −2 −1 0 1 2 3 −5 0 5 −4 −2 0 2 4 6 Input Input Input Input 12 Hour Cleaning Cycle 8 Hour Cleaning Cycle TrackingBPARegulationSignalBPARegulationSignal-Doubled (a) (b) (c) (d) 18 / 28
- 71. One Million Pools Mean Field Pool Model Stochastic simulation using N = 105 pools PI control The impact of exceeding capacity 0 20 40 60 80 100 120 140 1600 20 40 60 80 100 120 140 160 ReferenceOutputdeviation(MW) ReferenceOutputdeviation(MW) ReferenceOutputdeviation(MW) ReferenceOutputdeviation(MW) 0 20 40 60 80 100 120 140 160 t/hour 0 20 40 60 80 100 120 140 160 −1000 −500 0 500 1000 −1000 −500 0 500 1000 −1000 −500 0 500 1000 −1000 −500 0 500 1000 −20 −10 0 10 20 −20 −10 0 10 20 −10 0 10 −10 0 10 Input Input Input Input 12 Hour Cleaning Cycle 8 Hour Cleaning Cycle RegulationexceedscapacityIntegratorwind-upRecovery (a) (b) (c) (d) 19 / 28
- 72. Power GridControl WaterPump Batteries Coal GasTurbine BP BP BP C BP BP Voltage Frequency Phase HC Σ − Actuator feedback loop A LOAD Conclusions and Extensions
- 73. Conclusions and Extensions Conclusions Recap QUIZ: Why intelligence at the loads? 20 / 28
- 74. Conclusions and Extensions Conclusions Recap QUIZ: Why intelligence at the loads? To simplify control at the grid level 20 / 28
- 75. Conclusions and Extensions Conclusions Recap QUIZ: Why intelligence at the loads? To simplify control at the grid level A particular approach to distributed control is proposed 20 / 28
- 76. Conclusions and Extensions Conclusions Recap QUIZ: Why intelligence at the loads? To simplify control at the grid level A particular approach to distributed control is proposed The grid level control problem is simple because: Mean ﬁeld model is simple, and a good approximation of ﬁnite system 20 / 28
- 77. Conclusions and Extensions Conclusions Recap QUIZ: Why intelligence at the loads? To simplify control at the grid level A particular approach to distributed control is proposed The grid level control problem is simple because: Mean ﬁeld model is simple, and a good approximation of ﬁnite system LTI approximation is positive real, which implies minimum phase 20 / 28
- 78. Conclusions and Extensions Conclusions Are the customers happy? No time for details, but no ... 21 / 28
- 79. Conclusions and Extensions Conclusions Are the customers happy? No time for details, but no ... QoS without local control −100 −80 −60 −40 −20 0 20 40 60 80 100 0 0.01 0.02 0.03 0.04 0.05 fractionofpools Histogram of pool health Gaussian distribution 21 / 28
- 80. Conclusions and Extensions Conclusions Are the customers happy? No time for details, but no ... QoS without local control −100 −80 −60 −40 −20 0 20 40 60 80 100 0 0.01 0.02 0.03 0.04 0.05fractionofpools Histogram of pool health Gaussian distribution There will be “rare events” in which the pool is under- or over-cleaned. 21 / 28
- 81. Conclusions and Extensions Conclusions Are the customers happy? No time for details, but no ... QoS without local control −100 −80 −60 −40 −20 0 20 40 60 80 100 0 0.01 0.02 0.03 0.04 0.05fractionofpools Histogram of pool health Gaussian distribution There will be “rare events” in which the pool is under- or over-cleaned. Proposed approach: Additional layer of control at each load, so that hard constraints on performance can be assured. 21 / 28
- 82. Conclusions and Extensions Conclusions Are the customers happy? No time for details, but no ... QoS without local control −100 −80 −60 −40 −20 0 20 40 60 80 100 0 0.01 0.02 0.03 0.04 0.05fractionofpools Histogram of pool health Gaussian distribution There will be “rare events” in which the pool is under- or over-cleaned. Proposed approach: Additional layer of control at each load, so that hard constraints on performance can be assured. This will create some modeling error at grid level. Preliminary experiments on the pool model: No loss of performance at grid level. 21 / 28
- 83. Conclusions and Extensions Conclusions The customers are happy Proposed approach: Additional layer of control at each load, so that hard constraints on performance can be assured. QoS without local control −100 −80 −60 −40 −20 0 20 40 60 80 100 0 0.01 0.02 0.03 0.04 0.05 fractionofpools Histogram of pool health Gaussian distribution 22 / 28
- 84. Conclusions and Extensions Conclusions The customers are happy Proposed approach: Additional layer of control at each load, so that hard constraints on performance can be assured. QoS without local control QoS with local control −100 −80 −60 −40 −20 0 20 40 60 80 100 0 0.01 0.02 0.03 0.04 0.05 fractionofpools Histogram of pool health Conditional Gaussian distribution Histogram of pool health Gaussian distribution −100 −80 −60 −40 −20 0 20 40 60 80 100 Load opts-out when its QoS is outside of prescribed bounds Analysis: LQG approximation for individual load – CDC 2014 22 / 28
- 85. Conclusions and Extensions Conclusions Many issues skipped because they are topics of current research 1 Information at the load: Should these loads simply act as frequency regulators? Measurement of local frequency deviation will suﬃce. 23 / 28
- 86. Conclusions and Extensions Conclusions Many issues skipped because they are topics of current research 1 Information at the load: Should these loads simply act as frequency regulators? Measurement of local frequency deviation will suﬃce. Or, more information may be valuable: Two measurements at each load, the BA command, and local frequency measurements. 23 / 28
- 87. Conclusions and Extensions Conclusions Many issues skipped because they are topics of current research 1 Information at the load: Should these loads simply act as frequency regulators? Measurement of local frequency deviation will suﬃce. Or, more information may be valuable: Two measurements at each load, the BA command, and local frequency measurements. 2 Information at the BA: Does this macro control view suﬃce? Power Grid Actuation Control WaterPump Batteries HVAC Coal GasTurbine BP BP BP BP BP Baseline Generation Disturbances from nature Measurements: Voltage Frequency Phase HC Σ − 23 / 28
- 88. Conclusions and Extensions Conclusions Many issues skipped because they are topics of current research 1 Information at the load: Should these loads simply act as frequency regulators? Measurement of local frequency deviation will suﬃce. Or, more information may be valuable: Two measurements at each load, the BA command, and local frequency measurements. 2 Information at the BA: Does this macro control view suﬃce? Power Grid Actuation Control WaterPump Batteries HVAC Coal GasTurbine BP BP BP BP BP Baseline Generation Disturbances from nature Measurements: Voltage Frequency Phase HC Σ − Does the grid operator need to know the real-time power consumption of each population of loads? 23 / 28
- 89. Conclusions and Extensions Conclusions Many issues skipped because they are topics of current research 3 How can we engage consumers? 24 / 28
- 90. Conclusions and Extensions Conclusions Many issues skipped because they are topics of current research 3 How can we engage consumers? The formulation of contracts with customers requires a better understanding of the value of ancillary service, as well as consumer preferences. 24 / 28
- 91. Conclusions and Extensions Conclusions Many issues skipped because they are topics of current research 3 How can we engage consumers? The formulation of contracts with customers requires a better understanding of the value of ancillary service, as well as consumer preferences. We will not use real time prices, even via aggregator, if we want responsive, reliable ancillary service ReferenceOutputdeviation(MW) −300 −200 −100 0 100 200 300 -2 -1 -3 2 3 0 1 Input TrackingBPARegulationSignal (a) 24 / 28
- 92. Conclusions and Extensions Conclusions Thank You! 25 / 28
- 93. Control Techniques FOR Complex Networks Sean Meyn Pre-publication version for on-line viewing. Monograph available for purchase at your favorite retailer More information available at http://www.cambridge.org/us/catalogue/catalogue.asp?isbn=9780521884419 Markov Chains and Stochastic Stability S. P. Meyn and R. L. Tweedie August 2008 Pre-publication version for on-line viewing. Monograph to appear Februrary 2009 π(f)<∞ ∆V (x) ≤ −f(x) + bIC(x) Pn (x, · ) − π f → 0 sup C Ex[SτC(f)]<∞ References
- 94. References References: Demand Response S. Meyn, P. Barooah, A. Buˇsi´c, and J. Ehren. Ancillary service to the grid from deferrable loads: the case for intelligent pool pumps in Florida (Invited). In Proceedings of the 52nd IEEE Conf. on Decision and Control, 2013. A. Buˇsi´c and S. Meyn. Passive dynamics in mean ﬁeld control. ArXiv e-prints: arXiv:1402.4618. Submitted to the 53rd IEEE Conf. on Decision and Control (Invited), 2014. S. Meyn, Y. Chen, and A. Buˇsi´c. Individual risk in mean-ﬁeld control models for decentralized control, with application to automated demand response. Submitted to the 53rd IEEE Conf. on Decision and Control (Invited), 2014. J. L. Mathieu. Modeling, Analysis, and Control of Demand Response Resources. PhD thesis, Berkeley, 2012. D. Callaway and I. Hiskens, Achieving controllability of electric loads. Proceedings of the IEEE, 99(1):184–199, 2011. S. Koch, J. Mathieu, and D. Callaway, Modeling and control of aggregated heterogeneous thermostatically controlled loads for ancillary services, in Proc. PSCC, 2011, 1–7. H. Hao, A. Kowli, Y. Lin, P. Barooah, and S. Meyn Ancillary Service for the Grid Via Control of Commercial Building HVAC Systems. ACC 2013 (much more on our websites) 26 / 28
- 95. References References: Markov Models I. Kontoyiannis and S. P. Meyn. Spectral theory and limit theorems for geometrically ergodic Markov processes. Ann. Appl. Probab., 13:304–362, 2003. I. Kontoyiannis and S. P. Meyn. Large deviations asymptotics and the spectral theory of multiplicatively regular Markov processes. Electron. J. Probab., 10(3):61–123 (electronic), 2005. E. Todorov. Linearly-solvable Markov decision problems. In B. Sch¨olkopf, J. Platt, and T. Hoﬀman, editors, Advances in Neural Information Processing Systems, (19) 1369–1376. MIT Press, Cambridge, MA, 2007. M. Huang, P. E. Caines, and R. P. Malhame. Large-population cost-coupled LQG problems with nonuniform agents: Individual-mass behavior and decentralized ε-Nash equilibria. IEEE Trans. Automat. Control, 52(9):1560–1571, 2007. H. Yin, P. Mehta, S. Meyn, and U. Shanbhag. Synchronization of coupled oscillators is a game. IEEE Transactions on Automatic Control, 57(4):920–935, 2012. P. Guan, M. Raginsky, and R. Willett. Online Markov decision processes with Kullback-Leibler control cost. In American Control Conference (ACC), 2012, 1388–1393, 2012. V.S.Borkar and R.Sundaresan Asympotics of the invariant measure in mean ﬁeld models with jumps. Stochastic Systems, 2(2):322-380, 2012. 27 / 28
- 96. References References: Economics G. Wang, M. Negrete-Pincetic, A. Kowli, E. Shaﬁeepoorfard, S. Meyn, and U. Shanbhag. Dynamic competitive equilibria in electricity markets. In A. Chakrabortty and M. Illic, editors, Control and Optimization Theory for Electric Smart Grids. Springer, 2011. G. Wang, A. Kowli, M. Negrete-Pincetic, E. Shaﬁeepoorfard, and S. Meyn. A control theorist’s perspective on dynamic competitive equilibria in electricity markets. In Proc. 18th World Congress of the International Federation of Automatic Control (IFAC), Milano, Italy, 2011. S. Meyn, M. Negrete-Pincetic, G. Wang, A. Kowli, and E. Shaﬁeepoorfard. The value of volatile resources in electricity markets. In Proc. of the 10th IEEE Conf. on Dec. and Control, Atlanta, GA, 2010. I.-K. Cho and S. P. Meyn. Eﬃciency and marginal cost pricing in dynamic competitive markets with friction. Theoretical Economics, 5(2):215–239, 2010. U.S. Energy Information Administration. Smart grid legislative and regulatory policies and case studies. December 12 2011. http://www.eia.gov/analysis/studies/electricity/pdf/smartggrid.pdf 28 / 28