National Renewable Energy Laboratory Postdoctoral Researcher: Computational Applied Mathematician for Reinforcement Learning Control and Optimization of Energy Systems in Golden, Colorado
Postdoctoral Researcher: Computational Applied Mathematician for Reinforcement Learning Control and Optimization of Energy Systems
CO - Golden
Postdoc (Fixed Term)
Hours Per Week
The Complex Systems Simulation and Optimization Group in the NREL Computational Science Center has an opening for a full-time Postdoctoral Researcher - Computational Applied Mathematician for Reinforcement Learning Control and Optimization of Energy Systems. Artificial intelligence (AI) is rapidly becoming ubiquitous in renewable energy research. In particular, deep reinforcement learning (RL) for optimization and control of energy systems has the potential to transform our renewable energy future through advanced automation and control. We are looking for a dynamic researcher with a strong technical background to help pursue this goal. Stochastic optimization and sequential decision problems arise in a variety of areas of renewable energy research including power system planning and management, wind farm optimization, battery management, building energy management, transportation, even molecular design. At the same time, recent advances in RL suggest that complex decision making can be optimized via a combination of machine learning, modeling, and simulation, often with clear advantages over traditional optimization-based methods. Our initial efforts along these lines have opened the door to larger, sustained research programs in which an applied mathematician capable of exploring the deep underpinnings of designing and applying RL and multi-agent RL algorithms specifically tailored to NREL problems is necessary.
The successful candidate will collaborate to develop, adapt, improve, and scale cutting edge RL methods to real world projects in support of NREL and EERE mission and goals. The candidate will be tasked with core understanding of the optimal formulation of these algorithms. How deep RL approaches can be utilized in multi-agent control and optimization problems? How is existing domain knowledge to be utilized? How is complex parallel simulation code best incorporated into a scalable RL algorithm? What is the best way to combine traditional optimization and control algorithms and RL? We are looking for a researcher can to construct innovative RL algorithms from the ground up while appreciating domain-specific considerations. We are looking for a researcher who is able and excited to design novel RL approaches and bring the power of RL to pressing real-world
problems (not just demonstrations and games).
This Postdoc will collaborate with NREL and partner researchers to develop and apply RL for use in real-world energy systems. Primary challenges include identification of appropriate application areas, development and adaptation of methods for these areas, and scaling of methods to solve relevant problems.
Key areas of study include:
Model-based RL. There are a variety of efforts to use models in RL, but this is not a settled research area. We believe a combination of “top-down” incorporation of existing domain knowledge into “bottom-up” RL methods holds great promise, but this approach is not well developed.
Scalable RL. Supercomputers in principle allow for RL on many thousands of CPUs and hundreds of GPUs, yet existing algorithms and implementations do not yet take advantage of them. The successful candidate will participate in the development of scalable RL algorithms, especially for continuous and/or mixed spaces and that involve integration with complex parallel simulation code.
Hybrid methods. Classical optimization-based control such as Model Predictive Control (MPC) has many benefits, including interpretability and convergence guarantees, but (especially in real-time or for nonlinear systems) can become intractable. Methods that combine the best of both AI and classical stochastic optimization, e.g. that combine MPC with RL, are of great interest.
Multi-agent RL. Distributed optimization and control problems arise naturally in many clean energy applications. Methods that can address challenges in multi-agent RL (such as non-stationarity and credit assignment) and, more generally, enable distributed training and control, must be better understood in the context of clean energy applications.
Safe RL. For safety critical systems, there is well founded skepticism around using model-free RL as the primary controller due to challenges in handling complex system constraints and guaranteeing feasible control trajectories under uncertainty. Constraint aware, “safe” RL thus constitutes an important building block for bringing RL to bear on many real-world problems.
Evaluate and track the state of reinforcement learning research, especially for sequential decision problems in complex real-world spaces.
Author, present and assist in the preparation of technical papers, reports and conference proceedings on topics related to modern AI control methods and their application to energy systems.
Must be a recent PhD graduate within the last three years.
Additional Required Qualifications
Expertise in deep reinforcement learning, including fundamental knowledge and core mathematical underpinnings of neural networks, deep understanding of Markov Decision Processes, state-of-the-art learning algorithms, and their application to continuous and/or large state-action spaces.
Expertise in stochastic optimal control, including optimization and control with and without uncertainty, dynamic programming, classical control theory, (stochastic) model predictive control, Kalman filters, scenario generation, etc.
Experience with game theory, complex systems theory, agent-based learning, and distributed optimization.
Strong background in mathematics, statistics, and probability.
Strong background in physics and engineering, esp. of power systems, grid-integrated buildings, and/or transportation systems.
Strong programming skills, especially experience using deep learning frameworks (e.g. TensorFlow) on high performance computing/GPU platforms. Sample code from previous related projects/courses is a plus.
Annual Salary Range (based on full-time 40 hours per week)
Annual Salary Range: $68,000 - $112,200
Offers will typically be made in the bottom half of the listed range. NREL takes into consideration a candidate’s education, training, and experience, as well as the position's work location, expected quality and quantity of work, required travel (if any), external market and internal value, including seniority and merit systems, and internal pay alignment when determining the salary level for potential new employees. In compliance with the Colorado Equal Pay for Equal Work Act, a potential new employee’s salary history will not be used in compensation decisions.
Benefits include medical, dental, and vision insurance; short-term disability insurance; pension benefits; 403(b) Employee Savings Plan with employer match*; life and accidental death and dismemberment (AD&D) insurance; personal time off (PTO) and sick leave; and paid holidays. NREL employees may be eligible for, but are not guaranteed, performance-, merit-, and achievement- based awards that include a monetary component. Some positions may be eligible for relocation expense reimbursement.
* Based on eligibility rules
Please note that in order to be considered an applicant for any position at NREL you must submit an application form for each position for which you believe you are qualified. Applications are not kept on file for future positions. Please include a cover letter and resume with each position application.
NREL is an Equal Opportunity/Affirmative Action Employer. All qualified applicants will receive consideration for employment without regard basis of age (40 and over), color, disability, gender identity, genetic information, marital status, military or veteran status, national origin/ancestry, race, religion, creed, sex (including pregnancy, childbirth, breastfeeding), sexual orientation, and any other applicable status protected by federal, state, or local laws.
EEO is the Law (http://www.dol.gov/ofccp/regs/compliance/posters/ofccpost.htm) | Pay Transparency Nondiscrimination (https://www.dol.gov/ofccp/pdf/pay-transp_English_unformattedESQA508c.pdf) | Reasonable Accommodations (http://www.nrel.gov/careers/employment-policies.html)
E -Verify www.dhs.gov/E-Verify |For information about right to work, click here (http://www.justice.gov/sites/default/files/crt/legacy/2013/08/13/FinalOSCPosterEN08_01_2013.pdf) for English or here (http://www.justice.gov/crt/file/813271/download) for Spanish.
E-Verify is a registered trademark of the U.S. Department of Homeland Security. This business uses E-Verify in its hiring practices to achieve a lawful workforce.
The National Renewable Energy Laboratory (NREL) is a leader in the U.S. Department of Energy’s effort to secure an environmentally and economically sustainable energy future. With locations in Golden and Boulder, Colorado, and a satellite office in Washington, D.C., NREL is the primary laboratory for research, development, and deployment of renewable energy technologies in the United States.
NREL is subject to Department of Energy (DOE) access restrictions. All candidates must be authorized to access the facility per DOE rules and guidance within a reasonable time frame for the specified position in order to be considered for an interview. DOE rules for site access during the interview process are the same regardless of whether the candidate is interviewed on-site, off-site, or via telephone or videoconference. Additionally, DOE contractor employees are prohibited from participating in certain Foreign Government Talent Recruitment Programs (FGTRPs). If a candidate is currently participating in an FGTRP, they will be required to disclose their participation after receiving an offer of employment and may be required to disengage from participation in the FGTRP prior to commencing employment. Any offer of employment is conditional on the ability to obtain work authorization and to be granted access to NREL by the Department of Energy (DOE). We understand that COVID-19 may have caused delays or closures at offices, consulates, and embassies. However, NREL cannot make exceptions to work authorization and access requirements, and exceptions to these requirements are not being made for COVID-19 related delays.
Please review the information on our Hiring Process (https://www.nrel.gov/careers/hiring-process.html) website before you create an account and apply for a job. We also hope you will learn more about NREL (https://www.nrel.gov/about/) , visit our Careers site (https://www.nrel.gov/careers/) , and continue to search for job opportunities (https://nrel.wd5.myworkdayjobs.com/NREL) at the lab.