Research

Guaranteed Tubes for Safe Learning across Autonomy Architectures (NSEF-SLES)

As learning-enabled systems grow rapidly in complexity and prevalence, the safety of such systems has become a paramount concern. In response to this critical challenge, a novel solution, Data-enabled Simplex (DeSimplex) architecture, is introduced. Grounded in rigorous mathematical principles and systematic methodologies for data collection, DeSimplex provides a verifiably safe framework that enables learning-enabled systems to adapt and perform effectively amidst unforeseen changes in system dynamics, extreme events, environmental hazards, and irregular system behaviors. Its core principle lies in transferring control authority from high-performance learning-based components to high-assurance solutions whenever safety boundaries are at risk of being violated, thus ensuring end-to-end safety. Such a switch also allows learning components to actively gather new data and insights from the operating environments, perpetually refining their models to optimize both performance and safety. Two interconnected research thrusts will be pursued: i) improving high-performance autonomy with reliable uncertainty quantification methods to ensure data-driven adaptability and precise measurement of uncertainties; ii) developing high-assurance autonomy architectures and establishing switching rules for verifiable observability and controllability. The Principal Investigators (PIs) are ideally prepared for this endeavor, offering a wide range of technical expertise, extensive practical experience and technology transitions, a track record of successful collaborations, and numerous significant research contributions in relevant fields.

A Training-Testing-Benchmarking Environment for Learning-Enabled Control Frameworks and Algorithms (AFOSR-DURIP)

The long-term goal of the proposed equipment is to promote breakthroughs in learning-enabled control (LEC) frameworks and algorithms that can safely and reliably run on physical systems. The results can be applied to the domains with a high level of autonomy yet with dynamically varying environments, e.g., autonomous driving and urban air mobility. This testbed is fundamental for performing experiments that will verify the safety and reliability of already existing LEC frameworks and algorithms as well as those that are still in development. When fully operational, the proposed environment will allow fast iterations between training, testing, and benchmarking, with sufficient computational powers enabled by the GPU server. More importantly, the extensive training and testing within the proposed environment will refine the workflow, generate software tools, and enhance the quality and performance of LEC frameworks and algorithms when deployed to physical systems. Last but not least, the availability of such an environment will significantly benefit research efforts in (cooperative) trajectory planning, decentralized collaboration and coordination of heterogeneous multiagent systems, L1 adaptive augmentation for autonomous aerial/ground vehicles, and other relevant state-of-the-art research projects at UIUC.

Autonomous Vehicles in Air Transportation Engineering (NASA-ULI)

The objective of this research is to develop a Robust and Resilient Autonomy framework with principled and verifiable architectures to deal with uncertainties and off-nominal situations to enable safe and efficient Advanced Air Mobility. Research themes include learning-enabled control and autonomy, verification and validation, fault detection, and systems integration.

Our proposal, “Robust Resilient Autonomy for Advanced Air Mobility,” has been selected by NASA for their 5th round of ULI funding: https://www.nasa.gov/press-release/nasa-selects-four-university-teams-for-aviation-projects/.

Check more details on our website: https://aviate.illinois.edu/.

Safe Reinforcement Learning for Robotic Systems in Non-Stationary Environments (NSF-AoF)

Reinforcement learning (RL) has shown impressive performance in controlling complex robotic systems for various tasks such as locomotion, manipulation, and playing sports, e.g., table tennis. Reinforcement learning enables a robot to autonomously discover an optimal behavior through trial-and-error interactions with its environment. However, environmental perturbations could easily cause a behavior policy trained in an old environment to fail in a perturbed environment. The failure is unacceptable for safety-critical robotic systems such as self-driving cars, drones, flying taxis, and construction machines. Existing robust methods try to consider all scenarios during the training phase and seek a fixed policy, leading to conservative behaviors. Existing adaptive methods try to update their behavior policies in the perturbed environment. Still, they will only do that after the robot has “felt a difference” through its interaction with the environment. In contrast, humans could leverage their perception for prediction in the new environment and adjust their behavior even before interacting. In light of these conditions, this project envisions a new framework for safe and efficient RL in the presence of environmental changes leveraging fast adaptation and perception-based prediction. The framework will enable robotic and autonomous systems robustly and safely operate, learn and adapt in the real world.
[Slides

Publications

[1] Y. Cheng, P. Zhao, F. Wang, D. J. Block, and N. Hovakimyan. Improving the robustness of reinforcement learning policies with L1 adaptive control. IEEE Robotics and Automation Letters, in press, 2022.
[2] Y. Cheng, P. Zhao, M. Gandhi, B. Li, E. Theodorou and N. Hovakimyan. Robustifying reinforcement learning policies with L1 adaptive control, ICRA Workshop on Safe Robot Control with Learned Motion and Environment Models, 2021.
[3] P. Zhao, Z. Guo, A. Gahlawat, H. Kang and N. Hovakimyan. Guaranteed nonlinear tracking in the presence of DNN-learned dynamics with contraction metrics and disturbance estimation. arXiv preprint arXiv:2112.08222, 2022.

AI for Agricultural Management and Soil Carbon Sequestration (C3.AI)

Soil carbon sequestration in croplands has tremendous potential to help mitigate climate change; however, it is challenging to develop the optimal management practices for maximization of the sequestered carbon as well as the crop yield. This project aims to develop an intelligent agricultural management system using deep reinforcement learning (RL) and large-scale soil and crop simulations. To achieve this, we propose to build a simulator to model and simulate the complex soil-water-plant-atmosphere interaction, which will run on high-performance computing platforms. Massive simulations using such platforms allow the evaluation of the effects of various management practices under different weather and soil conditions in a timely and cost-effective manner. By formulating the management decision as an RL problem, we can leverage the state-of-the-art algorithms to train management policies through extensive interactions with the simulated environment. The trained policies are expected to maximize the stored organic carbon while maximizing the crop yield in the presence of uncertain weather conditions. The whole system will be tested using data of soil and crops in both mid-west of the United States and the central region of Portugal. The proposed research has great potential for impact on climate change and food security, two of the most significant challenges currently facing humanity.
[Slides
Acknowledgment: Funded by C3.ai Digital Transformation Institute (C3DTI).
Publications:

[1] J. Wu, R. Tao, P. Zhao, N. Martin and N. Hovakimyan. Optimizing nitrogen management with deep reinforcement learning and crop simulations. Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp.1712–1720, 2022.
[2] R. Tao, P. Zhao, J. Wu, et al. Optimizing Crop management with reinforcement learning and
imitation learning. arXiv preprint arXiv:2209.09991.

Foundations of Social Information Dynamics and Control (DOD)

Science made significant progress in modeling myriads of physical propagation phenomena. Advances in human understanding of physical propagation and interaction effects in different domains led to such results as Newton's laws of motion, Maxwell's equations of electrodynamics, Boltzmann's law of thermodynamics, Schrödinger's equation of quantum mechanics, Einstein's field equations, Planck's law of radiation, and many others. Yet, the manner in which information propagates in social spaces and the effects that such propagation has on those exposed to the information signals remain poorly understood. It is time to rectify this shortcoming by establishing laws of information dynamics as well as the corresponding social mathematics that describe information's effects on social systems.

Acoustically Aware Control Laws (NASA)

This project covers tasks that contribute to building a framework for acoustically-aware vehicles seeking to minimize noise signatures through a hierarchical control mechanism. The framework is developed for the multi-propulsor vehicle configurations that are emerging in Distributed Electric Propulsion, eVTOL and hybrid aircraft designed to operate in urban environments. The proposed task is focused on the objective of developing trajectory generation methods and trajectory tracking control laws to simultaneously meet mission objectives and community noise reduction requirements via propeller phase synchronization. This task lays the groundwork for potential future flight tests on a NASA selected vehicle.

Machine Learning Methods for Estimating Propeller Source Noise Spheres

Acoustic estimation is needed to simulate the effect of vehicle motion on the acoustic profile. Typically, simulating the noise created by vehicle propellers requires significant computational effort and is not suitable for online applications. Instead, we consider using a set of pre-generated (or measured) acoustic sources to estimate the acoustic source in a different flight condition. We use a neural network that learns the model parameters from the given acoustic sources and can predict the acoustics at new flight conditions fast enough to use online.
The given acoustic sources describe the tonal acoustic pressure emitted in each direction by the propeller. As the vehicle moves, the flight conditions change, and the amplitude and directivity of the acoustic pressure change. We consider changes in propeller rotation rate, freestream speed, and freestream angle-of-attack in our flight condition model, with acoustic sources given at 200 flight conditions. Each acoustic source is parameterized in terms of amplitude and phase. By accurately predicting these quantities, we can compute the acoustic pressure generated by the propeller in a specific direction at a given time.

Controller Design for Propeller Phase Synchronization with Aeroacoustic Performance Metrics

[Article
Active noise reduction using phase control takes advantage of the propellers used in a distributed electric air vehicle by treating each propeller as an independent acoustic source. These acoustic sources can destructively interfere if the propellers are synchronized. With this method, the radiated sound power around the vehicle can be reduced. The purpose of this work is to design a controller that regulates propeller positions to reduce the radiated sound power through destructive interference and to demonstrate the performance of the controller through acoustic testing.

In this scenario, the propeller phase angles smoothly transition between no offset and some desired offset, as shown in Figure a. This transition occurs by changing the motor speeds, but without affecting the total thrust produced. For identical propellers, this means that if a propeller is slowing down to change phase, its counterpart’s speed must increase to make up for the loss in thrust. For this reason, Figure b shows that the propeller speeds diverge when the phase transition is occurring. Despite these speed changes, the vehicle orientation, shown in Figure c, and position are not affected.
In this figure, the trajectory position, orientation, and acoustic performance are shown. The vehicle position trajectory is shown as a black line, and the vehicle orientation is shown at equally spaced time points along the trajectory. The acoustic performance is simulated for each simulated state of the vehicle. Since the sound heard at the observer locations changes drastically based on the position and orientation of the vehicle, the plotted SPL is the maximum value heard during the entire flight duration. Though this visualization does not directly show the acoustic effect of a particular vehicle state, the information is still somewhat localized, since the attenuation due to distance prevents distant states contributing to the maximum.

Trajectory Generation for Distributed Electric Propulsion Vehicles with Propeller Synchronization

[Article
In this work, we propose a method for generating dynamically feasible trajectories for an acoustically aware vehicle with propeller phase control. The trajectory generation procedure allows both propeller phase control and navigation objectives to be considered simultaneously. The presented method is demonstrated where the mission objectives are given as a desired position and phase trajectory. From these trajectories, the full desired state of the vehicle is calculated. Furthermore, the control inputs that realize the desired mission objectives are computed. The acoustic performance for the given trajectory is estimated in terms of sound pressure level as a function of tracking performance. The method is demonstrated in simulation, where the vehicle must navigate through an urban environment with both spatial and acoustic constraints. In the presented scenario, the vehicle must follow a given flight path, and can only reduce sound pressure level by changing the propeller phase targets.

The results of the acoustic tests are demonstrated for three cases: uncontrolled phase, robust phase control, and aggressive phase control. For each of these cases, the controller performance is shown in terms of the motor phase error, shown in the figure above. A detailed view of the sound pressure level at the blade passage frequency over time is also provided in the figure below. The overall performance during the time interval is given in terms of the short-time Fourier transform calculated with a Hanning window, 90% overlap, and a frequency resolution of 5 Hz, as shown in the figure below.

Socio-Mathematics (DoD Basic Research Office)

"Socio-Mathematics": Foundations of Social Information Dynamics and Control

The objective of this project is to develop a mathematical theory, based on models founded in social and cognitive science, to inform defense against malicious manipulation of information flow and belief dynamics on online media. Emphasis is made on closed-loop operation, where two (or more) parties in conflict execute real-time maneuvers in the information space. Confrontation in military domains, such as land and air, is based on well-understood dynamics that inform the assignment of effects to counteract threats. In the domain of information warfare, however, the fundamental underlying system dynamics are poorly understood. The proposed work bridges the above gap by developing theoretical models that combine social sensing, social/cognitive modeling, learning, stochastic control, game theory, and network embedding in order to enable prediction of adversarial action, inform reasoning about effects in the information space, and design policies to counter malicious information operations.
Acknowledgment: Funded by DoD Basic Research Office.

REFERENCES

IMPACT OF CONFIRMATION BIAS ON COMPETITIVE INFORMATION SPREAD IN SOCIAL NETWORK

Yanbing Mao, Emrah Akyol, and Naira Hovakimyan

IEEE Transactions on Control of Network Systems, 2021.

SOCIAL SYSTEM INFERENCE FROM NOISY OBSERVATION

Yanbing Mao, Naira Hovakimyan, Tarek Abdelzaher, and Evangelos Theodorou

arXiv preprint arXiv:2012.03075, 2020.

Safe Operation of Connected Vehicles (STII)

Safe Operation of Connected Vehicles in Complex and Unforeseen Environments

This project seeks to advance the state-of-the-art by designing a proactive/reactive adaptation and learning architecture for connected vehicles, unifying techniques in spatio-temporal data fusion, machine learning, and robust adaptive control.

Robust Vehicle Lane Keeping Control with Networked Proactive Adaptation

Road condition is an important environmental factor for autonomous vehicle control. A dramatic change in the road condition from the nominal status is a source of uncertainty that can lead to a system failure. Once the vehicle encounters an uncertain environment, such as hitting an ice patch, it is too late to reduce the speed, and the vehicle can lose control. To cope with future uncertainties in advance, we study a proactive robust adaptive control architecture for autonomous vehicles’ lane-keeping control problems. The data center generates a prior environmental uncertainty estimate by combining weather forecasts and measurements from anonymous vehicles through a spatio-temporal filter. The prior estimate contributes to designing a robust heading controller and nominal longitudinal velocity for proactive adaptation to each new abnormal condition. Then the control parameters are updated based on posterior information fusion with on-board measurements.

Virtual Sully (NSF CPS)

"Virtual Sully": Autopilot with Multilevel Adaptation for Handling Large Uncertainties.

An interesting project regarding autopilots and their ability to mitigate risks inspired by the US Airways Flight 1549 in 2009 and Captain Chesley Sullenberger ("Sully") and his co-pilot Jeffrey Skiles.

News

Backup Plan Constrained Model Predictive Control

This article proposes a new safety concept: backup plan safety. The backup plan safety is defined as the ability to complete one of the alternative missions in the case of primary mission abortion. To incorporate this new safety concept in control problems, we formulate a feasibility maximization problem that adopts additional (virtual) input horizons toward the alternative missions on top of the input horizon toward the primary mission. Cost functions for the primary and alternative missions construct multiple objectives, and multi-horizon inputs evaluate them. To address the feasibility maximization problem, we develop a multi-horizon multi-objective model predictive path integral control (3M) algorithm. Model predictive path integral control (MPPI) is a sampling-based scheme that can help the proposed algorithm deal with nonlinear dynamic systems and achieve computational efficiency by parallel computation. Simulations of the aerial vehicle and ground vehicle control problems demonstrate the new concept of backup plan safety and the performance of the proposed algorithm.

Emergency Landing

L1-GP: L1 Adaptive Control with Bayesian Learning

 [Webpage] [Poster] [Article]
L1-GP is an architecture based on L1 adaptive control and Gaussian Process Regression (GPR) for safe simultaneous control and learning. On one hand, the L1 adaptive control provides stability and transient performance guarantees, which allows for GPR to efficiently and safely learn the uncertain dynamics. On the other hand, the learned dynamics can be conveniently incorporated into the L1 control architecture without sacrificing robustness and tracking performance. Subsequently, the learned dynamics can lead to less conservative designs for performance/robustness tradeoff.

Urban Aerial Mobility (NASA)

L1-GP: L1 Adaptive Control with Bayesian Learning

L1 Adaptive MPPI Architecture for Robust and Agile Control of Multirotors

Deep Incrementally Stabilizing Control

[Article
The work presents a method for jointly producing incrementally exponentially stable (IES) dynamics and the associated control policies, by leveraging the function approximation capabilities of deep neural networks (DNNs) to learn both jointly. The methodology’s main feature is the construction of hypothesis spaces that are analytically verified to produce IES dynamics and control policies. The analytical verification ensures that inviolable hard constraints need not be enforced as soft constraints in the DNN optimization. Thus, we entirely avoid the overly conservative and problematic post-optimization verification of candidate DNN solutions. We provide numerical experimentation to validate the proposed methodology.

SYNDROME: SYNergetic DROne delivery network in MEtropolis (NSF NRI)

Overview

Risk-Sensitive Rendezvous Algorithm for Heterogeneous Agents in Urban Environments

Demand for fast and economical parcel deliveries in urban environments has risen considerably in recent years. A framework envisions efficient last-mile delivery in urban environments by leveraging a network of ride-sharing vehicles, where Unmanned Aerial Systems (UASs) drop packages on said vehicles, which then cover the majority of the distance before final aerial delivery. Notably, we consider the problem of planning a rendezvous path for the UAS to reach a human driver, who may choose between N possible paths and has uncertain behavior, while meeting strict safety constraints. The long planning horizon and safety constraints require robust heuristics that combine learning and optimal control using Gaussian Process Regression, sampling-based optimization, and Model Predictive Control. The resulting algorithm is computationally efficient and shown to be effective in a variety of qualitative scenarios.

VR environment: Socially Aware Motion Planning for a Flying Robot with Model Predictive Path Integral Control

Proximity Queries for Absolutely Continuous Parametric Curves

In motion planning problems for autonomous robots, such as self-driving cars, the robot must ensure that its planned path is not in close proximity to obstacles in the environment. However, the problem of evaluating the proximity is generally non-convex and serves as a significant computational bottleneck for motion planning algorithms. In this video, we present methods for a general class of absolutely continuous parametric curves to compute: (i) the minimum separating distance, (ii) tolerance verification, and (iii) collision detection. 

Curve – Convex PolygonCurve – Curve

The paper and the code can be downloaded through arXiv and github.

Machine Learning

The problem of learning a global map using local observations by multiple agents lies at the core of many control and robotic applications. Once this global map is available, autonomous agents can make optimal decisions accordingly. In this project, we study a distributed RL algorithm for multi-agent UAV applications. In the distributed RL, each agent makes state observations through local processing. The agents are able to communicate over a sparse randomly changing communication network and can collaborate to learn the optimal global value function corresponding to the aggregated local rewards without centralized coordination.

This project aims to explore the use of small aerial and ground co-robots in domestic environment to assist older population in their daily activities. Study shows that for the types of daily activity assistance, fetching objects from the floor or another room, reaching for objects, and finding/delivering items are among those tasks which are preferred to be completed by robot. To that end, various techniques including machine learning, mechanical design, system analysis, and control design have been explored and integrated in this research program.

The goal of this project is to provide the foundations to address human related concerns that arise in multiple human-robot systems, where robots have to perform tasks in the presence of (and in cooperation with) humans. In particular, we are targeting the fundamental understanding of two issues that are crucial in the integration of robotic systems into real-life human populated environments: first, how humans perceive autonomous mobile robots as a function of robots’ appearance and behavior; second, how to design and control mobile robots to improve the level of comfort and perceived safety of the people present in the environment.

Security of Cyber-Physical Systems

This research investigates resilient cyber-physical systems (CPS) against coordinated cyber and physical attacks (CCPA). When physical damage is coupled with cyber attacks, it raises a unique challenge with respect to CPS safety. Although significant research has been done to counter either cyber or physical attacks, the existing approaches fail to work harmoniously to counter CCPA. In addition, complete verification of CPS against CCPA is still a great challenge, since there is a lack of unified scientific understanding of resilient CPS and supporting technologies against CCPA. We propose an integrated architecture for CPS security and fault-tolerance, which is an implementable and theoretically substantiated framework with reduced complexity for its software architecture.
To illustrate the challenges and validate the theoretical approaches in this integrated CPS framework, we focus on UAV platforms. During flight tests, the output signals of the autonomous UAVs are sampled by sensors. Based on the sensed data, control decisions are made by the flight control system. To ensure safety, the state of the aircraft, such as angle of attack and/or pitch angle, must stay inside an appropriate stability envelope. The fault and attack models under consideration are generic, including mechanical faults due to physical faults/attacks and the resulting stability envelope faults (e.g., the active envelope is not appropriate for the aircraft with the faulty dynamics), complex controller software faults due to software defects or cyber attacks (e.g., malware execution or interference in low-level computing resources during critical points), and sensor faults due to sensor faults/interference/spoofing.
(a) A quadcopter drone controlled by an open-source autopilot when it is killed by a malicious Linux kernel module. (b) The drone is controlled by a virtual controller sandboxed in a virtual machine. Upon the detection of a security violation (such as the process termination, which leads to an open-loop state), the safety controller takes over the control and keeps the drone in a controllable state.

Security and Safety Monitor Design for CCPA Detection:

Safety Controller Design:

Cooperative Control of Multi-Agent Systems (AFOSR)

Worldwide, there has been growing interest in the use of autonomous vehicles to execute cooperative missions of increasing complexity without constant supervision of human operators. In aerospace, for instance, unmanned systems have become ubiquitous in both military and civilian applications. Today, for example, unmanned air vehicles must execute military reconnaissance and strike operations, border patrol missions, forest fire detection, police surveillance, and recovery operations. Similarly, in the marine environment, autonomous marine vehicles are deployed to find mines in coastal waters, inspect underwater structures, help in the construction of marine habitat mappings in regions that are inaccessible to humans, and study the causes behind the disappearance of coral reefs, to name but a few.In all these mission scenarios, the use of a cooperative group of vehicles supported by an inter-vehicle communications network (rather than a single heavily equipped vehicle) provides robustness to system failures, increases system overall reliability, and improves mission efficiency, especially when operating in large spatial domains. However, despite significant progress in the field of cooperative control, several challenges still need to be addressed to develop strategies capable of yielding robust performance of a fleet of vehicles in the presence of complex vehicle dynamics, communications constraints, and partial vehicle failures.

At the ACRL, in collaboration with NASA, the Naval Postgraduate School (NPS, Monterey, CA), and the Instituto Superior Técnico (IST, Lisbon, Portugal), we develop, implement, and test robust decentralized strategies for cooperative motion control of multiple vehicles that must meet stringent spatial and temporal constraints. Figure 1 depicts an architecture of the cooperative control framework adopted in our work.


Figure 1. Architecture of the cooperative control framework

In this work, tools from real-time optimization, Lyapunov-based stability analysis, robust control, graph theory, and logic-based communications are brought together for the development of decentralized robust cooperative control algorithms which achieve the following objectives (click for details):

Cyber-Physical-Human Systems (NSF CPS)

Cyber-Physical-Human (CPH) Systems present a particular class of safety-critical applications, where the interaction between the dynamics of the system and the cyber elements of its operation can be influenced by the human operator and the interaction between these three elements needs to be regulated for various objectives. CPH systems consist of three main components: physical elements modeling the systems to be controlled, cyber elements representing the communication links and software, and human operators who partially monitor the operation of the system and can interfere on as needed basis. Our research in CPH systems aims at development of robust, fault-tolerant architectures that would ensure predictable operation of the system with the given hardware constraints, despite the uncertainties in physical processes,  and cyber and human faults.

1. Control and Scheduling Co-design
In networked control systems (NCSs), the feedback loops are closed by real-time communication networks.  On one hand, the introduction of networking can be advantageous in terms of lower system costs due to streamlined installation and maintenance costs.  On the other hand, however, networking can introduce significant challenges for control of these systems with desired specifications. Communication, especially wireless communication, takes place in a discrete-time manner.  Because of the limited channel capacity, the data transmission has to be scheduled in an appropriate manner for proper operation of the control system.  Otherwise, the quality of service cannot be guaranteed, which may degrade the system performance or even render the system unstable. Similar challenges appear in computer-controlled systems.  To address these issues, our research focuses on systematic co-design approaches of both feedback control and network/computation scheduling.Our preliminary results in this direction present scheduling algorithms for real-time implementation of L1 adaptive controller. Event-triggering schedules the data transmission dependent upon errors exceeding certain threshold. We show that with the proposed event-triggering schemes the states and the input in the networked system can be arbitrarily close to those of a stable reference system by increasing the sampling frequency and the transmission frequency. Stability conditions, in terms of event threshold and allowable transmission delays, are also derived, which serve as the guidance in real-time scheduling. The performance bounds are quantified in terms of hardware constraints.

2. Fault tolerance in CPH Systems
Faults in CPH systems could be due to violation of assumptions in the physical and cyber elements, as well as due to human errors. Physical faults refer to the unpredictable factors/accidents that have severe impact on the physical components in the system, thus violating the main assumptions used in the modeling and control analysis. Cyber faults mainly refer to all errors in the computer and/communication systems, such as CPU overflow, communication jam, software errors, and mistakes in decision-making algorithms. Human faults are the mistakes made by the human operator. These different types of faults can appear jointly, sequentially or separately, creating catastrophic or hazardous conditions for the operation of the system. We are particularly concerned with understanding of pilot models and design of architectures, which can provide accurate situation awareness on distributing the actions between the human operator and the automation.

3. Designing Human-Centered Automation and Cockpit Displays to Enhance Pilot Situation Awareness
This research presents an interface system display that is conceived to improve pilot situation awareness with respect to a flight envelope protection system, developed for a midsized transport aircraft. The new display is designed to complement existing cockpit displays by augmenting them with information related to both aircraft state and control automation. In particular, the proposed display provides cues about the state of automation directly in terms of pilot control actions, in addition to flight parameters. The study also presents results of a piloted simulation-based evaluation, which was designed to validate the developed interface by assessing the relevance of the displayed information, as well as the adequacy of the display layout.

Video of a prototype architecture for human-automation interaction to prevent aircraft loss-of-control (LoC) incidents or accidents.

References:

  1. Kasey A. Ackerman, Donald A. Talleur, Ronald S. Carbonari, Enric Xargay, Benjamin D. Seefeldt, Alex Kirlik, Naira Hovakimyan, and Anna C. Trujillo. “Automation Situation Awareness Display for a Flight Envelope Protection System", Journal of Guidance, Control, and Dynamics, Vol. 40, No. 4, 2017, pp. 964-980.
  2. Kasey Ackerman, Enric Xargay, Donald A. Talleur, Ronald S. Carbonari, Alex Kirlik, Naira Hovakimyan, Irene M. Gregory, Christine M. Belcastro, Anna Trujillo, and Benjamin D. Seefeldt. “Flight Envelope Information-Augmented Display for Enhanced Pilot Situational Awareness", AIAA Infotech @ Aerospace, AIAA SciTech Forum, (AIAA 2015-1112)
  3. Wang, X., Kharisov, E. and Hovakimyan, N., “Real-Time L1 Adaptive Control Algorithm in Uncertain Networked Control Systems,” submitted to IEEE Transactions on Automatic Control, 2011. [pdf]
  4. Wang, X., Sun, Y. and Hovakimyan, N., “Asynchronous task execution in networked control systems using decentralized event-triggering," Systems & Control Letters Vol.  61, No.  9,  2012, pp. 936-944.
  5. Wang, X. and Hovakimyan, N., “A Decoupled Design in Distributed Control of Uncertain Networked Control Systems,"  American Control Conference, 2012.
  6. Wang, X., Sun, H., Hovakimyan, N. and Başar, T., “Bounds on Transmission Rates and Performance in Quantized Network Systems," IFAC World Congress, 2011.
  7. Wang, X. and Hovakimyan, N., “L1 Adaptive Control of Event‐Triggered Networked Systems," American Control Conference, 2010.

Contact:

Naira Hovakimyan nhovakim (at) illinois (dot) edu

To read more about the NSF and their funding of this project, click here.

L1 Adaptive Control

Background

L1 Adaptive Control is a novel theory for the design of robust adaptive control architectures using fast adaptation schemes.

The key feature of L1 adaptive control is the decoupling of the adaptation loop from the control loop, which enables arbitrarily fast adaptation without sacrificing robustness. In fact, in L1 adaptive control architectures, the rate of the adaptation loop can be set arbitrarily high, subject only to hardware limitations (computational power and high-frequency sensor noise), while the tradeoff between performance and robustness can be addressed through conventional methods from classical and robust control. This separation between adaptation and robustness is achieved by explicitly building the robustness specification into the problem formulation, with the understanding that the uncertainties in any feedback loop can be compensated for only within the bandwidth of the control channel. From an architectural perspective, this modification of the problem formulation leads to the insertion of a bandwidth-limited filter in the feedback path, which ensures that the control signal stays in the desired frequency range.

On one hand, fast adaptation allows for compensation of the undesirable effects of rapidly varying uncertainties and significant changes in the system dynamics. Fast adaptation is also critical to achieve predictable transient performance for system’s both signals, input and output, without enforcing persistency of excitation or resorting to high-gain feedback. On the other hand, the bandwidth-limited filter keeps the robustness margins bounded away from zero in the presence of fast adaptation. To this extent, the bandwidth and the structure of this filter define the tradeoff between performance and robustness of the closed-loop adaptive system.

To learn more about the theory of L1 Adaptive Control, please follow this link.

Applications

The features of L1 Adaptive Control described above have been verified –consistently with the theory– in a large number of flight tests and experiments in mid- to high-fidelity simulation environments. Brief overviews of the application of L1 Adaptive Control can be found below:

AirSTAR
As part of the IRAC Project, an L1 flight control system was flight tested on the NASA’s AirSTAR Generic Transport Model (GTM) aircraft. The results of the flight tests demonstrated that, in the presence of aircraft component failure and significant changes in aircraft dynamics, the L1 flight control system is able to maintain aircraft safe operation and predictable performance with reduced pilot workload during both standard flight conditions and unusual flight regimes, like stall and post-stall. More >>

Learjet
The first manned L1 flight control test was conducted by the U.S. Air Force Test Pilot School at Edwards Air Force Base, CA. This project consisted of flying and handling qualities assessment of Calspan’s variable-stability Learjet aircraft augmented with an L1 adaptive flight control law. The variable-stability capability of the aircraft was used to alter its apparent dynamics while in flight, allowing the validation of the L1 flight control law against a set of off-nominal aircraft configurations, some of them with aggressive tendencies to adverse pilot-aircraft interaction. All of these off-nominal configurations were opaque to the control law, and no fault detection and isolation methods were employed. The results demonstrated that the L1 flight control law was able to significantly restore the flying qualities of a baseline Learjet model, and also recover consistent and safe handling qualities. The evaluation also included a series of straight-in landings with two different aircraft configurations.More >>

F-16
An L1 adaptive flight control system has been designed for the U.S. Air Force’s VISTA F-16 aircraft. This project will study the system’s performance in the more dynamically challenging environment provided by the VISTA F-16. With the VISTA F-16, additional failure configurations will be tested to demonstrate the ability of the L1 flight control law to compensate for off-nominal dynamics, actuator failures, and other types of uncertainties not included in the Learjet flight tests. More >>

Learn-to-Fly
Learn-to-Fly is a new framework that aims to eliminate or reduce ground-based aero-dynamic modeling in the traditional development paradigm for aerial vehicles in favor of in-flight modeling and control law determination. At the initial learning stage, large uncertainties or errors inevitably exist with the learned model. For guaranteeing vehicle safety in the presence of such uncertainties and model errors, researchers from NASA Langley Research Center (LaRC) and Prof. Naira Hovakimyan’s lab at the University of Illinois at Urbana-Champaign designed an L1 adaptive control (L1AC) law to actively estimate and compensate for the discrepancy between the intermediately learned dynamics and the actual dynamics. The framework is validated by flight tests on UAVs performed by NASA LaRC in 2017 and 2019. More >>

SIMONA
As a new effort in the development and technology transition of L1 adaptive control to general aviation aircraft, a simulation handling qualities assessment for a small business jet augmented with an L1 adaptive flight control system was conducted in collaboration with the Delft University of Technology (The Netherlands). The main objective of the study was to investigate the ability of an L1 adaptive FCS to provide enhanced handling qualities and maneuverability margins for safe landing in the presence of failures and in different atmospheric conditions. The experiments were conducted on the TU Delft’s 6DOF motion-based SIMONA Research Simulator. More >>

Energy Applications
L1 adaptive control has attracted StatOil and Schlumberger for possible applications in managed pressure drilling and rotary steerable systems. More >>

Hard Disk Control
In cooperation with Seagate Technology, ACRL has investigated applications of diverse adaptive schemes to increase overall tracking performance of hard disk drives under operational vibrations disturbances, including

  1. dynamic control allocation for dual actuator vibration cancelation
  2. design of adaptive actuator augmentation to mitigate drive to drive variation
  3. active adaptive feedforward vibration cancellation methods
  4. multi narrow-band adaptive disturbance observer to reduce tracking error

Optical Soliton Propagation
L1 adaptive control scheme has been developed for the problem of active dispersion management for propagation of solitons along uncertain fibers. More >>

Stochastic Systems with Communication Constraints

Recent progress in communication technologies and their use in feedback control systems motivate to look deeper into the interplay of control and communication in the closed-loop feedback architecture. Among several research directions on this topic, a great deal of attention has been given to the fundamental limitations of feedback control in the presence of communication constraints. Our work considers continuous-time systems in the presence of limited information and quantifies Bode-like performance limitations for this class of systems in terms of mutual information rates. We also provide extension of Bode-like integrals to switched discrete-time systems. The results obtained so far led to new opportunities for optimal estimation over Gaussian channels with noiseless feedback.

Selected publications:

  1. Li, D. and Hovakimyan, N., “Bode-like Integral for Continuous-Time Closed-Loop Systems in the Presence of Limited Information,” IEEE Transactions on Automatic Control (in review) [pdf]
  2. Li, D. and Hovakimyan, N., “Bode-Like Integral for Stochastic Switched Systems in the Presence of Limited Information,” Automatica (in review) [pdf]
  3. Li, D. and Hovakimyan, N., “Optimal State Estimation Over Gaussian Channels with Noiseless Feedback,” in IEEE Conference on Decision and Control, Orlando, FL, 2011. [pdf]

Contact:

Naira Hovakimyan nhovakim (at) illinois (dot) edu

Singular Characteristics of First Order PDEs

Singular surfaces  typically arise in optimal control problems or differential games, where either the value function of the underlying Hamilton-Jacobi-Bellman-Isaacs equation is non-smooth or the initial or terminal conditions are non-smooth. In that case, the method of regular characteristics does not allow for construction of the optimal phase portrait in the entire space. The method of singular characteristics has been explored towards the construction of optimal trajectories in the entire phase space.  Pursuit-evasion games on manifolds with non-unique geodesic lines present a nice application for the method of singular characteristics. Some recent applications include connectivity maintenance among non-cooperative agents, pursued in collaboration with Sourabh Bhattacharya and Tamer Başar.

Selected Publications:

  1. Hovakimyan, N. and Melikyan, A., “Geometry of Pursuit-Evasion on Second-Order Rotation Surfaces", in Dynamics and Control, Vol. 10, pp. 297-312, 2000.
  2. Melikyan, A., Akhmetzhanov, A. and Hovakimyan, N., “Initial Value and Terminal Value Problems for Hamilton-Jacobi Equation,” in Systems & Control Letters, Vol. 56, pp. 714-721, 2007. [pdf]
  3. Bhattacharya, S., Başar, T and Hovakimyan, N., “Singular Surfaces in Multi-Agent Connectivity Maintenance Games,” in IEEE Conference on Decision and Control, Orlando, FL, 2011. [pdf]

Contact:

Naira Hovakimyan
Sourabh Bhattacharya
nhovakim (at) illinois (dot) edu
sbhattac (at) illinois (dot) edu

Biological Networks

Mitogen activated protein kinase (MAPK) cascade is evolutionally preserved in all eukaryotic cells, and regulates cellular activities as gene expression, mitosis, differentiation, and apoptosis. Recent years have witnessed increase activity in reshaping the MAPK cascade through engineered feedback loops, by referring to heuristic tuning mechanisms to synthesize the feedback. A problem of interest is to determine whether information regarding the underlying biochemical reactions can be used to synthesize robust feedback that will ensure that the resultant circuit has the desired properties. Jointly with Vishwesh Kulkarni from University of Minnesota we consider the problem of engineering feedback in MAPK cascade to synthesize an oscillator of the desired frequency.  We show how the L1-control theory can be used for a robust synthesis of the oscillator and we validate the theory in simulations.

References:

  1. Kulkarni, V., Paranjape, A., Ghusinga, K.R., and Hovakimyan, N., “Synthesis of robust tunable oscillators using mitogen activated protein kinase cascades," in Systems and Synthetic Biology, 4:331–341, 2010. [pdf]

Transitions

Learjet Flight Test Videos

The following videos are from flight testing of a Learjet augmented with an L1 adaptive flight control law. It was the first time an L1 adaptive system was tested in flight on a manned aircraft, and it represents an important step towards the introduction of this technology into commercial aviation.

[youtube width="480" height="360"]https://www.youtube.com/watch?v=apy_Mbfa8t4[/youtube]

Manned Flight Test of an L1 Adaptive Flight Control Law: Low-Frequency Tracking Task

[youtube width="480" height="360"]https://www.youtube.com/watch?v=5Fa53PGOTzo[/youtube]

Manned Flight Test of an L1 Adaptive Flight Control Law: Landing

[youtube width="480" height="360"]https://youtu.be/_AiLW-VYdOQ[/youtube]

Flight Test of L1 Adaptive Augmentation on Learjet: Lifting Body

Raymarine Evolution Autopilot using L1 Adaptive Control

Raymarine has marketed their Evolution Autopilot for marine vessels. The autopilot encompasses an L1 adaptive controller and eliminates the need for a complicated setup and calibration. More information on the Evolution Autopilot can be found at the product page.
Raymarine Evolution Autopilot

Energy Applications

L1 adaptive control has attracted StatOil and Schlumberger for possible applications in managed pressure drilling and rotary steerable systems.
Recent Publications

  • H. Mahdianfar, N. Hovakimyan, A. Pavlov, and O. M. Aamo, L1 Adaptive Output Regulator Design with Application to Managed Pressure Drilling, in Journal of Process Control, vol. 42, pp. 1-13, 2016.
  • Z. Li, N. Hovakimyan, and G.-O. Kaasa, Bottom hole pressure estimation and L1 adaptive control in managed pressure drilling system, in International Journal of Adaptive Control and Signal Processing, Vol. 31, pp. 545–561, 2016.

IFAC Award paper

“The Article: 'L1 adaptive manoeuvring of unmanned high-speed water craft' got the prestigious 'Best Paper Award' at the IFAC Conference MCMC 2012 (Marine craft manoeuvring control) in Arenzano, Italy."
Click here to read more details and download the full paper.

Helicopter Flight Simulator

L1 controller for a generic light utility helicopter. The simulation is manually piloted and includes sensor noise and actuator saturation. A vertical speed controller is active that commands hover if the collective lever is pulled back to a region near the original hover position.

L1 adaptive control on a light utility helicopter.

A Comprehensive Flight Control Design and Experiment of a Tail-sitter UAV

In this paper, the authors present an autonomous Take-Off and Landing of a Tail-sitter UAV
Abstract:
“There have been ongoing interests in a type of aircraft that are capable of vertical take-off/landing (VTOL) for greater operability and high-speed horizontal flight capability for maximal mission range. A possible solution for such application is tail-sitters, which takes off vertically and transitions into a horizontal flight. During the entire mission of a tail-sitter from take-off to landing, it goes through largely varying dynamic characteristics. In this paper, we propose a set of controllers for horizontal, vertical, and transition flight regimes. Especially, for transition, in conjunction with conventional multi-loop feedback, we use L1 adaptive control to supplement the linear controllers. The proposed controller were first validated with simulation models and then validated in actual flight tests to successfully demonstrate its capability to control the vehicle over the entire operating range."
Click here to view the full paper.
Below you can find the flight test results.

Flight testing of a tailsitter implemented with L1 adaptive control.

L1 adaptive depth control of an underwater vehicle in the presence of uncertainties and disturbances

“This video shows experimental results of depth control obtained at LIRMM (Laboratoire d'Informatique de Robotique et de Microélectronique de Montpellier), University Montpellier 2 – CNRS, France. The horizontal displacements are left uncontrolled. The L1 adaptive controller is applied to control the depth. This controller is robust towards uncertainties (e.g. floatbility, damping…) and well rejects external disturbancies (impacts, tether drag…). The prototype is a modified AC-ROV."

L1 adaptive depth control of an underwater vehicle in the presence of uncertainties and disturbances.

AutoQuad L1 testing

What would happen if one motor shuts down? Check out this video made in The Netherlands, where L1 controller is implemented on an hexarotor UAV's autopilot. Even when one propeller is missing, stability performance are guaranteed.

AutoQuad L1 adaptive control flight test with one motor shut down.

Non-cascaded Dynamic Inversion Design for Quadrotor Position Control with L1 Augmentation

Flight test results from TU Munich. The work is the result of this paper.
Abstract:
“This paper presents a position control design for quadrotors, aiming to exploit the physical capability and maximize the full control bandwidth of the quadrotor. A novel non-cascaded dynamic inversion design is used for the baseline con-trol, augmented by an L1 adaptive control in the rotational dy-namics. A new implementation technique is developed in the linear reference model and error controller; so that without causing any inconsistency, nonlinear states can be limited to their physical constraints. The L1 adaptive control is derived to compensate plant uncertainties like inversion error, disturbances, and pa-rameter changes. Simulation and experiment tests have been per-formed to verify the effectiveness of the designs and the validi-ty of the approach."

Quadrotor Position Control with L1 Augmentation.