The problem of learning a global map using local observations by multiple agents lies at the core of many control and robotic applications. Once this global map is available, autonomous agents can make optimal decisions accordingly. In this project, we study a distributed RL algorithm for multi-agent UAV applications. In the distributed RL, each agent makes state observations through local processing. The agents are able to communicate over a sparse randomly changing communication network and can collaborate to learn the optimal global value function corresponding to the aggregated local rewards without centralized coordination.
This project aims to explore the use of small aerial and ground co-robots in domestic environment to assist older population in their daily activities. Study shows that for the types of daily activity assistance, fetching objects from the floor or another room, reaching for objects, and finding/delivering items are among those tasks which are preferred to be completed by robot. To that end, various techniques including machine learning, mechanical design, system analysis, and control design have been explored and integrated in this research program.
The goal of this project is to provide the foundations to address human related concerns that arise in multiple human-robot systems, where robots have to perform tasks in the presence of (and in cooperation with) humans. In particular, we are targeting the fundamental understanding of two issues that are crucial in the integration of robotic systems into real-life human populated environments: first, how humans perceive autonomous mobile robots as a function of robots’ appearance and behavior; second, how to design and control mobile robots to improve the level of comfort and perceived safety of the people present in the environment.
Security and Safety Monitor Design for CCPA Detection:
Safety Controller Design:
Worldwide, there has been growing interest in the use of autonomous vehicles to execute cooperative missions of increasing complexity without constant supervision of human operators. In aerospace, for instance, unmanned systems have become ubiquitous in both military and civilian applications. Today, for example, unmanned air vehicles must execute military reconnaissance and strike operations, border patrol missions, forest fire detection, police surveillance, and recovery operations. Similarly, in the marine environment, autonomous marine vehicles are deployed to find mines in coastal waters, inspect underwater structures, help in the construction of marine habitat mappings in regions that are inaccessible to humans, and study the causes behind the disappearance of coral reefs, to name but a few.In all these mission scenarios, the use of a cooperative group of vehicles supported by an inter-vehicle communications network (rather than a single heavily equipped vehicle) provides robustness to system failures, increases system overall reliability, and improves mission efficiency, especially when operating in large spatial domains. However, despite significant progress in the field of cooperative control, several challenges still need to be addressed to develop strategies capable of yielding robust performance of a fleet of vehicles in the presence of complex vehicle dynamics, communications constraints, and partial vehicle failures.
At the ACRL, in collaboration with NASA, the Naval Postgraduate School (NPS, Monterey, CA), and the Instituto Superior Técnico (IST, Lisbon, Portugal), we develop, implement, and test robust decentralized strategies for cooperative motion control of multiple vehicles that must meet stringent spatial and temporal constraints. Figure 1 depicts an architecture of the cooperative control framework adopted in our work.
Figure 1. Architecture of the cooperative control framework
In this work, tools from real-time optimization, Lyapunov-based stability analysis, robust control, graph theory, and logic-based communications are brought together for the development of decentralized robust cooperative control algorithms which achieve the following objectives (click for details):
Cyber-Physical-Human (CPH) Systems present a particular class of safety-critical applications, where the interaction between the dynamics of the system and the cyber elements of its operation can be influenced by the human operator and the interaction between these three elements needs to be regulated for various objectives. CPH systems consist of three main components: physical elements modeling the systems to be controlled, cyber elements representing the communication links and software, and human operators who partially monitor the operation of the system and can interfere on as needed basis. Our research in CPH systems aims at development of robust, fault-tolerant architectures that would ensure predictable operation of the system with the given hardware constraints, despite the uncertainties in physical processes, and cyber and human faults.
1. Control and Scheduling Co-design
In networked control systems (NCSs), the feedback loops are closed by real-time communication networks. On one hand, the introduction of networking can be advantageous in terms of lower system costs due to streamlined installation and maintenance costs. On the other hand, however, networking can introduce significant challenges for control of these systems with desired specifications. Communication, especially wireless communication, takes place in a discrete-time manner. Because of the limited channel capacity, the data transmission has to be scheduled in an appropriate manner for proper operation of the control system. Otherwise, the quality of service cannot be guaranteed, which may degrade the system performance or even render the system unstable. Similar challenges appear in computer-controlled systems. To address these issues, our research focuses on systematic co-design approaches of both feedback control and network/computation scheduling.Our preliminary results in this direction present scheduling algorithms for real-time implementation of L1 adaptive controller. Event-triggering schedules the data transmission dependent upon errors exceeding certain threshold. We show that with the proposed event-triggering schemes the states and the input in the networked system can be arbitrarily close to those of a stable reference system by increasing the sampling frequency and the transmission frequency. Stability conditions, in terms of event threshold and allowable transmission delays, are also derived, which serve as the guidance in real-time scheduling. The performance bounds are quantified in terms of hardware constraints.
2. Fault tolerance in CPH Systems
Faults in CPH systems could be due to violation of assumptions in the physical and cyber elements, as well as due to human errors. Physical faults refer to the unpredictable factors/accidents that have severe impact on the physical components in the system, thus violating the main assumptions used in the modeling and control analysis. Cyber faults mainly refer to all errors in the computer and/communication systems, such as CPU overflow, communication jam, software errors, and mistakes in decision-making algorithms. Human faults are the mistakes made by the human operator. These different types of faults can appear jointly, sequentially or separately, creating catastrophic or hazardous conditions for the operation of the system. We are particularly concerned with understanding of pilot models and design of architectures, which can provide accurate situation awareness on distributing the actions between the human operator and the automation.
3. Designing Human-Centered Automation and Cockpit Displays to Enhance Pilot Situation Awareness
This research presents an interface system display that is conceived to improve pilot situation awareness with respect to a flight envelope protection system, developed for a midsized transport aircraft. The new display is designed to complement existing cockpit displays by augmenting them with information related to both aircraft state and control automation. In particular, the proposed display provides cues about the state of automation directly in terms of pilot control actions, in addition to flight parameters. The study also presents results of a piloted simulation-based evaluation, which was designed to validate the developed interface by assessing the relevance of the displayed information, as well as the adequacy of the display layout.
- Kasey A. Ackerman, Donald A. Talleur, Ronald S. Carbonari, Enric Xargay, Benjamin D. Seefeldt, Alex Kirlik, Naira Hovakimyan, and Anna C. Trujillo. “Automation Situation Awareness Display for a Flight Envelope Protection System", Journal of Guidance, Control, and Dynamics, Vol. 40, No. 4, 2017, pp. 964-980.
- Kasey Ackerman, Enric Xargay, Donald A. Talleur, Ronald S. Carbonari, Alex Kirlik, Naira Hovakimyan, Irene M. Gregory, Christine M. Belcastro, Anna Trujillo, and Benjamin D. Seefeldt. “Flight Envelope Information-Augmented Display for Enhanced Pilot Situational Awareness", AIAA Infotech @ Aerospace, AIAA SciTech Forum, (AIAA 2015-1112)
- Wang, X., Kharisov, E. and Hovakimyan, N., “Real-Time L1 Adaptive Control Algorithm in Uncertain Networked Control Systems,” submitted to IEEE Transactions on Automatic Control, 2011. [pdf]
- Wang, X., Sun, Y. and Hovakimyan, N., “Asynchronous task execution in networked control systems using decentralized event-triggering," Systems & Control Letters Vol. 61, No. 9, 2012, pp. 936-944.
- Wang, X. and Hovakimyan, N., “A Decoupled Design in Distributed Control of Uncertain Networked Control Systems," American Control Conference, 2012.
- Wang, X., Sun, H., Hovakimyan, N. and Başar, T., “Bounds on Transmission Rates and Performance in Quantized Network Systems，" IFAC World Congress, 2011.
- Wang, X. and Hovakimyan, N., “L1 Adaptive Control of Event‐Triggered Networked Systems," American Control Conference, 2010.
|Naira Hovakimyan||nhovakim (at) illinois (dot) edu|
To read more about the NSF and their funding of this project, click here.
L1 Adaptive Control is a novel theory for the design of robust adaptive control architectures using fast adaptation schemes.
The key feature of L1 adaptive control is the decoupling of the adaptation loop from the control loop, which enables arbitrarily fast adaptation without sacrificing robustness. In fact, in L1 adaptive control architectures, the rate of the adaptation loop can be set arbitrarily high, subject only to hardware limitations (computational power and high-frequency sensor noise), while the tradeoff between performance and robustness can be addressed through conventional methods from classical and robust control. This separation between adaptation and robustness is achieved by explicitly building the robustness specification into the problem formulation, with the understanding that the uncertainties in any feedback loop can be compensated for only within the bandwidth of the control channel. From an architectural perspective, this modification of the problem formulation leads to the insertion of a bandwidth-limited filter in the feedback path, which ensures that the control signal stays in the desired frequency range.
On one hand, fast adaptation allows for compensation of the undesirable effects of rapidly varying uncertainties and significant changes in the system dynamics. Fast adaptation is also critical to achieve predictable transient performance for system’s both signals, input and output, without enforcing persistency of excitation or resorting to high-gain feedback. On the other hand, the bandwidth-limited filter keeps the robustness margins bounded away from zero in the presence of fast adaptation. To this extent, the bandwidth and the structure of this filter define the tradeoff between performance and robustness of the closed-loop adaptive system.
To learn more about the theory of L1 Adaptive Control, please follow this link.
The features of L1 Adaptive Control described above have been verified –consistently with the theory– in a large number of flight tests and experiments in mid- to high-fidelity simulation environments. Brief overviews of the application of L1 Adaptive Control can be found below:
As part of the IRAC Project, an L1 flight control system was flight tested on the NASA’s AirSTAR Generic Transport Model (GTM) aircraft. The results of the flight tests demonstrated that, in the presence of aircraft component failure and significant changes in aircraft dynamics, the L1 flight control system is able to maintain aircraft safe operation and predictable performance with reduced pilot workload during both standard flight conditions and unusual flight regimes, like stall and post-stall. More >>
The first manned L1 flight control test was conducted by the U.S. Air Force Test Pilot School at Edwards Air Force Base, CA. This project consisted of ﬂying and handling qualities assessment of Calspan’s variable-stability Learjet aircraft augmented with an L1 adaptive ﬂight control law. The variable-stability capability of the aircraft was used to alter its apparent dynamics while in ﬂight, allowing the validation of the L1 ﬂight control law against a set of off-nominal aircraft conﬁgurations, some of them with aggressive tendencies to adverse pilot-aircraft interaction. All of these off-nominal conﬁgurations were opaque to the control law, and no fault detection and isolation methods were employed. The results demonstrated that the L1 ﬂight control law was able to signiﬁcantly restore the ﬂying qualities of a baseline Learjet model, and also recover consistent and safe handling qualities. The evaluation also included a series of straight-in landings with two different aircraft conﬁgurations.More >>
An L1 adaptive flight control system has been designed for the U.S. Air Force’s VISTA F-16 aircraft. This project will study the system’s performance in the more dynamically challenging environment provided by the VISTA F-16. With the VISTA F-16, additional failure configurations will be tested to demonstrate the ability of the L1 flight control law to compensate for off-nominal dynamics, actuator failures, and other types of uncertainties not included in the Learjet flight tests. More >>
As a new effort in the development and technology transition of L1 adaptive control to general aviation aircraft, a simulation handling qualities assessment for a small business jet augmented with an L1 adaptive flight control system was conducted in collaboration with the Delft University of Technology (The Netherlands). The main objective of the study was to investigate the ability of an L1 adaptive FCS to provide enhanced handling qualities and maneuverability margins for safe landing in the presence of failures and in different atmospheric conditions. The experiments were conducted on the TU Delft’s 6DOF motion-based SIMONA Research Simulator. More >>
L1 adaptive control has attracted StatOil and Schlumberger for possible applications in managed pressure drilling and rotary steerable systems. More >>
Hard Disk Control
In cooperation with Seagate Technology, ACRL has investigated applications of diverse adaptive schemes to increase overall tracking performance of hard disk drives under operational vibrations disturbances, including
- dynamic control allocation for dual actuator vibration cancelation
- design of adaptive actuator augmentation to mitigate drive to drive variation
- active adaptive feedforward vibration cancellation methods
- multi narrow-band adaptive disturbance observer to reduce tracking error
Optical Soliton Propagation
L1 adaptive control scheme has been developed for the problem of active dispersion management for propagation of solitons along uncertain fibers. More >>
Recent progress in communication technologies and their use in feedback control systems motivate to look deeper into the interplay of control and communication in the closed-loop feedback architecture. Among several research directions on this topic, a great deal of attention has been given to the fundamental limitations of feedback control in the presence of communication constraints. Our work considers continuous-time systems in the presence of limited information and quantifies Bode-like performance limitations for this class of systems in terms of mutual information rates. We also provide extension of Bode-like integrals to switched discrete-time systems. The results obtained so far led to new opportunities for optimal estimation over Gaussian channels with noiseless feedback.
- Li, D. and Hovakimyan, N., “Bode-like Integral for Continuous-Time Closed-Loop Systems in the Presence of Limited Information,” IEEE Transactions on Automatic Control (in review) [pdf]
- Li, D. and Hovakimyan, N., “Bode-Like Integral for Stochastic Switched Systems in the Presence of Limited Information,” Automatica (in review) [pdf]
- Li, D. and Hovakimyan, N., “Optimal State Estimation Over Gaussian Channels with Noiseless Feedback,” in IEEE Conference on Decision and Control, Orlando, FL, 2011. [pdf]
|Naira Hovakimyan||nhovakim (at) illinois (dot) edu|
- Hovakimyan, N. and Melikyan, A., “Geometry of Pursuit-Evasion on Second-Order Rotation Surfaces", in Dynamics and Control, Vol. 10, pp. 297-312, 2000.
- Melikyan, A., Akhmetzhanov, A. and Hovakimyan, N., “Initial Value and Terminal Value Problems for Hamilton-Jacobi Equation,” in Systems & Control Letters, Vol. 56, pp. 714-721, 2007. [pdf]
- Bhattacharya, S., Başar, T and Hovakimyan, N., “Singular Surfaces in Multi-Agent Connectivity Maintenance Games,” in IEEE Conference on Decision and Control, Orlando, FL, 2011. [pdf]
|nhovakim (at) illinois (dot) edu
sbhattac (at) illinois (dot) edu
Mitogen activated protein kinase (MAPK) cascade is evolutionally preserved in all eukaryotic cells, and regulates cellular activities as gene expression, mitosis, differentiation, and apoptosis. Recent years have witnessed increase activity in reshaping the MAPK cascade through engineered feedback loops, by referring to heuristic tuning mechanisms to synthesize the feedback. A problem of interest is to determine whether information regarding the underlying biochemical reactions can be used to synthesize robust feedback that will ensure that the resultant circuit has the desired properties. Jointly with Vishwesh Kulkarni from University of Minnesota we consider the problem of engineering feedback in MAPK cascade to synthesize an oscillator of the desired frequency. We show how the L1-control theory can be used for a robust synthesis of the oscillator and we validate the theory in simulations.
- Kulkarni, V., Paranjape, A., Ghusinga, K.R., and Hovakimyan, N., “Synthesis of robust tunable oscillators using mitogen activated protein kinase cascades," in Systems and Synthetic Biology, 4:331–341, 2010. [pdf]
Learjet Flight Test Videos
The following videos are from flight testing of a Learjet augmented with an L1 adaptive flight control law. It was the first time an L1 adaptive system was tested in flight on a manned aircraft, and it represents an important step towards the introduction of this technology into commercial aviation.
Raymarine Evolution Autopilot using L1 Adaptive Control
Raymarine has marketed their Evolution Autopilot for marine vessels. The autopilot encompasses an L1 adaptive controller and eliminates the need for a complicated setup and calibration. More information on the Evolution Autopilot can be found at the product page.
L1 adaptive control has attracted StatOil and Schlumberger for possible applications in managed pressure drilling and rotary steerable systems.
- H. Mahdianfar, N. Hovakimyan, A. Pavlov, and O. M. Aamo, L1 Adaptive Output Regulator Design with Application to Managed Pressure Drilling, in Journal of Process Control, vol. 42, pp. 1-13, 2016.
- Z. Li, N. Hovakimyan, and G.-O. Kaasa, Bottom hole pressure estimation and L1 adaptive control in managed pressure drilling system, in International Journal of Adaptive Control and Signal Processing, Vol. 31, pp. 545–561, 2016.
IFAC Award paper
“The Article: ‘L1 adaptive manoeuvring of unmanned high-speed water craft’ got the prestigious ‘Best Paper Award’ at the IFAC Conference MCMC 2012 (Marine craft manoeuvring control) in Arenzano, Italy."
Click here to read more details and download the full paper.
Helicopter Flight Simulator
L1 controller for a generic light utility helicopter. The simulation is manually piloted and includes sensor noise and actuator saturation. A vertical speed controller is active that commands hover if the collective lever is pulled back to a region near the original hover position.
A Comprehensive Flight Control Design and Experiment of a Tail-sitter UAV
In this paper, the authors present an autonomous Take-Off and Landing of a Tail-sitter UAV
“There have been ongoing interests in a type of aircraft that are capable of vertical take-off/landing (VTOL) for greater operability and high-speed horizontal flight capability for maximal mission range. A possible solution for such application is tail-sitters, which takes off vertically and transitions into a horizontal flight. During the entire mission of a tail-sitter from take-off to landing, it goes through largely varying dynamic characteristics. In this paper, we propose a set of controllers for horizontal, vertical, and transition flight regimes. Especially, for transition, in conjunction with conventional multi-loop feedback, we use L1 adaptive control to supplement the linear controllers. The proposed controller were first validated with simulation models and then validated in actual flight tests to successfully demonstrate its capability to control the vehicle over the entire operating range."
Click here to view the full paper.
Below you can find the flight test results.
L1 adaptive depth control of an underwater vehicle in the presence of uncertainties and disturbances
“This video shows experimental results of depth control obtained at LIRMM (Laboratoire d’Informatique de Robotique et de Microélectronique de Montpellier), University Montpellier 2 – CNRS, France. The horizontal displacements are left uncontrolled. The L1 adaptive controller is applied to control the depth. This controller is robust towards uncertainties (e.g. floatbility, damping…) and well rejects external disturbancies (impacts, tether drag…). The prototype is a modified AC-ROV."
AutoQuad L1 testing
What would happen if one motor shuts down? Check out this video made in The Netherlands, where L1 controller is implemented on an hexarotor UAV’s autopilot. Even when one propeller is missing, stability performance are guaranteed.
Non-cascaded Dynamic Inversion Design for Quadrotor Position Control with L1 Augmentation
Flight test results from TU Munich. The work is the result of this paper.
“This paper presents a position control design for quadrotors, aiming to exploit the physical capability and maximize the full control bandwidth of the quadrotor. A novel non-cascaded dynamic inversion design is used for the baseline con-trol, augmented by an L1 adaptive control in the rotational dy-namics. A new implementation technique is developed in the linear reference model and error controller; so that without causing any inconsistency, nonlinear states can be limited to their physical constraints. The L1 adaptive control is derived to compensate plant uncertainties like inversion error, disturbances, and pa-rameter changes. Simulation and experiment tests have been per-formed to verify the effectiveness of the designs and the validi-ty of the approach."