Software Framework for Mission Planning and State Visualization
Implementation of a Software for Control and Visualisation of autonomous Unmanned Air Vehicles
Prof. Dr.-Ing. Klaus-Dieter Kuhnert
Prof. Dr.-Ing. habil. Konstantin Kondak
In coordination with :
Institute of Robotics and Mechatronics
German Aerospace Centre (DLR), Oberpfaffenhofen
Matr. Nr: 1118641
The field of Aerospace is filled with challenges and excitement, challenges that are difficult and critical but at the same time can be solved by precise engineering. The development of certifiable sensors and high quality computing systems have opened new perspectives for unmanned aerial vehicles. This trend has pushed forward development of more intricate systems demanding much more system tasks with flight systems than were earlier possible. This wave in the field of avionics and autopilot systems allows for smaller 'flying robots' or 'Unmanned Air Systems' with functionalities that is unmatched even when compared to established solutions for commercial aviation. Besides small air planes, the interest in unmanned flying objects such as airships, helicopters and multi-copters is constantly growing. These 'flying robots' are intended to be used in various civilian purposes such as Airborne traffic data information, Safety related monitoring of local areas from air, Wildlife monitoring using thermal imaging, 3D modelling of historic monuments, Solar powered stratospheric platforms for earth observation, navigation and communication etc. The German Aerospace Centre (DLR) has been in the forefront of developing applications in the field of Aerospace. The Robotics and Mechatronics Centre (RMC) at Munich is a competence centre for DLR research and development in the area of mechanics, electronics and information technology for the realization of "intelligent mechanisms" which interact with their environment. The core competence of RMC is the interdisciplinary design, computer-aided optimization and simulation, as well as implementation of complex mechatronic systems and human-machine interfaces. Within the RMC, the Institute of Robotics and Mechatronics develops a wide array of robots to enable humans to interact more safely and efficiently with their surrounding environments. The robots are designed to act in surroundings inaccessible or dangerous to humans as well as to support humans in everyday life and work.
The Flying Robots group at the Institute of Robotics and Mechatronics is involved in developing and experimenting with state of the art Unmanned Air Vehicles (UAVs). According to Peter van Blyenburgh, UAVs are defined as uninhabited and reusable motorized aerial vehicles, which are remotely controlled, semi-autonomous, autonomous, or have a combination of these capabilities, and that can carry various types of payloads, making them capable of performing specific tasks within the earth's atmosphere, or beyond, for a duration. At the group, the missions with the UAVs usually involve the use of manipulators to perform specific tasks. A wide range of UAV platforms such as helicopters, fixed wind aircraft's etc. are used for performing these missions. The platforms are differentiated by their physical characteristics, such as power plant, airframe, and fuel capacity, with complementary hardware and software. For these complex missions, the need of having a comprehensive software for the overall control and visualisation remains a key necessary at the group. Owing to this need, a Ground Control Software(GCS) that can be used in these challenging scenarios would be developed. The task is complex as the system has ground and air borne components built around various platforms and having varying degrees of requirements. The software would consists of various components starting with the integration and processing of the sensor data on the UAV to the User display for Mission Planning, control and visualisation of the UAV. The proposed system would also be used for more complex missions such as cooperative flights between multiple UAVs. Thus the task involves developing an architecture that can accommodate all the components as well as programming and developing individual modules for it.
Focus of the Thesis
The GCS that would be developed for the control and visualisation of the UAV would be consisting of various interconnected modules. The Master thesis would be focussed on suggesting an architecture for the GCS along with the development of selected modules. The selection is achieved in such a way so as to make the realisation of the Thesis possible in the given time frame and also to include various competencies such as concept development and programming skills.
- Software Architecture - The Software Architecture would be dealing with developing a concept for a reliable architecture for the GCS. The architecture would comply with all the high level requirements outlined for the system. It should define the function, interfaces and requirements for each module in the GCS.
- Communication Interface - The Communication module is an important part of the system and can be considered as the backbone of the GCS. All the communication carried out between the UAV and the GS would be brokered through this module. This module needs to be robust and highly reliable. Any failure in this module would ultimately affect the working of the complete GCS itself. This module would be designed, implemented and tested using an Object Oriented programming language.
- Visualisation Interface - This module would be a part of the GS and would be used for visualising the state of the UAV. The information such as the Flight control computers (FCC) Shared memory (SHM) contents, current mission information etc. would be visualised using this module. This module should work in real time constraints as any decision relating to safety and emergency from the Ground station (GS) would be dependent on the information available through it.
- Error handling - Error handling forms the core of any competent software architecture and is responsible for guaranteeing the safe and reliable operation of the GCS under critical conditions. As a part of the Master Thesis, a concept for the Error handling framework would be suggested.
The implementation of the Final Software Architecture is the result of continuous brainstorming and step by step development of the models that were suggested to arrive at the final GCS architecture. The main components of the architecture and their functions are as listed below
- Communication - DDS_Communicator module for communication between UAV and ground station.
- Visualisation - LabVIEW for Visualisation and control for UAV
- Data Logging - MySql database along with RTI Database Integration services to realise Data Logging
- Mission Planning - Rafcon, a tool developed at DLR for Mission planning.
- Mission Verification - Divine for Mission verification
Figure 1: GCS Architecture
The communication module, DDS_Communicator (DDS-CM) links the UAV and the GS. It brokers the communication between the two in a full duplex communication mechanism. The existing communication was carried out by a module called AR_Communicator(ARC). This was developed at the flying robots group and used a text file for its configuration. In the scheme of using a more reliable and efficient alternative to the ARC, the DDS-CM is developed based upon the RTI DDS Middle-ware(MW). The task of the DDS-CM is to copy the contents of the Flight control computer(FCC) Share memory(SHM) to the network and to copy the data from the network to the SHM thus enabling full duplex communication. The following lists the salient features of this communication module
- Fully configurable using a single XML configuration file
- Possibility of running multiple instances on the same physical machine using different logical networks
- Independent from other parts of the GCS architecture
- Incorporates all the features of RTI DDS MW
The DDS-CM is developed in C/C++. The DDS-CM can be used in two modes
- As a Publisher
- As a Subscriber
In the Publishing mode, it publishes the data from the FCC SHM to the Topics, whereas in the Subscribing mode, it subscribes to the data from the Topics and stores it to the FCC SHM. Figure 2 gives an overview of both the modes.
Figure 2: Operating modes for DDS-CM
The Visualisation interface would be used to visualise the state of the UAV. The state of the UAV essentially means the progress of the mission, any critical conditions that it is facing, a snapshot of the SHM contents, the data produced by various sensors and actuators on the UAV etc. LabVIEW due to its powerful features for designing customisable interactive front end displays within a short time has been used for visualisation. The RTI DDS also offers a free tool-kit for integration with LabVIEW and thus LabVIEW could easily be integrated in the GCS.
The LabVIEW Visualiser front end is as shown below. In the Block Diagram (Back end), it is made of multiple Readers that are connected to their respective Topics.
Figure 4: LabVIEW Visualiser front end
Error handling Framework
The Error handling framework called the 'Guardian Model' is introduced in the Thesis as concept for error handling for UAV systems. The Guardian Model is based on the notion of 'Global and Local exceptions'. During the conception, the model was based on the Behaviour Tree(BT) formalism but it can be adopted to be used with any other formalism like state machines etc. The Local exceptions(LE) are raised by modules running on the FCC while Global Exceptions(GE) are raised by the Guardian. The Guardian Model is shown in Figure 5
Figure 5: The Guardian Model
GCS Integration and Testing
The Integration brings together all the capabilities of the GCS. The integration involved testing if the individual components worked together as intended to realise the complete GCS. The LabVIEW Playback was added as an additional feature that can be used for post-mission analysis. To verify the integration of the GCS, a loop-back testing approach was adopted. This verified if the data properly flows from end to end. The loop-back cycle starts from the LabVIEW Controller where data was set in the data structures and was then written to the Topics. All the Topics have different names and are also characterised by different data types. Once the data is published on the Topics, the Subscriber in the DDS-CM is configured to subscriber to this data and store it in the FCC SHM. This completes half the cycle. The Publisher in the DDS-CM is configured to read the data from the SHM and publish it on the Topics. Once the data is published on the Topics, the LabVIEW Visualiser reads the respective data and displays them on the Visualiser front end thus completing the end to end flow. Thus the data that was set in the Controller is received back at the Visualiser. This verifies that the data is flowing properly along its path through various system components. During the flow, the data published on all the Topics is continuously logged in the MySql database. This is verified using the LabVIEW Playback feature. The following set-up was used to verify the GCS integration.
Figure 6: Test set-up for GCS Integration
Other testing methodologies that were used are listed below
- Unit Testing - To verify if DDS-CM classes and linkages work as per their requirements
- System testing - To verify the End to End (E2E) testing for the GCS
- Installation Testing - To verify if DDS-CM works fine on other FCC hardware
- Destructive Testing - To verify if DDS-CM performs well in case of invalid configuration file
- Performance Testing - To verify if DDS-CM performance in unreliable networks
The focal points for the Master Thesis that were highlighted in the beginning have been successfully accomplished. The results are discussed below
- A software architecture for the GCS has been suggested which incorporates all the design requirements.
- The existing MW possibilities were analysed and RTI Connext DDS was selected to be used as the MW for the GCS.
- The DDS-CM based on RTI DDS Connext was successfully designed, implemented and tested
- The DB was integrated in the GCS and tested to show its use as a reliable data logger
- A visualisation and control interface for the GCS was designed and integrated in the GCS architecture
- The Playback module was provided for use in the post-mission analysis of missions
Thus the objectives of the Thesis were met with astounding results and could be used for the realisation of the GCS.