We are in the “Smart” era. Large compagnies especially the GAFAM (Google, Apple, Facebook, Amazon, Microsoft) are increasingly seeking to make all of our everyday objects smart or intelligent. In this article, we aim to see the possibilities for drones to become intelligent and demystify all the legends about AIs.
Thanks to current technology, it is possible to make any device “intelligent”. It started with cellphones that became Smartphones. Then, these same phones / smartphones evolved again and have been able to answer our questions, notably with Siri and Google Assistant.
Now, this motivation to make the devices intelligent has gained all field of human activity. We’re talking about 4.0 Companies. All this is made possible, by a technology known as Artificial Intelligence.
What about drones, those littles gems of technology. They are tending to become more and more intelligent too. A multitude of research aims to make those flying devices more autonomous. We are going to talk about it, explain what is the Artificial Intelligence and demonstrate that we are far, even very far, from a possible rise of the machines as we can see in big Hollywood productions.
Artificial Intelligence, what is it?
The term Artificial Intelligence or AI is born in 1950 in the article “Computing Machinery and Intelligence” published by Alan Turing. Where he wonder if a machine can “think”. He explore this problematic and propose an experimentation (test of Turing) to know when a machine would become conscious. His idea of Artificial Intelligence was ahead of his time, and it takes many years to have the capacities of computer strong enough to support it.
To give you a better presentation of what is the Artificial Intelligence, I have asked the Dr. Rim Slama, expert in Machine Learning and Deep Learning, to write a small explanation.
“ Machine learning is an application of artificial intelligence (AI) that focuses on the developpment of computer programs with the ability to learn from experience without being explicitly programmed. Deep Learning, on the other hand, is just a type of Machine Learning, inspired by the structure of a human brain. DL algorithms uses a multi-layered structure called neural networks.”
Artificial Intelligence in drones
Drones are no exception, there are a lot of research about AI in the drone industry and many applications are already functional. In the field of Computer Vision, some compagnies have developed algorithms that differentiates a person, a vehicle or a simple object. This kind of algorithm can be very helpful in case of people assembly, the authorities are able to count the number of participants. One step further, it is possible to combine this algorithm to another one that can recognize faces. The drone can therefore recognize and follow a specific person. That useful, in case of a dangerous or wanted person is hiding in a crowd. All Face Recognition algorithm are not the same, all of them do not necessary need to see the entire face. Some of them can work with a small part of it. Another step further, with an Action Recognition algorithm, the drone is able to detect a dangerous or inappropriate behavior and report it to the operator.
This is only one example but there are a lot of possibilities. AI and Drones are two large fields and permit a large rang of combined creations. All around de world, we can see plenty research projects to make drones smarter by integrate some AI algorithms in drones.
Among those research projects, the biggest part is Computer Vision related. Due to the facility for drones to capture pictures from above. All those pictures can be processed to detect or recognize a lot of things. Other applications The aim is to automate human tasks to make it quicker, cheaper and maybe better.
For the future, the great trend is to make devices autonomous. Cars, drones, houses, …
For drones, the idea is that it must be aware of its surrounding environment and be able to take the good action at the good time. It must reacts to all expected or unexpected situation in the way to guarantee the safety and the security of everything around it (living beings or not). The goal is to ask the drone to do a simple task and it will do the task from takeoff to landing without any pilot.
For that the drone is equipped with plenty captors to detect everything in a predefined radius around it.
Written by: Dr. Rim Slama, Researcher and Teacher at Hénallux Kevin Neefs, Researcher at Hénallux and holder of a Belgian class 1 license
Authorities apply a risk-based approach. They created four types of operation : – Recreative purpose (low risk) – Class 2 – Class 1b – Class 1a (high risk)
Figure 1 : illustration of different classes from the lowest risk to highest risk. Figure 2 : Illustration as summary of the belgian regulation.
To be in the recreative classification, these rules must be followed : – Maximum Take-Off Mass strictly less than 1kg. – Over a private property with the owner’s autorisation. – Maximum height equal to 10m. – The flight must be operated during daylight. – Respect other’s privacy.
To be in the class 2 classification these rules must be followed : – Maximum Take-Off Mass strictly less than 5kg. – Over an uninhabited area : at 50m from buildings and people. – Maximum height equal to 45m. – The flight must be operated during daylight.
Some additional requirements are mandatory : – The drone must be registered. – The pilot must be aged of 16. – The pilote needs a certificate of competence issued by the BCAA.
For the certificate of competence, the pilot has to attend a theoretical training course on the following topics : – Air regulations – Weather – Navigation – Aircraft technology – Respect for privacy In addition, the pilot has to pass the practical examination with an examiner recognised by BCAA.
To be in the class 1 classification these rules must be followed : – Maximum Take-Off Mass strictly less than 150kg. – Over an uninhabited area : at 30m from buildings and people for class 1b. – Maximum height equal to 90m. – The flight must be operated during daylight. – Respect other’s privacy.
Some additional requirements are mandatory : – The drone must be registered. – The pilot must be aged of 18. – The pilote needs a telepilot license issued by the BCAA and a LAPLMedical certificate. – The company has to write an Operation manual.
Class 1b : – The pilot send a declaration of mission to the DGTA with an acknowledged receipt. – A notification must be sent to DGTA for each flight. Class 1a : – The UAV must have a certificate of conformity. – Authorisation of the BCAA is mandatory.
Drones operations are divided into two types : Recreative operation and Professional operations : – When you are in a recreative operation, the UAV (Unmanned Aerial Vehicle) is divided into two categories : Category A and Category B. – When you are in a professional operation, your mission has to enter in one of the four scenarios the authorities created. (S1, S2, S3 and S4)
Figure : overview of the four scenarios in which all professional drone operations must fit in.
The french authorities created ten rules to follow when you fly with a drone for recreation purposes
I do not fly over people.
I respect the maximum flight heights.
I never lose sight of my drone and I do not use it at night.
I do not fly my drone over public spaces in urban areas.
I never use my drone anywhere near airfields or airports.
I never fly over sensitive areas or protected sites.
I always respect other people’s privacy.
I do not broadcast my aerial pictures or videos without the consent of the concerned persons ans I do not make any commercial use of it.
I check the terms of my insurance policy concerning this activity.
When on doubt, I ask for more information.
Aircraft category A
The following types of UAV are in the category A : – Captive UAV (linked to the ground or to a person) that weights 150kg or less. – Non captive UAV that weights 25kg or less with the following engine limits :
UAV with a 250cm3 or less thermal engine.
UAV with a 15kW or less electric engine.
UAV with a 15kW or less turboprop.
UAV with reactor that the thrust is equal or less than 30daN and with a fuel-free thrust / weight ratio of 1.3 or less.
– Hot air balloon with total gas mass is on-board cylinders of 5kg or less.
Moreover, to operate an UAV that weights 800gr or more, the following rules have to be respected : – The UAV must be registered. – The pilot must follow an online training. – The UAV must have light and sensors.
Aircraft category B
All UAV that do not correspond to category A are in category B. And a permission of the authorities of the civil aviation is required to operate in this category.
The professional use of UAV is divided into four scenarios. During all the operation you have to stay in the limits of the scenario.
– Operate in rural and uninhabited areas. – UAV weighing 25kg or less. – Altitude maximum equal to 150m or less. – Distance UAV – pilot 200m or less.
– Operate in rural and uninhabited areas. – UAV weighing 25kg or less. – Altitude maximum equal to 50M or less. – Distance UAV – pilot 1km or less. – Visual contact is not mandatory.
– Operate in urban areas. – UAV weighing 8kg or less. – Altitude maximum equal to 150m or less. – Distance UAV – pilot 100m or less.
– Operate in uninhabited areas. – UAV weighing 2kg or less. – Altitude maximum equal to 150m or less. – Unlimited distance from pilot.
Drones operations are divided into three sections by UAV weight : – More than 250gr – More than 2kg – More than 5kg
For each weight limit, you have some rules to respect.
Figure : Explanatory illustration on the German legislation
250gr or more
– Each UAV in Germany that weights 250gr or more must be marked so that the keeper can be quickly identified. This marking has to take the form of a sticker bearing the name and address of the owner. – The UAV cannot fly over any residential property without permission of the owner.
2kg or more
To fly an UAV that weights 2kg or more, the pilot has to be a holder of a certificate of knowledge. This document can take the form of : – A pilot’s license. – A certificate issued by a body recognised by the Federal Aviation Office (16 year old). – A certificate issued by a mandated air sports association (14 year old and on model flying sites only).
5kg or more
Every flight that involved an UAV weighing 5kg or more must be permitted by the Federal State aviation authorities. In this category, the authorities can also allow flights beyond visual line of sight.
Other rules : Prohibited operations
All the following cases are strictly prohibited : – Fly in or over sensitive areas (police, emergency service operations, penal institutions or industrial plants, federal state authorities). – Fly over certain items of transport infrastructures (railways, highways). – Fly in aerodrome controled zones. – Fly at altitudes over 100m. – Fly on the way of a manned aircraft. – Fly during night-time. – Operate an UAV weighing more than 25kg.
The first article of this regulation is clear : Every aerial operation must be authorized.
The application for an authorization
Each operator of UAV is asked to pay a tax fee of 619,73 euro. There are some recommendations for the application : – Do not fly over 50m. – Do not fly over people or animals. – Do not fly over transport structures as railways or highways. – Do not take pictures or videos without the consent of the involved third party. – Do not fly in an area of 2km around any helipoty or aerodrome and in 5km of Luxembourg International Airport. – Operate in a virtual predefined area box. – Clearly delimit the working area and prevent the public from access. – Program the UAV to never leave the delimited working area box. – Keep always visual contact on the UAV and land immediately on approach of any manned aircraft. – Have a public liability insurance. – In case of dysfunction the UAV must immediately initiate a safe auto-landing manoeuvre.
Normally, all the locals regulations will be replace by the European regulation by July 2022. And the following rules will be implemented by each country of the EU.
The exploitation of UAV is divided into 3 categories : – The Open category. – The Specific category. – The Certified category.
The Open category is divided into 3 sub-categories : – The sub-category A1. – The sub-category A2. – The sub-category A3.
Each sub-category has UAV Classes assigned.
Category : “Open”
An operation can be considered as Open, it must stay in these limits : – Maximum height strictly less than 120m, but when flying within a horizontal distance of 50 metres from an artificial obstacle higher than 105 metres, the maximum height of the UAV operation may be increased up to 15 metres above the height of the obstacle at the request of the entity responsible for the obstacle. – Maximum weight for the UAV is strictly less than 25kg. – The pilot has to keep a safety distance between the UAV and people. – The visual contact with the UAV must remain during all the operation. – The UAV cannot carry any dangerous good or drop anything.
To fly in the subcategory A1, the pilot has to : – Be familiarised with the user’s manual of the UAV. – In the case of an unmanned aircraft class C1, has completed an online training course followed by completing successfully an online theoretical knowledge examination provided by the competent authority or by an entity recognised by the competent authority of the Member State of registration of the UAS operator. The examination shall comprise 40 multiple-choice questions distributed appropriately across the following subjects: air safety, airspace restriction, aviation regulation, human performance limitations, operational procedures, UAS general knowledge, privacy and data protection, insurance, security.
The operation must be conducted in such a way that a remote pilot of the unmanned aircraft does not overfly assemblies of people and reasonably expects that no uninvolved person will be overflown. In the event of unexpected overflight of uninvolved persons, the remote pilot shall reduce as much as possible the time during which the unmanned aircraft overflies those persons. (Fly over people not intentionally)
UAV Class C0
An UAV is classified as C0 if it respects the following limits : – Weighing less than 250gr. – Maximum speed less than 19m/s. – Be powered by electricity and have a nominal voltage not exceeding 24 V direct current (DC) or the equivalent alternating current (AC) voltage; its accessible parts shall not exceed 24 V DC or the equivalent AC voltage; internal voltages shall not exceed 24 V DC or the equivalent AC voltage unless it is ensured that the voltage and current combination generated does not lead to any risks or harmful electric shocks even when the UAS is damaged. – If equipped with a follow-me mode and when this function is on, stay in a range not exceeding 50 m from the remote pilot, and make it possible for the remote pilot to regain control of the UA. – The UAV must be marked with the following figure :
UAV Class C1
An UAV is classified as C1 if it respects the following limits : – Weighing less than 900gr. – Maximum vertical speed less than 19m/s. – Be powered by electricity and have a nominal voltage not exceeding 24 V direct current (DC) or the equivalent alternating current (AC) voltage; its accessible parts shall not exceed 24 V DC or the equivalent AC voltage; internal voltages shall not exceed 24 V DC or the equivalent AC voltage unless it is ensured that the voltage and current combination generated does not lead to any risks or harmful electric shocks even when the UAS is damaged. – Unless it is a fixed-wing UA, have a guaranteed A-weighted sound power level LWA. – have a direct remote identification that:
Allows the upload of the UAS operator registration number and exclusively following the process provided by the registration system;
Ensures, in real time during the whole duration of the flight, the direct periodic broadcast from the UA using an open and documented transmission protocol, of the following data, in a way that they can be received directly by existing mobile devices within the broadcasting range:
The UAS operator registration number;
The unique physical serial number of the UA compliant with standard ANSI/CTA-2063;
The geographical position of the UA and its height above the surface or take-off point;
The route course measured clockwise from true north and ground speed of the UA; and
The geographical position of the remote pilot or, if not available, the take-off point;
Ensures that the user cannot modify the data mentioned under paragraph (b);
– Be equipped with the following security systems :
Be equipped with a geo-awareness system that provides:
An interface to load and update data containing information on airspace limitations related to UA position and altitude imposed by the geographical zones, as defined by Article 15 of Implementing Regulation (EU) 2019/947, which ensures that the process of loading or updating such data does not degrade its integrity and validity;
A warning alert to the remote pilot when a potential breach of airspace limitations is detected;
Information to the remote pilot on the UA’s status as well as a warning alert when its positioning or navigation systems cannot ensure the proper functioning of the geo-awareness system;
Clear warning when battery reaches low level;
Be equipped with lights for the purpose of: (a) the controllability of the UA, (b) the conspicuity of the UA at night, the design of the lights shall allow a person on the ground to distinguish the UA from a manned aircraft;
– The UAV must be marked with the following figure :
To fly in subcategory A2 the pilot has to : – Meet all the sub-category A1 requirements. – Complete a self-practical training in the operating conditions of the subcategory A3. – Pass an additional theoretical examination that shall comprise at least 30 multiple-choice questions aimed at assessing the remote pilot’s knowledge of the technical and operational mitigations for ground risk, distributed appropriately across the following subjects: meteorology, UAS flight performance, technical and operational mitigations for ground risk.
The operation must be conducted in such a way that the unmanned aircraft does not overfly uninvolved persons and the UAS operations take place at a safe horizontal distance of at least 30 metres from them; the remote pilot may reduce the horizontal safety distance down to a minimum of 5 metres from uninvolved persons when operating an unmanned aircraft with an active low speed mode function and after evaluation of the situation regarding. (Fly close to people).
UAV Class C2
An UAV is classified as C2 if it respects the following limits : – Weighing less than 4kg. – Unless it is a fixed-wing UA, be equipped with a low-speed mode selectable by the remote pilot and limiting the maximum cruising speed to 3 m/s maximum. – Be powered by electricity and have a nominal voltage not exceeding 48 V direct current (DC) or the equivalent alternating current (AC) voltage; its accessible parts shall not exceed 48 V DC or the equivalent AC voltage; internal voltages shall not exceed 48 V DC or the equivalent AC voltage unless it is ensured that the voltage and current combination generated does not lead to any risks or harmful electric shocks even when the UAS is damaged. – Unless it is a fixed-wing UA, have the indication of the guaranteed A-weighted sound power level affixed on the UA and/or its packaging. – Have a direct remote identification. – Be equipped with C1 security systems and the following ones :
In the case of a tethered UA, have a tensile length of the tether that is less than 50 m and a mechanical strength that is no less than:
For heavier-than-air aircraft, 10 times the weight of the aerodyne at maximum mass;
For lighter-than-air aircraft, 4 times the force exerted by the combination of the maximum static thrust and the aerodynamic force of the maximum allowed wind speed in flight;
Unless tethered, in case of a loss of data link, have a reliable and predictable method for the UA to recover the data link or terminate the flight in a way for reducing the effect on third part in the air or on the ground;
Unless tethered, be equipped with a data link protected against unauthorised access to the command and control functions;
If the UA has a function that limits its access to certain airspace areas or volumes, this function shall operate in such a manner that it interacts smoothly with the flight control system of the UA without adversely affecting flight safety; in addition, clear information shall be provided to the remote pilot when this function prevents the UA from entering these airspace areas or volumes;
– The UAV must be marked with the following figure :
– No more requirements for the pilot in subcategory A3. – The operation must be conducted in an area where the remote pilot reasonably expects that no uninvolved person will be endangered within the range where the unmanned aircraft is flown during the entire time of the UAS operation. – The operation has to be conducted at a safe horizontal distance of at least 150 meters from residential, commercial, industrial or recreational areas. (Fly far from people)
UAV Class C3
An UAV is classified as C3 if it respects the following limits : – Weighing less than 25kg. – Have characteristic dimension of 3m maximum. – Be powered by electricity and have a nominal voltage not exceeding 48 V direct current (DC) or the equivalent alternating current (AC) voltage; its accessible parts shall not exceed 48 V DC or the equivalent AC voltage; internal voltages shall not exceed 48 V DC or the equivalent AC voltage unless it is ensured that the voltage and current combination generated does not lead to any risks or harmful electric shocks even when the UAS is damaged. – Unless it is a fixed-wing UA, have the indication of the guaranteed A-weighted sound power level affixed on the UA and/or its packaging. – Have a direct remote identification. – Be equipped with C2 security systems. – The UAV must be marked with the following figure :
UAV Class C4
An UAV is classified as C4 if it respects the following limits : – Weighing less than 25kg. – The UAV must be marked with the following figure :
Category : “Specific”
An operation can be considered as Specific, it must stay in these limits : – maximum characteristic dimension up to 3 meters in VLOS over controlled ground area except over assemblies of people. – maximum characteristic dimension up to 1 meter in VLOS except over assemblies of people. – maximum characteristic dimension up to 1 meter in BVLOS over sparsely populated areas. – maximum characteristic dimension up to 3 meters in BVLOS over controlled ground area. – Maximum height less than 120m.
The requirements for the category Specific : get an authorisation or enter a declaration or be in possession of a Light UAS Certificate. An application for authorisation must include a risk analysis and measures to mitigate those risks. The declaration containsa/an : – Administrative informations – Standard scenario – Engagement of the pilot to take all the necessary security measures – Insurance confirmation The application for LUC (Light UAS Certificate) contains : – A description of the UAS operator’s management system, including organisational structure and safety management system. – The name of the responsible UAS operator’s personnel, including the person responsible for authorising operations. – A statement that all the documentation submitted to the competent authority has been verified.
Category : “Certified”
Operations in the Certified category should, as a principle, be subject to rules on certification of the operator, and the licensing of remote pilots, in addition to the certification of the aircraft pursuant to Delegated Regulation.
This category is made for dangerous operations as : – People overflown. – People transport. – Dangerous goods transport.
Written by: Kevin Neefs, Researcher at Hénallux and holder of a Belgian class 1 license. Dr. Rim Slama,