SS
S. Stroobants
info
Please Note
<p>This page displays the records of the person named above and is not linked to a unique person identifier. This record may need to be merged to a profile.</p>
9 records found
1
There exists a wide array of possible applications for small, safe, cost-effective, and energy-efficient drones.
However, their development is hampered by limited payload capacity, which restricts both computational power and flight time.
Traditional control systems and sensor processing algorithms are ill-suited for these resource-constrained platforms since they typically rely on power-hungry processors and complex numerical methods.
This thesis investigates neuromorphic approaches to both state estimation and control for small drones.
Inspired by the energy-efficient and highly parallel processing of biological neural systems, neuromorphic computing leverages spiking neural networks (SNNs) that operate via discrete spikes, offering real-time, low-power processing capabilities for micro aerial vehicles (MAVs).
While previous work has applied neuromorphic methods to high-level perception tasks, their application to fundamental flight control -- such as precise attitude estimation and low-level control -- remains largely unexplored.
Following a review of the current state of neuromorphic computing, the research first explores its application to state estimation.
A recurrent SNN is designed to estimate the drone’s attitude from inertial measurement unit (IMU) data, achieving performance comparable to conventional methods like the complementary filter, despite employing a minimal network architecture.
The study then investigates event-based vision sensors by processing data from a downward-facing event camera to estimate the attitude and angular rates, enabling a quadrotor to achieve flight without inertial sensing -- a pioneering demonstration in the field.
Transitioning from estimation to control, the thesis uses neuromorphic algorithms to perform low-level control tasks.
A spiking PID controller is developed using a fixed network architecture, demonstrating altitude control using Intel's Loihi neuromorphic processor.
To address the challenge of precise integration inherent in spiking systems, the Input-Weighted Threshold Adaptation (IWTA) mechanism is introduced.
This innovative approach allows for precise integration of incoming signals and was used as the integral component of a neuromorphic PID controller, mitigating steady-state errors and compensating for sensor biases.
Ultimately, the work unifies estimation and control into a single end-to-end neuromorphic system deployed on a tiny 27g Crazyflie quadrotor. Trained via imitation learning on real flight data, the integrated network maps raw inertial sensor inputs directly to motor commands at a control frequency of 500Hz, achieving attitude tracking performance comparable to traditional controllers.
Overall, this thesis demonstrates that neuromorphic computing is a promising approach for low-level state estimation and control in flying drones, while also addressing the challenges of implementing such systems in real-world environments with sensor biases and persistent disturbances. ...
However, their development is hampered by limited payload capacity, which restricts both computational power and flight time.
Traditional control systems and sensor processing algorithms are ill-suited for these resource-constrained platforms since they typically rely on power-hungry processors and complex numerical methods.
This thesis investigates neuromorphic approaches to both state estimation and control for small drones.
Inspired by the energy-efficient and highly parallel processing of biological neural systems, neuromorphic computing leverages spiking neural networks (SNNs) that operate via discrete spikes, offering real-time, low-power processing capabilities for micro aerial vehicles (MAVs).
While previous work has applied neuromorphic methods to high-level perception tasks, their application to fundamental flight control -- such as precise attitude estimation and low-level control -- remains largely unexplored.
Following a review of the current state of neuromorphic computing, the research first explores its application to state estimation.
A recurrent SNN is designed to estimate the drone’s attitude from inertial measurement unit (IMU) data, achieving performance comparable to conventional methods like the complementary filter, despite employing a minimal network architecture.
The study then investigates event-based vision sensors by processing data from a downward-facing event camera to estimate the attitude and angular rates, enabling a quadrotor to achieve flight without inertial sensing -- a pioneering demonstration in the field.
Transitioning from estimation to control, the thesis uses neuromorphic algorithms to perform low-level control tasks.
A spiking PID controller is developed using a fixed network architecture, demonstrating altitude control using Intel's Loihi neuromorphic processor.
To address the challenge of precise integration inherent in spiking systems, the Input-Weighted Threshold Adaptation (IWTA) mechanism is introduced.
This innovative approach allows for precise integration of incoming signals and was used as the integral component of a neuromorphic PID controller, mitigating steady-state errors and compensating for sensor biases.
Ultimately, the work unifies estimation and control into a single end-to-end neuromorphic system deployed on a tiny 27g Crazyflie quadrotor. Trained via imitation learning on real flight data, the integrated network maps raw inertial sensor inputs directly to motor commands at a control frequency of 500Hz, achieving attitude tracking performance comparable to traditional controllers.
Overall, this thesis demonstrates that neuromorphic computing is a promising approach for low-level state estimation and control in flying drones, while also addressing the challenges of implementing such systems in real-world environments with sensor biases and persistent disturbances. ...
There exists a wide array of possible applications for small, safe, cost-effective, and energy-efficient drones.
However, their development is hampered by limited payload capacity, which restricts both computational power and flight time.
Traditional control systems and sensor processing algorithms are ill-suited for these resource-constrained platforms since they typically rely on power-hungry processors and complex numerical methods.
This thesis investigates neuromorphic approaches to both state estimation and control for small drones.
Inspired by the energy-efficient and highly parallel processing of biological neural systems, neuromorphic computing leverages spiking neural networks (SNNs) that operate via discrete spikes, offering real-time, low-power processing capabilities for micro aerial vehicles (MAVs).
While previous work has applied neuromorphic methods to high-level perception tasks, their application to fundamental flight control -- such as precise attitude estimation and low-level control -- remains largely unexplored.
Following a review of the current state of neuromorphic computing, the research first explores its application to state estimation.
A recurrent SNN is designed to estimate the drone’s attitude from inertial measurement unit (IMU) data, achieving performance comparable to conventional methods like the complementary filter, despite employing a minimal network architecture.
The study then investigates event-based vision sensors by processing data from a downward-facing event camera to estimate the attitude and angular rates, enabling a quadrotor to achieve flight without inertial sensing -- a pioneering demonstration in the field.
Transitioning from estimation to control, the thesis uses neuromorphic algorithms to perform low-level control tasks.
A spiking PID controller is developed using a fixed network architecture, demonstrating altitude control using Intel's Loihi neuromorphic processor.
To address the challenge of precise integration inherent in spiking systems, the Input-Weighted Threshold Adaptation (IWTA) mechanism is introduced.
This innovative approach allows for precise integration of incoming signals and was used as the integral component of a neuromorphic PID controller, mitigating steady-state errors and compensating for sensor biases.
Ultimately, the work unifies estimation and control into a single end-to-end neuromorphic system deployed on a tiny 27g Crazyflie quadrotor. Trained via imitation learning on real flight data, the integrated network maps raw inertial sensor inputs directly to motor commands at a control frequency of 500Hz, achieving attitude tracking performance comparable to traditional controllers.
Overall, this thesis demonstrates that neuromorphic computing is a promising approach for low-level state estimation and control in flying drones, while also addressing the challenges of implementing such systems in real-world environments with sensor biases and persistent disturbances.
However, their development is hampered by limited payload capacity, which restricts both computational power and flight time.
Traditional control systems and sensor processing algorithms are ill-suited for these resource-constrained platforms since they typically rely on power-hungry processors and complex numerical methods.
This thesis investigates neuromorphic approaches to both state estimation and control for small drones.
Inspired by the energy-efficient and highly parallel processing of biological neural systems, neuromorphic computing leverages spiking neural networks (SNNs) that operate via discrete spikes, offering real-time, low-power processing capabilities for micro aerial vehicles (MAVs).
While previous work has applied neuromorphic methods to high-level perception tasks, their application to fundamental flight control -- such as precise attitude estimation and low-level control -- remains largely unexplored.
Following a review of the current state of neuromorphic computing, the research first explores its application to state estimation.
A recurrent SNN is designed to estimate the drone’s attitude from inertial measurement unit (IMU) data, achieving performance comparable to conventional methods like the complementary filter, despite employing a minimal network architecture.
The study then investigates event-based vision sensors by processing data from a downward-facing event camera to estimate the attitude and angular rates, enabling a quadrotor to achieve flight without inertial sensing -- a pioneering demonstration in the field.
Transitioning from estimation to control, the thesis uses neuromorphic algorithms to perform low-level control tasks.
A spiking PID controller is developed using a fixed network architecture, demonstrating altitude control using Intel's Loihi neuromorphic processor.
To address the challenge of precise integration inherent in spiking systems, the Input-Weighted Threshold Adaptation (IWTA) mechanism is introduced.
This innovative approach allows for precise integration of incoming signals and was used as the integral component of a neuromorphic PID controller, mitigating steady-state errors and compensating for sensor biases.
Ultimately, the work unifies estimation and control into a single end-to-end neuromorphic system deployed on a tiny 27g Crazyflie quadrotor. Trained via imitation learning on real flight data, the integrated network maps raw inertial sensor inputs directly to motor commands at a control frequency of 500Hz, achieving attitude tracking performance comparable to traditional controllers.
Overall, this thesis demonstrates that neuromorphic computing is a promising approach for low-level state estimation and control in flying drones, while also addressing the challenges of implementing such systems in real-world environments with sensor biases and persistent disturbances.
The real-world application of small drones is mostly hampered by energy limitations. Neuromorphic computing promises extremely energy-efficient AI for autonomous flight but is still challenging to train and deploy on real robots. To reap the maximal benefits from neuromorphic computing, it is necessary to perform all autonomy functions end-to-end on a single neuromorphic chip, from low-level attitude control to high-level navigation. This research presents the first neuromorphic control system using a spiking neural network (SNN) to effectively map a drone's raw sensory input directly to motor commands. We apply this method to low-level attitude estimation and control for a quadrotor, deploying the SNN on a tiny Crazyflie. We propose a modular SNN, separately training and then merging estimation and control sub-networks. The SNN is trained with imitation learning, using a flight dataset of sensory-motor pairs. Post-training, the network is deployed on the Crazyflie, issuing control commands from sensor inputs at 500Hz. Furthermore, for the training procedure we augmented training data by flying a controller with additional excitation and time-shifting the target data to enhance the predictive capabilities of the SNN. On the real drone, the perception-to-control SNN tracks attitude commands with an average error of 3.0 degrees, compared to 2.7 degrees for the regular flight stack. We also show the benefits of the proposed learning modifications for reducing the average tracking error and reducing oscillations. Our work shows the feasibility of performing neuromorphic end-to-end control, laying the basis for highly energy-efficient and low-latency neuromorphic autopilots.
...
The real-world application of small drones is mostly hampered by energy limitations. Neuromorphic computing promises extremely energy-efficient AI for autonomous flight but is still challenging to train and deploy on real robots. To reap the maximal benefits from neuromorphic computing, it is necessary to perform all autonomy functions end-to-end on a single neuromorphic chip, from low-level attitude control to high-level navigation. This research presents the first neuromorphic control system using a spiking neural network (SNN) to effectively map a drone's raw sensory input directly to motor commands. We apply this method to low-level attitude estimation and control for a quadrotor, deploying the SNN on a tiny Crazyflie. We propose a modular SNN, separately training and then merging estimation and control sub-networks. The SNN is trained with imitation learning, using a flight dataset of sensory-motor pairs. Post-training, the network is deployed on the Crazyflie, issuing control commands from sensor inputs at 500Hz. Furthermore, for the training procedure we augmented training data by flying a controller with additional excitation and time-shifting the target data to enhance the predictive capabilities of the SNN. On the real drone, the perception-to-control SNN tracks attitude commands with an average error of 3.0 degrees, compared to 2.7 degrees for the regular flight stack. We also show the benefits of the proposed learning modifications for reducing the average tracking error and reducing oscillations. Our work shows the feasibility of performing neuromorphic end-to-end control, laying the basis for highly energy-efficient and low-latency neuromorphic autopilots.
In recent years, Artificial Neural Networks (ANN) have become a standard in robotic control. However, a significant drawback of large-scale ANNs is their increased power consumption. This becomes a critical concern when designing autonomous aerial vehicles, given the stringent constraints on power and weight. Especially in the case of blimps, known for their extended endurance, power-efficient control methods are essential. Spiking neural networks (SNN) can provide a solution, facilitating energy-efficient and asynchronous eventdriven processing. In this paper, we have evolved SNNs for accurate altitude control of a non-neutrally buoyant indoor blimp, relying solely on onboard sensing and processing power. The blimp’s altitude tracking performance significantly improved compared to prior research, showing reduced oscillations and a minimal steady-state error. The parameters of the SNNs were optimized via an evolutionary algorithm, using a Proportional- Derivative-Integral (PID) controller as the target signal. We developed two complementary SNN controllers while examining various hidden layer structures. The first controller responds swiftly to control errors, mitigating overshooting and oscillations, while the second minimizes steady-state errors due to nonneutral buoyancy-induced drift. Despite the blimp’s drivetrain limitations, our SNN controllers ensured stable altitude control, employing only 160 spiking neurons.
...
In recent years, Artificial Neural Networks (ANN) have become a standard in robotic control. However, a significant drawback of large-scale ANNs is their increased power consumption. This becomes a critical concern when designing autonomous aerial vehicles, given the stringent constraints on power and weight. Especially in the case of blimps, known for their extended endurance, power-efficient control methods are essential. Spiking neural networks (SNN) can provide a solution, facilitating energy-efficient and asynchronous eventdriven processing. In this paper, we have evolved SNNs for accurate altitude control of a non-neutrally buoyant indoor blimp, relying solely on onboard sensing and processing power. The blimp’s altitude tracking performance significantly improved compared to prior research, showing reduced oscillations and a minimal steady-state error. The parameters of the SNNs were optimized via an evolutionary algorithm, using a Proportional- Derivative-Integral (PID) controller as the target signal. We developed two complementary SNN controllers while examining various hidden layer structures. The first controller responds swiftly to control errors, mitigating overshooting and oscillations, while the second minimizes steady-state errors due to nonneutral buoyancy-induced drift. Despite the blimp’s drivetrain limitations, our SNN controllers ensured stable altitude control, employing only 160 spiking neurons.
Journal article
(2024)
-
F. Paredes-Vallés, J.J. Hagenaars, J. Dupeyroux, S. Stroobants, Y. Xu, G.C.H.E. de Croon
Biological sensing and processing is asynchronous and sparse, leading to low-latency and energy-efficient perception and action. In robotics, neuromorphic hardware for event-based vision and spiking neural networks promises to exhibit similar characteristics. However, robotic implementations have been limited to basic tasks with low-dimensional sensory inputs and motor actions because of the restricted network size in current embedded neuromorphic processors and the difficulties of training spiking neural networks. Here, we present a fully neuromorphic vision-to- control pipeline for controlling a flying drone. Specifically, we trained a spiking neural network that accepts raw event-based camera data and outputs low-level control actions for performing autonomous vision-based flight. The vision part of the network, consisting of five layers and 28,800 neurons, maps incoming raw events to ego-motion estimates and was trained with self-supervised learning on real event data. The control part consists of a single decoding layer and was learned with an evolutionary algorithm in a drone simulator. Robotic experiments show a successful sim-to- real transfer of the fully learned neuromorphic pipeline. The drone could accurately control its ego-motion, allowing for hovering, landing, and maneuvering sideways—even while yawing at the same time. The neuromorphic pipeline runs on board on Intel’s Loihi neuromorphic processor with an execution frequency of 200 hertz, consuming 0.94 watt of idle power and a mere additional 7 to 12 milliwatts when running the network. These results illustrate the potential of neuromorphic sensing and processing for enabling insect-sized intelligent robots.
...
Biological sensing and processing is asynchronous and sparse, leading to low-latency and energy-efficient perception and action. In robotics, neuromorphic hardware for event-based vision and spiking neural networks promises to exhibit similar characteristics. However, robotic implementations have been limited to basic tasks with low-dimensional sensory inputs and motor actions because of the restricted network size in current embedded neuromorphic processors and the difficulties of training spiking neural networks. Here, we present a fully neuromorphic vision-to- control pipeline for controlling a flying drone. Specifically, we trained a spiking neural network that accepts raw event-based camera data and outputs low-level control actions for performing autonomous vision-based flight. The vision part of the network, consisting of five layers and 28,800 neurons, maps incoming raw events to ego-motion estimates and was trained with self-supervised learning on real event data. The control part consists of a single decoding layer and was learned with an evolutionary algorithm in a drone simulator. Robotic experiments show a successful sim-to- real transfer of the fully learned neuromorphic pipeline. The drone could accurately control its ego-motion, allowing for hovering, landing, and maneuvering sideways—even while yawing at the same time. The neuromorphic pipeline runs on board on Intel’s Loihi neuromorphic processor with an execution frequency of 200 hertz, consuming 0.94 watt of idle power and a mere additional 7 to 12 milliwatts when running the network. These results illustrate the potential of neuromorphic sensing and processing for enabling insect-sized intelligent robots.
Neuromorphic processing promises high energy efficiency and rapid response rates, making it an ideal candidate for achieving autonomous flight of resource-constrained robots. It can be especially beneficial for complex neural networks as are used for high-level visual perception. However, fully neuromorphic solutions also need to tackle low-level control tasks. Remarkably, it is currently still challenging to replicate even basic low-level controllers such as proportional-integral-derivative (PID) controllers. Specifically, it is difficult to incorporate the integral and derivative parts. To address this problem, we propose a neuromorphic controller that incorporates proportional, integral, and derivative pathways during learning. Our approach includes a novel input threshold adaptation mechanism for the integral pathway. This Input-Weighted Threshold Adaptation (IWTA) introduces an additional weight per synaptic connection, which is used to adapt the threshold of the post-synaptic neuron. We tackle the derivative term by employing neurons with different time constants. We first analyze the performance and limits of the proposed mechanisms and then put our controller to the test by implementing it on a microcontroller connected to the open-source tiny Crazyflie quadrotor, replacing the innermost rate controller. We demonstrate the stability of our bio-inspired algorithm with flights in the presence of disturbances. The current work represents a substantial step towards controlling highly dynamic systems with neuromorphic algorithms, thus advancing neuromorphic processing and robotics. In addition, integration is an important part of any temporal task, so the proposed Input-Weighted Threshold Adaptation (IWTA) mechanism may have implications well beyond control tasks.
...
Neuromorphic processing promises high energy efficiency and rapid response rates, making it an ideal candidate for achieving autonomous flight of resource-constrained robots. It can be especially beneficial for complex neural networks as are used for high-level visual perception. However, fully neuromorphic solutions also need to tackle low-level control tasks. Remarkably, it is currently still challenging to replicate even basic low-level controllers such as proportional-integral-derivative (PID) controllers. Specifically, it is difficult to incorporate the integral and derivative parts. To address this problem, we propose a neuromorphic controller that incorporates proportional, integral, and derivative pathways during learning. Our approach includes a novel input threshold adaptation mechanism for the integral pathway. This Input-Weighted Threshold Adaptation (IWTA) introduces an additional weight per synaptic connection, which is used to adapt the threshold of the post-synaptic neuron. We tackle the derivative term by employing neurons with different time constants. We first analyze the performance and limits of the proposed mechanisms and then put our controller to the test by implementing it on a microcontroller connected to the open-source tiny Crazyflie quadrotor, replacing the innermost rate controller. We demonstrate the stability of our bio-inspired algorithm with flights in the presence of disturbances. The current work represents a substantial step towards controlling highly dynamic systems with neuromorphic algorithms, thus advancing neuromorphic processing and robotics. In addition, integration is an important part of any temporal task, so the proposed Input-Weighted Threshold Adaptation (IWTA) mechanism may have implications well beyond control tasks.
Compelling evidence has been given for the high energy efficiency and update rates of neuromorphic processors, with performance beyond what standard Von Neumann architectures can achieve. Such promising features could be advantageous in critical embedded systems, especially in robotics. To date, the constraints inherent in robots (e.g., size and weight, battery autonomy, available sensors, computing resources, processing time, etc), and particularly in aerial vehicles, severely hamper the performance of fully-autonomous on-board control, including sensor processing and state estimation. In this work, we propose a spiking neural network capable of estimating the pitch and roll angles of a quadrotor in highly dynamic movements from six-degree of freedom inertial measurement unit data. With only 150 neurons and a limited training dataset obtained using a quadrotor in a real world setup, the network shows competitive results as compared to state-of-the-art, non-neuromorphic attitude estimators. The proposed architecture was successfully tested on the Loihi neuromorphic processor on-board a quadrotor to estimate the attitude when flying. Our results show the robustness of neuromorphic attitude estimation and pave the way toward energy-efficient, fully autonomous control of quadrotors with dedicated neuromorphic computing systems.
...
Compelling evidence has been given for the high energy efficiency and update rates of neuromorphic processors, with performance beyond what standard Von Neumann architectures can achieve. Such promising features could be advantageous in critical embedded systems, especially in robotics. To date, the constraints inherent in robots (e.g., size and weight, battery autonomy, available sensors, computing resources, processing time, etc), and particularly in aerial vehicles, severely hamper the performance of fully-autonomous on-board control, including sensor processing and state estimation. In this work, we propose a spiking neural network capable of estimating the pitch and roll angles of a quadrotor in highly dynamic movements from six-degree of freedom inertial measurement unit data. With only 150 neurons and a limited training dataset obtained using a quadrotor in a real world setup, the network shows competitive results as compared to state-of-the-art, non-neuromorphic attitude estimators. The proposed architecture was successfully tested on the Loihi neuromorphic processor on-board a quadrotor to estimate the attitude when flying. Our results show the robustness of neuromorphic attitude estimation and pave the way toward energy-efficient, fully autonomous control of quadrotors with dedicated neuromorphic computing systems.
Conference paper
(2022)
-
D.A. Olejnik, S. Wang, J.J.G. Dupeyroux, S. Stroobants, M. Karasek, C. de Wagter, G.C.H.E. de Croon
This paper discusses a low-cost, open-source and open-hardware design and performance evaluation of a low-speed, multi-fan wind system dedicated to micro air vehicle (MAV) testing. In addition, a set of experiments with a flapping wing MAV and rotorcraft is presented, demonstrating the capabilities of the system and the properties of these different types of drones in response to various types of wind. We performed two sets of experiments where a MAV is flying into the wake of the fan system, gathering data about states, battery voltage and current. Firstly, we focus on steady wind conditions with wind speeds ranging from 0.5 m S-1 to 3.4 m S-1. During the second set of experiments, we introduce wind gusts, by periodically modulating the wind speed from 1.3 m S−1 to 3.4 m S−1 with wind gust oscillations of 0.5 Hz, 0.25 Hz and 0.125 Hz. The “Flapper” flapping wing MAV requires much larger pitch angles to counter wind than the “CrazyFlie” quadrotor. This is due to the Flapper's larger wing surface. In forward flight, its wings do provide extra lift, considerably reducing the power consumption. In contrast, the CrazyFlie's power consumption stays more constant for different wind speeds. The experiments with the varying wind show a quicker gust response by the CrazyFlie compared with the Flapper drone, but both their responses could be further improved. We expect that the proposed wind gust system will provide a useful tool to the community to achieve such improvements.
...
This paper discusses a low-cost, open-source and open-hardware design and performance evaluation of a low-speed, multi-fan wind system dedicated to micro air vehicle (MAV) testing. In addition, a set of experiments with a flapping wing MAV and rotorcraft is presented, demonstrating the capabilities of the system and the properties of these different types of drones in response to various types of wind. We performed two sets of experiments where a MAV is flying into the wake of the fan system, gathering data about states, battery voltage and current. Firstly, we focus on steady wind conditions with wind speeds ranging from 0.5 m S-1 to 3.4 m S-1. During the second set of experiments, we introduce wind gusts, by periodically modulating the wind speed from 1.3 m S−1 to 3.4 m S−1 with wind gust oscillations of 0.5 Hz, 0.25 Hz and 0.125 Hz. The “Flapper” flapping wing MAV requires much larger pitch angles to counter wind than the “CrazyFlie” quadrotor. This is due to the Flapper's larger wing surface. In forward flight, its wings do provide extra lift, considerably reducing the power consumption. In contrast, the CrazyFlie's power consumption stays more constant for different wind speeds. The experiments with the varying wind show a quicker gust response by the CrazyFlie compared with the Flapper drone, but both their responses could be further improved. We expect that the proposed wind gust system will provide a useful tool to the community to achieve such improvements.
The third generation of artificial intelligence (AI) introduced by neuromorphic computing is revolutionizing the way robots and autonomous systems can sense the world, process the information, and interact with their environment. Research towards fulfilling the promises of high flexibility, energy efficiency, and robustness of neuromorphic systems is widely supported by software tools for simulating spiking neural networks, and hardware integration (neuromorphic processors). Yet, while efforts have been made on neuromorphic vision (event-based cameras), it is worth noting that most of the sensors available for robotics remain inherently incompatible with neuromorphic computing, where information is encoded into spikes. To facilitate the use of traditional sensors, we need to convert the output signals into streams of spikes, i.e., a series of events (+1,-1) along with their corresponding timestamps. In this paper, we propose a review of the coding algorithms from a robotics perspective and further supported by a benchmark to assess their performance. We also introduce a ROS (Robot Operating System) toolbox to encode and decode input signals coming from any type of sensor available on a robot. This initiative is meant to stimulate and facilitate robotic integration of neuromorphic AI, with the opportunity to adapt traditional off-the-shelf sensors to spiking neural nets within one of the most powerful robotic tools, ROS.
...
The third generation of artificial intelligence (AI) introduced by neuromorphic computing is revolutionizing the way robots and autonomous systems can sense the world, process the information, and interact with their environment. Research towards fulfilling the promises of high flexibility, energy efficiency, and robustness of neuromorphic systems is widely supported by software tools for simulating spiking neural networks, and hardware integration (neuromorphic processors). Yet, while efforts have been made on neuromorphic vision (event-based cameras), it is worth noting that most of the sensors available for robotics remain inherently incompatible with neuromorphic computing, where information is encoded into spikes. To facilitate the use of traditional sensors, we need to convert the output signals into streams of spikes, i.e., a series of events (+1,-1) along with their corresponding timestamps. In this paper, we propose a review of the coding algorithms from a robotics perspective and further supported by a benchmark to assess their performance. We also introduce a ROS (Robot Operating System) toolbox to encode and decode input signals coming from any type of sensor available on a robot. This initiative is meant to stimulate and facilitate robotic integration of neuromorphic AI, with the opportunity to adapt traditional off-the-shelf sensors to spiking neural nets within one of the most powerful robotic tools, ROS.
The great promises of neuromorphic sensing and processing for robotics have led researchers and engineers to investigate novel models for robust and reliable control of autonomous robots (navigation, obstacle detection and avoidance, etc.), especially for quadrotors in challenging contexts such as drone racing and aggressive maneuvers. Using spiking neural networks, these models can be run on neuromorphic hardware to benefit from outstanding update rates and high energy efficiency. Yet, low-level controllers are often neglected and remain outside of the neuromorphic loop. Designing low-level neuromorphic controllers is crucial to remove the standard PID, and therefore benefit from all the advantages of closing the neuromorphic loop. In this paper, we propose a parsimonious and adjustable neuromorphic PID controller, endowed with a minimal number of 93 neurons sparsely connected to achieve autonomous, onboard altitude control of a quadrotor equipped with Intel's Loihi neuromorphic chip. We successfully demonstrate the robustness of our proposed network in a set of experiments where the quadrotor is requested to reach a target altitude from take-off. Our results confirm the suitability of such low-level neuromorphic controllers, ultimately with a very high update frequency.
...
The great promises of neuromorphic sensing and processing for robotics have led researchers and engineers to investigate novel models for robust and reliable control of autonomous robots (navigation, obstacle detection and avoidance, etc.), especially for quadrotors in challenging contexts such as drone racing and aggressive maneuvers. Using spiking neural networks, these models can be run on neuromorphic hardware to benefit from outstanding update rates and high energy efficiency. Yet, low-level controllers are often neglected and remain outside of the neuromorphic loop. Designing low-level neuromorphic controllers is crucial to remove the standard PID, and therefore benefit from all the advantages of closing the neuromorphic loop. In this paper, we propose a parsimonious and adjustable neuromorphic PID controller, endowed with a minimal number of 93 neurons sparsely connected to achieve autonomous, onboard altitude control of a quadrotor equipped with Intel's Loihi neuromorphic chip. We successfully demonstrate the robustness of our proposed network in a set of experiments where the quadrotor is requested to reach a target altitude from take-off. Our results confirm the suitability of such low-level neuromorphic controllers, ultimately with a very high update frequency.