Autonomous robots hold great potential for positive impacts on society by applying them to tasks that are hazardous, repetitive, or complex and difficult for humans to perform. To achieve these tasks, autonomous robots require the ability to perceive environmental changes and create corresponding motion plans, which involve a combination of perception and motion planning techniques. Building a perception pipeline that can detect all relevant details in a dynamic environment is challenging and computationally expensive. To address this issue, the raw data output of a distance-measuring sensor can be used as a perception pipeline directly, transferring most of the computational load to the motion planner. However, most motion planning techniques cannot handle this high computational load. The motion planning technique optimization fabrics offers a promising solution, which utilizes a combination of differential equations to design robot behavior with extremely low computational complexity.

This thesis proposes a method for local motion planning that combines the low computational complexity of optimization fabrics with direct sensor integration. Our goal is to develop a method that enables autonomous robots to perceive and respond to changes in their environment quickly and efficiently. Our method, called direct sensor integrated (DSI) optimization fabrics, utilizes collision-avoidance differential equations generated for each raw data point collected from a distance measuring sensor. We adapted the regular optimization fabrics method to incorporate sensor data directly, and our analytical analysis shows that direct sensor integration does not require scaling adjustments to the regular collision-avoidance differential equations. By combining optimization fabrics with direct sensor integration, DSI optimization fabrics offers a promising solution to local motion planning. This method can enable autonomous robots to handle tasks that are challenging for humans to perform without needing a complex perception pipeline.

We conducted several experiments to assess our method, utilizing a simulated LiDAR-equipped point robot. First, we show empirically that an adjustment is required to properly scale the collision-avoidance differential equations, resulting in similar collision-avoidance behavior across sensor resolution. Second, we demonstrate that DSI optimization fabrics is feasible regarding computational complexity, achieving a frequency of 23 Hz even with the maximum sensor resolution of 2048 LiDAR rays. Third, we demonstrate the effect of varying the sensor resolution on performance in multiple goal-reaching scenarios. We measure performance by monitoring the time-to-goal, total path length, minimum clearance from obstacles, and success rates. Fourth, we compare our method to regular optimization fabrics with a simulated perception pipeline in a scenario with static obstacles and a scenario with dynamic obstacles. In both scenarios, we show comparable performance regarding time-to-goal, total path length, minimum clearance from obstacles, and success rates. Finally, we showcase a real-world application of our method.

Mobile manipulators will be deployed in supermarkets for a large variety of tasks, for instance, for restocking products. The operation time of mobile manipulators can be reduced by generating coupled trajectories for the base and the robot's arm. When planning for high Degree of Freedom (DOF) robots, such as a mobile manipulator, in an obstacle-cluttered environment, the graph construction for sampling-based planners is time-consuming. If changes in the environment occur, most sampling-based algorithms reconstruct the entire graph. In some dynamic environments, planning can be simplified by the assumption of an incrementally changing environment; this is a mostly static environment where slight changes occur that do not violate the connectivity of the free configuration space, indicating that a significant part of the graph remains valid.

The main contribution of this thesis is a new motion planning algorithm: the adaptive roadmap algorithm (ARM). ARM is a multi-query sampling-based motion planning algorithm that can locally adapt vertices and edges of the graph to account for incremental changes in the environment to allow faster planning than algorithms that reconstruct the entire graph. ARM generates a 3D grid to represent the workspace. The grid cells are marked as occupied or free based on the presence of obstacles in the environment. To determine what vertices and edges of the roadmap need to be updated due to a change in the occupancy of the 3D grid by an incremental change, ARM assigns the vertices and edges to the 3D grid cells. ARM performs this assignment based on the workspace representations of the vertices and edges of the roadmap by 3D bounding boxes surrounding robot configurations. If the occupancy of one or multiple grid cells is changed due to an obstacle, the algorithm resamples the vertices associated with the occupied grid cells and removes the edges associated with the occupied grid cells. Then, the updated roadmap is used for motion planning, and if additional changes occur, this roadmap update is repeated.

We carried out different experiments in simulation performing coupled motion planning for mobile manipulators. A simplified implementation of ARM, which enables the implementation in the Robot Operating System, reported a 35-40% speedup of the planning time compared to the single-query algorithm rapidly-exploring random tree, which reconstructs the entire graph for every new query or change in the environment. The speedup the simplified implementation gained compared to existing planners will be magnified for the non-simplified ARM as the roadmap adaptation by ARM is 10% faster than by the simplified ARM. Further experiments demonstrated that the algorithm successfully adapted the roadmap for a real-world system, not merely in simulation. We conclude that local roadmap adaptation by our proposed algorithm allows faster planning than algorithms that reconstruct the entire graph for mobile manipulators in incrementally changing environments.

Modern retail stores are increasingly implementing automated systems with the aim of assisting both the customer and employee. Much research is conducted on robotic applications within such environments; one of these applications concerns deploying an autonomous mobile manipulator to assist humans in retail stores. Such a robot requires the ability to cope with unexpected environmental changes, like encountering obstructions in retail store aisles. Generally, the first step in the pipeline of an autonomous mobile robot is focused on gathering and processing environmental information using perception sensors like cameras and LiDARs. Subsequently, this information is used to plan the mobile robot's path, which is typically done without allowing interaction with the environment. For object detection techniques, objects are first localized, after which they are classified towards their object type category. Another approach is to classify objects towards their functional categories, which are commonly referred to as affordances. Research in the field of affordance classification mostly focuses on kitchen, garden and working tools due to the availability of affordance datasets for these objects. However, no work on affordances in retail store environments has been conducted to this date. More specifically, there is no dataset publicly available that allows mobile manipulators to identify how to interact with retail store related objects in such environments. This work has investigated the adaptation of an instance segmentation network to localize and classify objects on floors of retail stores into affordance classes. These affordance classes relate to the functional capabilities of mobile manipulators, like graspable or pushable. To achieve this, an affordance dataset consisting of retail store related objects is essential. To overcome the scarcity in this data, a novel dataset consisting of 3237 images with pixel-level affordance annotations was successfully created using an automated data generation approach. This work has shown that such an approach can be used to minimize human labour in terms of acquiring annotated data drastically. A state-of-the-art instance segmentation network was trained using this synthetically generated data and was tested on both synthetic and real image data. The evaluation revealed that the use of synthetic data for training allows for inference on real image data, yet this may compromise the localization and especially the classification performance. Further, the observation of objects that are assigned to both affordance classes has introduced the aspect of the subjectivity of affordances.

We are living in an aging society, which is putting an increasingly heavy strain on our healthcare system. As people age many become less mobile, leading to loneliness and a deteriorating health. Subsequently elderly often end up in nursing homes; an experience which is unpleasant as well as expensive. Assisting the elderly with robotic devices can help increase their mobility, therefore reducing healthcare costs and increasing elderly satisfaction. This thesis follows up on the master thesis of R. Berci-Hajnovics, where she sets out to design a novel concept for an assistive, motorized shopping trolley aimed at reducing the physical burden of carrying groceries over uneven terrain. In the current thesis, a control system for the proposed assistive shopping trolley is designed with the goal of attenuating disturbances like road inclination and added mass on the trolley’s dynamic behavior. Predictability of its behavior is considered as well in order to gain the confidence of the elderly user. An analysis of potentially suitable control types for implementation in the assistive shopping trolley has been performed, wherein their characteristics are compared on several different aspects relating to their predictability and disturbance attenuating behavior. Based on this analysis, a controller implementing a so called Disturbance Observer (DOB) is deemed best suited for the job. A DOB uses a dynamic model of the controlled plant to estimate the influence of present disturbances on the plant dynamics. This estimation can consequently be used to produce an opposing force, thereby canceling the effects of the disturbances. Furthermore, a DOB includes a filter which provides additional capabilities, such as robustness to uncertainty within the model and filtering of high-frequency noise in the data to further increase its performance. A DOB is most suitable for the current cause due to its overall adequate performance with respect to the imposed criteria, as well as its relatively high disturbance attenuation accuracy due to its ability to use the plant’s dynamics in making an estimation of the disturbances at play. Following the results of the analysis, this thesis proposes the implementation of a controller including a DOB for application in the assistive shopping trolley. In its design, a model of the trolley dynamics is being used, which provides a basis for the controller’s internal dynamic model. This model is furthermore used as a representation of the trolley plant in a stability analysis of the trolley-controller closed-loop behavior. The model is based on the dynamics of an inverted pendulum on wheels, with an additional constraint on the position of the trolley handle to represent the interaction with the human user. By neglecting minor, nonlinear dynamic effects such as variations in the trolley’s orientation and air drag, the general model is simplified to a linear equation. The controller’s internal model is constructed to represent the trolley’s nominal (desired) behavior, determined to be that of an empty trolley on a flat road, by substituting corresponding nominal values in the parameters of the general model. Subsequently, criteria for robust stability of the trolley-controller’s closed loop behavior are determined using a H_∞-approach; In this approach the system’s behavior is analyzed given the ‘worst-case’ uncertainties. Furthermore, the system’s performance with respect to disturbance estimation and noise filtering is analyzed using its loop transfer function. The insights gained from these analyses are used in the design of the controller and to make recommendations for future work. The obtained controller is implemented in the Robotic Operating System (ROS) framework; an open-source robotics platform which makes use of a decentralized system of processes, simplifying the creation and sharing of complicated software structures across a wide variety of applications. For this purpose, the controller is subdivided into multiple elements, each performing a specific function. These elements, called ‘nodes’, are submitted to a variety of unit tests to verify their proper implementation. The controller’s performance is put to the test in simulation using ROS’s accompanying simulation engine, ‘Gazebo’. The controlled trolley’s acceleration error is compared to that of a regular trolley in several scenario’s, where different types of disturbances are applied. The results show that the controller indeed reduces the effects of disturbances on the trolley’s behavior; the controller is able to recognize the trolley’s changing behavior due to an encountered disturbance and it correctly attenuates its effect by ordering the required compensatory motor torque. More research is required with respect to the effect of uncertainty within the model dynamics to ensure stability and controllability of the trolley system. Moreover, the closed-loop system response should be tested in the presence of various types of noise to determine how well the current results apply when the controller is used in a real-life scenario.