Automated bin-picking is a difficult task that requires solving multiple robotic vision problems including object detection and grasp proposal generation. Current methods use deep learning to approach each of the vision problems of bin-picking separately with the main focus on generating the grasp proposals. For grasp proposal generation, neural networks are taught to detect grasp locations by directly proposing or scoring locations. The problem with this method is that it binds the network to a specific decision making process that is encoded into the dataset. We propose a novel architecture we call map-grasp that does not propose grasp locations but instead produces property maps describing measurable, physical object properties while simultaneously detecting and classifying items in an image. We devised a method to automatically generate a dataset containing these property maps from a point cloud, from which we can then determine a grasping location based on custom criteria. We train our network on a training set containing 1845 images and then test it on a physical system where we attempt over 700 grasps on 66 different items. We compare our performance on a current state-of-the-art analytical method and show that our network can outperform it, achieving an 82.16% first-try success rate and a 89.73% total clear rate.

As e-commerce continues to grow, warehouses face the challenge of keeping up with product flows. Vanderlande provides a pick-and-place robot (SIR) that helps in improving capacity. However, SIR's capacity is limited by its vacuum gripper and its inability to solve failures. The aim of this thesis is to improve SIR by adding a second gripper and reasoning capabilities. To this end, a grasp synthesis approach is developed that complements the vacuum gripper, based on existing approaches in literature and the devices available to SIR. Furthermore, an ontology is developed according to Methontology that allows the robot to reason about item properties and failures, while providing operators with insight into its thought process. The improved system is referred to as RSIR. The grasp synthesis approach, applicable to parallel-jaw grippers, synthesizes grasps by detecting parallel edges in depth data. Tests on real depth data showed that the approach synthesized reasonable grasps in reasonable time for a variety of items, some of which are not graspable by SIR's vacuum gripper. It did not need a segmentation of the items to do so. Furthermore, RSIR was tested on a virtual robotic set-up. The results showed that RSIR reasoned how best to complete a task when provided with information about a robot's components and possible item properties. RSIR also drew conclusions based on the execution of the task, and reasoned what actions should be performed to prevent failures from re-occurring. The above results show RSIR's potential for autonomously estimating item properties and selecting the best approach for grasping an item. In practice, this decreases downtime and thus improves capacity. Because RSIR is designed to be expandable to related tasks, its benefits could be useful in more of Vanderlande's applications than just the SIR system.

Suction based robotic actuators have potential for the bin-picking industry, but are currently not usable due the needed speed, accuracy and ability to handle novel and adversarial objects. An evaluation of the state of the art grasp pipeline developed by Mahler et al. [1] for detecting grasps on novel objects lead us to split the problem of robotic grasp generation into a global and a local component. The state of the art solution had the ability to fill the role of global evaluator leaving a local evaluator to be developed. This local grasp evaluator was obtained by training a neural network on a suction based grasp dataset, which was created using a newly developed annotation tool. The proposed grasp pipeline obtained by combining these two showed a 95.27% pick success rate for a random setup and a 90.28% success rate for solely novel objects.

Robotically manipulating objects can be very challenging when not all of the environment can be fully observed, e.g. in environments which are physically and visually accessible from only a single side. By using multimodal sensory feedback and symbolic reasoning, conclusions can be drawn about the presence of objects that cannot be observed directly. This paper presents the Symbolic Reasoning for Partially Observable Environments (SyRePOE) system, which uses an ontology to maintain its world model and a reasoner to infer information about unobservable objects. SyRePOE is demonstrated in simulation and on a real robot, where it is tasked with stocking a retail shelf.