Convolutional Cross-View Pose Estimation

We propose a novel end-to-end method for cross-view pose estimation. Given a ground-level query image and an aerial image that covers the query's local neighborhood, the 3 Degrees-of-Freedom camera pose of the query is estimated by matching its image descriptor to descriptors of local regions within the aerial image. The...

IntrApose: Monocular Driver 6 DOF Head Pose Estimation Leveraging Camera Intrinsics

We present intrApose, a novel method for continuous 6 DOF head pose estimation from a single camera image without prior detection or landmark localization. We argue that using camera intrinsics alongside the intensity information is essential for accurate pose estimation. The proposed head pose estimation framework is crop-aware and scale...

PEM: Perception Error Model for Virtual Testing of Autonomous Vehicles

Even though virtual testing of Autonomous Vehicles (AVs) has been well recognized as essential for safety assessment, AV simulators are still undergoing active development. One particular challenge is the problem of including the Sensing and Perception (S&P) subsystem into the virtual simulation loop in an efficient and effective manner....

Chessboard Focal Plane Array for a CMOS-Integrated Terahertz Camera

This article presents the development of a focal plane array (FPA) for terahertz imaging applications with a near diffraction-limited resolution achieved through a very tight sampling of the focal plane. The antenna array is integrated with direct detectors in a 22-nm CMOS technology and operates from 200 to 600 GHz. The tight sampling of the...

Detecting darting out pedestrians with occlusion aware sensor fusion of radar and stereo camera

Early and accurate detection of crossing pedestrians is crucial in automated driving in order to perform timely emergency manoeuvres. However, this is a difficult task in urban scenarios where pedestrians are often occluded (not visible) behind objects, e.g., other parked vehicles. We propose an occlusion aware fusion of stereo camera and...

Case Study of a Global Shutter CIS--Part 1: Angular Dependency of the Light Sensitivity

This article focuses on the angular dependency of the light sensitivity of a commercially available CMOS camera with a global shutter (storage node (SG) in the charge domain) and shared pixel architecture. The angular dependency is characterized as a function of both the wavelength and the angle of incidence of the incoming light. The...

Drift Reduction for Monocular Visual Odometry of Intelligent Vehicles Using Feedforward Neural Networks

In this paper, an approach for reducing the drift in monocular visual odometry algorithms is proposed based on a feedforward neural network. A visual odometry algorithm computes the incremental motion of the vehicle between the successive camera frames, then integrates these increments to determine the pose of the vehicle. The proposed neural...

Contact Localization of Continuum and Flexible Robot Using Data-driven Approach

Continuum robots such as robotic catheters are increasingly being used in minimally invasive surgery. Compliance contributes to enhanced safety during e.g. catheter insertion, however, estimation of contact force and location may help clinicians avoiding exerting excessive force. Ultimately this could lead to faster and safer interventions....

A Joint Extrinsic Calibration Tool for Radar, Camera and Lidar

We address joint extrinsic calibration of lidar, camera and radar sensors. To simplify calibration, we propose a single calibration target design for all three modalities, and implement our approach in an open-source tool with bindings to Robot Operating System (ROS). Our tool features three optimization configurations, namely using error...

Visual Navigation in Real-World Indoor Environments Using End-to-End Deep Reinforcement Learning

Visual navigation is essential for many applications in robotics, from manipulation, through mobile robotics to automated driving. Deep reinforcement learning (DRL) provides an elegant map-free approach integrating image processing, localization, and planning in one module, which can be trained and therefore optimized for a given environment....

Audiovisual Analysis of Music Performances: Overview of an Emerging Field

In the physical sciences and engineering domains, music has traditionally been considered an acoustic phenomenon. From a perceptual viewpoint, music is naturally associated with hearing, i.e., the audio modality. Moreover, for a long time, the majority of music recordings were distributed through audio-only media, such as vinyl records,...

Automatic Defect Detection of Fasteners on the Catenary Support Device Using Deep Convolutional Neural Network

The excitation and vibration triggered by the long-term operation of railway vehicles inevitably result in defective states of catenary support devices. With the massive construction of high-speed electrified railways, automatic defect detection of diverse and plentiful fasteners on the catenary support device is of great significance for...

The MatchNMingle dataset: A novel multi-sensor resource for the analysis of social interactions and group dynamics in-the-wild during free-standing conversations and speed dates

We present MatchNMingle, a novel multimodal/multisensor dataset for the analysis of free-standing conversational groups and speed-dates in-the-wild. MatchNMingle leverages the use of wearable devices and overhead cameras to record social interactions of 92 people during real-life speed-dates, followed by a cocktail party. To our knowledge,...

Driving characteristics and adaptive cruise control: A naturalistic driving study

With the increasing number of vehicles equipped with Adaptive Cruise Control (ACC), it becomes important to assess its impact on traffic flow efficiency, in particular with respect to capacity and queue discharge rate. Simulation studies and surveys suggest that ACC has both positive and negative effects on traffic flow, but empirical...

Deep Learning Object-Recognition in a Design-to-Robotic-Production and -Operation Implementation

This paper presents a new instance in a series of discrete proof-of-concept implementations of comprehensively intelligent built-environments based on Design-to-Robotic-Production and -Operation (D2RP&O) principles developed at Delft University of Technology (TUD). With respect to D2RP, the featured implementation presents a customized...

Video Acceleration Magnification

The ability to amplify or reduce subtle image changes over time is useful in contexts such as video editing, medical video analysis, product quality control and sports. In these contexts there is often large motion present which severely distorts current video amplification methods that magnify change linearly. In this work we propose a method...

Combined optical sizing and acoustical characterization of single freely-floating microbubbles

In this study we present a combined optical sizing and acoustical characterization technique for the study of the dynamics of single freely-floating ultrasound contrast agent microbubbles exposed to long burst ultrasound excitations up to the milliseconds range. A co-axial flow device was used to position individual microbubbles on a...

Transmission electron imaging in the Delft multibeam scanning electron microscope 1

Our group is developing a multibeam scanning electron microscope (SEM) with 196 beams in order to increase the throughput of SEM. Three imaging systems using, respectively, transmission electron detection, secondary electron detection, and backscatter electron detection are designed in order to make it as versatile as a single beam SEM. This...