· · · Institutional Repository

Many public or open areas are facilitated with cameras at every angle to monitor the security of that area for keeping the citizens safe, which is known as the surveillance system. For this moment, the best solution to approach a safety environment has to be done by human. Even though a human is the most intelligent creature in this world, still there are some shortcomings from the existing way. Because of these kinds of shortcomings, human keeps finding new discoveries to replace them and make the best of it. In order to support this surveillance system, a recognition and tracking system is built to detect an abandoned luggage in the public transportation area such as train central station and airport. The goal of this project is to design and implement an algorithm which will be able to detect abandoned luggage using the captured images or videos from the camera as the input of the system. The algorithm realizes image segmentation and image tracking, creates blobs of objects, labels the blobs and finally gives warning when an abandoned luggage is detected. Also a database is developed to store all the media data.

Since the stone age the human race seeks for strategies to extend its viewing range. With the rise of technology in the twentieth century, cameras are found to be a very useful tool to survey a large area with limited resources. With an increasing numbers of cameras, it becomes more difficult to watch every monitor and prevent incidents in the surveillance area. For the last decades, research seeks for possibilities to automatize the process of video surveillance. For this thesis, we approach the surveillance task from the human perspective: we try to emulate what human operators do when they watch the monitors. To perform this task, state-of-the-art techniques from Computer Vision and Artificial Intelligence are applied. An object tracking technique called P-N Learning is used that enables the tracker to learn from its mistakes. The Java Agent Development Framework (JADE) is used to enable communication between agents in the FIPA Agent Communication Language standard. A surveillance system model is designed that detects suspicious behavior in a non-public area. Its task is to alert the operators about suspicious events to give them the chance to investigate and take action. Two prototype applications are implemented and experiments are conducted to show the performance. We showed the proof-of-concept of a system which is able to emulate operators and can potentially outperform a human being. Once the system knows what is considered suspicious behavior it can be automatically detected.

Object Tracking has been a very active area in the field of C omputer Vision. Over the years, a variety of approaches have been put forth to solve this problem and though many of them have demonstrate considerable success none of them have been completely successful. With more methods being written each day, the evaluation of such systems becomes a very important task. If an evaluation system exists that is able to point out specific flaws in the stage of development, it can lead to a very robust and improved algorithm. This work attempts to create such an evaluation framework. Given an algorithm that detects people and simultaneously tracks them, we evaluate its output by considering the complexity of the input scene. Some videos used for the evaluation are recorded using the Kinect sensor and a benchmark dataset from the PETS workshop is also used. To analyze the performance of the tracking system,the reasons due to which the algorithm might fail are investigated and quantified over the entire video sequence. A set of features called Scene C omplexity Measures are obtained for each input frame. The variability in the algorithm performance is modeled by these complexity measures using various regression models. From the regression statistics, we show that we can compare the performance of two different algorithms and also quantify the relative influence of the scene complexity measures on a given algorithm.

At the Man Machine Interaction research group at the Delft University of Technology research is being done on the subject of aggression detection in trains. The goal of this project is to research different aspects of train surveillance, including video surveillance, but also audio surveillance storyboard based modeling. This thesis discusses the current state of the art methods and techniques that or being applied, or could be applied to the task of automated video surveillance. This work discusses the application to the video surveillance problem of several methods, most notably motion detection, face tracking, face recognition and facial expression analysis.

Video surveillance is found in different areas, such as public buildings, metro stations and miliary areas. These systems have become increasingly cost efficient, allowing for larger systems as well. Traditionally multiple security cameras are positioned throughout the area, and linked to monitor screens. These screens are monitored by security pesonnel. Unfortunately humans suffer from certain flaws when it comes to surveillance. For example, humans may lose focus when nothing happens for a long period of time. Though computers may no be as good at computer vision and reasoning as human operators, they provide different advantages. For instance, computers are capable of working 24 hours a day, 7 hours a week. By using an automated intelligent multi-camera video surveillance system the security personnel could be supported in their work, allowing for less security flaws in monitoring the area. For this Bachelor project the project group has developed such an automated intelligent multi-camera video surveillance system. In particular, the system was developed for monitoring the 'Netherlands Defence Academy' (NLDA) area of the 'Koninklijk Instituut voor de Marine' (KIM). In order to develop the system, a scientific approach was used in combination with an incremental, agile development technique called Scrum. Weekly meetings were held with the goup, the supervisors and the domain experts. The system consists of two applications: a Client written in C++ with OpenCV and a Server application written in Java. The Client application is attached to a security camera, and will then detect moving objects, classify them as either human or non-human, and determine their location relative to the camera. For each detected object it will then transmit this information to the Server application. This Server application gathers the information from the Clients and combines the information into actual objects in the monitored area. It then reasons about these objects by using their history of GPS locations to detect whether suspicious situations are occurring. Examples of such situations are when a person enters a restricted area, when a person suddenly starts running or when a person has been following another person or a period of time. In case of a suspicious situation the security personnel is alarmed, and relevant information is displayed to allow the personnel to take action if required. Ensuring code quality, maintainability and extendability of the system was an important aspect of the project. Using a diverse set of tools we ensured that these properties of the system were maintained throughout the project. It is simple to change image processing modules in the Client, or add new reasoning rules to the Server. This resulted in a much appreciated 5 star rating from the Software Improvement Group. Another important aspect of the system was testing. Using Google Test for C++ and JUnit for Java the parts of the code with simple in- and output behavior were automatically tested. The more complex code parts were tested manually using recorded video, or simulation files generated with a specially developed application. A working beta implementation of the complete system has been delivered as a proof of concept and all initially set requirements were fulfilled.

An increasing number of cameras is being used to monitor a growing range of environments. Consequently, surveillance systems consist of an increasing number of screens to display all incoming video streams, making it difficult for observers to maintain a mental model of spatial relations between videos. A number of systems is developed aiming at improving the observer’s spatial awareness by integrating videos with their spatial context, a 3D model of the monitored environment. In these systems, video content can be viewed from virtual cameras that correspond with the cameras in the real world. To switch between views on certain videos, the observer has to make a transition between the corresponding virtual cameras by some means of navigation. While studying state-of-the-art 3D surveillance systems, we have observed that not much attention has been paid on exploring more sophisticated viewpoint transitions. In this thesis, different classes of camera relations are identified. For each class, a viewpoint transition mechanism is developed with the focus on reducing distortion of the video information during transitions. In order to navigate through the virtual environment, viewpoint transitions have to be initiated to switch between videos. As part of this thesis, a number of concepts have been developed that provide controls to the viewpoint transitions in the form of 3D elements that are added to the virtual environment. We have implemented our navigation concepts in a prototype, which was used for a user study. This thesis concludes with the results from this user study and our vision on future work in the field of context-aware surveillance.

The Acoustic Sensor Network (ASN) has emerged as an important research area, because acoustic sensors can significantly increase situational awareness in many situations. Although little is currently known about acoustic surveillance, Thales Nederland is interested in the potential offered by an ASN in urban environments and specifically for classification. This is because the operational problem is not merely to detect targets, but also to localize and classify them in a robust way. Current radar implementations do not provide enough performance to classify targets in complex urban environments while now acoustic sensors are seen as an extra source of information. Therefore, a challenging project was initiated to investigate the potential and feasibility of a passive ASN to localize and classify targets. Three different kind of targets were investigated for acoustic surveillance: guns (muzzle blast), vehicles (running piston engine) and humans (walking pedestrian). Can a passive ASN be deployed in urban environments to localize and classify them only by their emitted sound? It is a great challenge to cope with the received signals using a passive ASN in urban environment, because signals can - even within the same classes - differ significantly. This leads to a technical challenge when it comes to achieving robust localization and correct classification. Two project objectives were defined. Firstly, extract target information and use propagation models to localize the targets. Secondly, extract features which allow a classification method to discriminate between the different target classes. Experimental data processing had to be designed, implemented and evaluated with measured data for a performance indication. To find target features and to investigate localization possibilities, extensive acoustic analysis is done on the three targets. The emitted energy of the gun was the dominant feature of the muzzle blast. The dominant features of the running piston engine were the harmonics. The walking pedestrian had characteristic time interval features between the footsteps. An experimental framework was designed with a signal processor, localizer and classifier. The signal processor has to process the recorded signal in such a way that the localizer and classifier can use the result. The localizer is designed which can localize targets time-based and power-based. The feature extraction of the classifier provided discriminative features which allowed a classification method to discriminate between the classes. The designed components are combined and experimentally implemented (proof of concept) with four microphones and tested for a system performance indication. The time-based localization performed well, but the power-based localization requires extensive calibration to perform proper. The dominant target features were extracted and allowed an experimental classification tree to discriminate between the classes. Passive acoustic surveillance is possible, but the system performance depends very much on the operational situation (e.g. background noise). The performance mainly depends on the signal to noise ratio (SNR) and the SNR depends on the target class. The potential for localizing and classifying walking pedestrians is very low. Vehicle power-based localization and detection has potential, but good microphone hardware is required. Gun localization and classification has the highest potential and feasibility. Although there are some difficulties, throughout this project it became clear that acoustic sensors are able to provide extra information and features. This can be used to further increase the robustness and integrity of urban surveillance systems.

The deployment of visual surveillance and monitoring systems has reached massive proportions. Consequently, a need to automate the processes involved in retrieving useful information from surveillance videos, such as detecting and counting objects, and interpreting their individual and joint behavior, has emerged. The existing methods targeting such automation and working towards "smart" surveillance solutions largely rely on pattern recognition solutions trained offline in a supervised fashion. In this thesis we address the challenge of minimizing the level of supervision in automating the smart surveillance mechanisms. In particular, we focus on the development of fully autonomous detectors of objects that are typical for the observed scene. Using such solutions, objects can be observed and counted (e.g. pedestrians in a shopping street, cars and trucks on a highway), but also atypical objects can be detected leading to automatic security alerts. The rationale behind our proposed solution is that the characteristics of a scene or objects involved can be learned if sufficient time for observation is given, and if the measured features characterizing the properties of the scene and objects are generic enough. In the first step, we demonstrate how candidates for the typical objects can be separated from the scene background using an existing robust motion segmentation mechanism and our novel shadow removal method. In the second step, the set of candidate objects is filtered using an automatically learned perspective deformation model to remove irrelevant objects and compensate for the imperfections of the motion segmentation and shadow removal processes. Finally, in the third step, a module is developed, in which the candidate typical objects are used as positive training samples to train an advanced typical object detector. The effectiveness of this three-step approach is demonstrated first on the case of a single typical object class present in the observed scene. Then, we also show how the proposed approach can be expanded to simultaneously detect two such object classes, if present.

Air Traffic Control (ATC) centers aim at ensuring safety of aircrafts cruising in their area. The information required to face this mission includes the data provided by primary and Secondary Surveillance Radar (SSR). The first one indicates the presence of an aircraft, whereas the second gives information on its identity and altitude. All aircrafts contain a transponder, which send replies to the secondary radar in a semi-automatic mode, indeed it is an exchange. The increase of the air traffic implies that in a near future the actual SSR radar will not be able to perform correctly, and that requires to improve the quality of the SSR radar. This thesis proposes a possible improvement of the SSR. We propose to replace at reception the rotating antenna by an antenna array to gain spatial diversity, in order to perform beamforming. Given the density of the traffic, high-resolution techniques are mandatory to separate the sources. This is a blind source separation problem, but unlike standard cases, the sources are sending packets (not continuously), the packets do not completely overlap (non-stationary situation), the alphabet is binary but not antipodal ({0, 1} instead of {+1,â1}). And the carrier frequencies are not identical. Among the problems to solve, two main issues are the non- synchronisation of the sources, and the non-calibration of the antenna. This thesis presents new contributions to this field, including the identifiability of parameters and related Cramer-Rao bounds, and the design of receiver algorithms taking into account the specific encoding of the data (such as the MDA and the ZCMA algorithms presented herein). The performance of these algorithms is tested by extensive computer simulations as well as actual measurements; the setup of the experimental platform is also part of the thesis framework.

After The Great Fire in the spring of 2008, the Faculty of Architecture of the TU Delft (Bouwkunde) has equipped its new, temporary lodgings with flexible workspaces for students, teachers and researchers. Together with organisational changes, this spatial solution involves an acute change in what was before a university department based on principles of academic freedom and liberal institutions. Protest was waved aside. This combined intervention involves a ‘revolution from above’, transforming of an ‘old-fashioned’ faculty into an authoritarian organisation.

The steadily growing amount of orbital debris increases the probability and amount of collisions between two objects in orbit about the Earth. These collisions in turn create even more debris, and it is therefore important to keep track of future conjunctions. The U.S. Space Surveillance Network (SSN) uses ground- and space-based sensors to observe and track objects of about 10 cm and larger, of which the orbital information is coded in Two-Line Element (TLE) sets and listed in a catalog currently containing about 20,000 objects, which is partly distributed to the public. Using this TLE data, the Simplified General (and Deep-space) Perturbations 4 (SGP4/SDP4) analytical propagator is used to propagate the orbits of these objects, and includes secular, long-period and short-period perturbations due to the Earth's gravity field including J2, J3 and J4 and resonance effects for 12- and 24-hr orbits, as well as perturbations due to atmospheric drag, solar radiation, and gravitational attraction of the Sun and the Moon. The propagated orbits are used to predict conjunctions of pairs of objects. However, due to the large and increasing amount of objects in the catalog, numerically analysing all pairs would be too time-consuming. Therefore, numerous fast filters and sieves were designed to limit the search space of conjunction analysis, by discarding object pairs that are proven to never be able to conjunct. Four implementations of the classical perigee-apogee filter, next to six sieves with a new fine conjunction detection method, were analysed, implemented, and tested in terms of performance. The filter makes use of the altitude band of an object, and can be applied pre-hand. A method based on minimum and maximum radius determined from ephemerides, was found to be the most accurate and reliable in long-term application, while being able to be fine-tuned to the performance needs of a conjunction analysis process. Increasingly complex sieves are subsequently applied to the ephemerides at each time instance in a time interval, in order to efficiently discard object pairs. Eight possible improvements to the underlying theory and application of these sieves were made, resulting in one optimal combination of these improvements, and eventually resulting in a conjunction analysis system that is almost three times as fast as the best found reference method.