In the near future a large increase in elderly population is expected. One of the solutions to this problem is to introduce service robots which can provide a day to day care, allowing elderly to live longer independently. However performing basic household tasks proposes many challenges as the robot will have to act in a dynamic environment and reuse its previous knowledge to adapt to novel situations. Therefore this thesis introduces a modular and reusable software architecture able to efficiently and robustly solve a variety of complex tasks in household environments. Most of such tasks require robust object manipulation independent of object properties, location and orientation or motion of the object. In order to solve this problem most of the state of the art methods require prior knowledge on object shape properties, optimal points to grasp it as well as the best grasping configurations. In this thesis a novel method is presented that allows manipulation of unknown objects for which no prior information on object properties or environment is necessary and all grasping knowledge is built online. This method consists of locating the object, tracking the object and finally grasping the object using a constant feedback loop from the object tracker. This method together with the software architecture have been implemented and tested on the robot Robby, equipped with a mobile base, a single DOF robot arm and an underactuated gripper. Through numerous tests performed in household environments benefits of using the proposed approach is shown.