3D visual understanding of the surrounding environment is vital for successful mobile robotic tasks such as autonomous navigation or general object interaction. However, current systems have limited perceptual capabilities in the sense that they are not very well adaptable to unknown environments. Human operators, on the other hand, are experts in adapting to previously unknown information. Hence, human-robot teaming in which the human helps the robot to adapt to new environments and the robot assists in automated object recognition to efficiently feed the control environment of the operator is advantageous. In this work, we propose an object recognition and localization system for mobile robots, based on deep learning, and we study the adaptation of the resulting robotic perception to a new environment. We propose two methods to teach the robot a new object category: using prior knowledge and using limited operator input. We conducted several experiments to show the feasibility of proposed methods. © 2018 IEEE.