Bipedal locomotion has been widely studied in recent years, where passive safety (i.e., a biped rapidly brakes without falling) is deemed to be a pivotal problem. To realize safe 3-D walking, existing works resort to nonlinear optimization techniques based on simplified dynamics models, requiring hand-tuned reference trajectories. In this article, we propose to integrate safety constraints into constrained task-space imitation learning, endowing a humanoid robot with adaptive walking capability. Specifically, unlike previous work using nonlinear and coupled capturability dynamics, we first linearize the 3-D capture conditions using appropriate extreme values and then seamlessly incorporate them into constrained imitation learning. Furthermore, we propose novel heuristic rules to define control points, enabling adaptive locomotion learning. The resulting framework allows robots to learn locomotion skills from a few demonstrations efficiently and apply the learned skills to unseen 3-D scenarios while satisfying the constraints for passive safety. Unlike deep enforcement learning, our framework avoids the need of a large number of iterations or sim-to-real transfer. By virtue of the task-space adaptability, the proposed imitation learning framework can reuse collected demonstrations in a new robot platform. We validate our method by hardware experiments on Walker2 robot and simulations on COMAN robot.@en