This work describes an energy efficient solution for secure communication between an Implantable Medical Device (IMD) and a user. For the security part a communication protocol has been selected which supports entity-authentication, message-authentication and confidentiality. For achieving low power and energy consumption the following setup has been chosen. The IMD processor has been partitioned into two modules. The first module supports the main implant function, while the second module is responsible for secure communication with the outside world. In this work we have implemented the latter module: we have designed a 5-stage-pipeline RISC, application specific processor along with its compiler and optimized them to efficiently support the IMD security workload. The processor has been synthesized for a 90nm CMOS ASIC technology. The comparison between the base-line processor and its optimized version resulted in the following improvements: 41.4% decrease in execution time, 11.8% increase in the executed instructions per cycle (IPC) and 37% decrease in energy consumption. These improvements came at the slight cost of 8% increase in power and 7% increase in area.