Over the last decades, the European transmission system has made many profound changes in the network and has focused on three main concepts: i) flexibility, ii) integration, and iii) sustainability to increase the technological innovations, and to improve the market design. The currently used method by transmission system operators (TSOs) is trying to accomplish these requirements, but it is important to realize that each TSO has its own grid protocols and standards. Consequently, all major TSOs in the interconnected meshed European transmission system are facing a huge difficulty in maintaining a strong operational coordination to work together as a one single European technical market model. In order to guarantee the highest security of electricity supply, it is necessary to structure a stable, reliable and secure analytical AC framework that takes into consideration the stochastic nature of system in-feeds in the daily operational planning. In this thesis it is analyzed how incorporation of smart technologies such as HVDC transmission can be used as a smart grid solution to improve the power system security and lower the risk in different adjacent areas/zones. The proposed risk-based security assessment (RBSA) methodology based on Monte-Carlo sampling is employed to investigate the security of the system and to quantify the expected system risk. It is shown that the market optimal HVDC power set-points may result in unnecessarily high risk when subjected to the unavoidable uncertainty of inputs (fluctuations in load and RES) inherent to day-ahead forecasting. A detailed comparison of market optimal versus security optimal HVDC power set-point is presented. It is proposed to properly adapt the HVDC set-points with respect to the actual operating situation, which can be quite different from the day-ahead point forecast. Moreover, it is shown that by being able to adapt HVDC set-points in realtime operation, further more serious and more costly remedial actions such as active re-dispatch and load shedding, can be avoided. Furthermore, a study with two HVDC transmission lines is performed to show the necessity of coordinated control of the HVDC lines, and how this can reduce the stress in the network by acting as a tool to shift generation.