Artificial intelligence (AI) has the potential to revolutionize many industries across the world. However, artificial intelligence systems raise a series of ethical challenges that hamper their sustainable development. These challenges have often been discussed from a philosophical or theoretical perspective, but rarely from the more practical perspective of businesses. The lack of such a perspective for the ICT-sector in the Netherlands has formed a barrier to develop adequate ethical governance mechanisms. Specifically, the scarcity of practical data of ethical challenges and governance of business in the industry has made it difficult to establish the quality of the current code of ethics for the industry and the need for additional ethical governance measures. This research provides a descriptive study of ethical challenges and ethical governance of AI for ICT-businesses in the Netherlands. A form of mixed methodology (i.e., the exploratory sequential method) combines literature, interviews, and a questionnaire to acquire the relevant data. The results of these three methods are triangulated to identify room for improvement in the current code of ethics and identify the need for additional governance measures. Explainability, fairness, safety, and privacy were identified as the most pressing ethical challenges for businesses in the industry. This research also found that the first three of these challenges are currently insufficiently addressed in the code of ethics for the ICT-industry. Moreover, this study found a significantly low level of ethical governance of AI among small and medium-sized enterprises as compared to large companies. Future research should focus on further normative argumentation on principles for explainability, fairness, and safety to improve the code of ethics. Furthermore, this work lays the basis for researching and developing different concrete ethical governance measures of AI for the industry, especially for SMEs.