Manufacturing companies today face increasing global competition, labor shortages, and pressure to maximize efficiency. Automation is a key strategy for maintaining competitiveness, but determining the optimal level of automation in production line design is complex. Companies must weigh the strengths of human operators, such as flexibility and problem-solving, against the consistency and efficiency offered by machines. Although these decisions are crucial, there is currently no comprehensive methodology that allows companies to systematically evaluate the trade-offs between different levels of automation. This evaluation should integrate both quantitative and qualitative factors during the early design stages of a production line. To achieve a comprehensive understanding, the following key factors have been identified to systematically assess the trade-offs associated with automation options: production performance, cost, quality, flexibility and work environment.

This research adopts a design science methodology to develop and refine a stepwise evaluation framework for automation trade-offs in production line design. The methodology begins with a thorough documentation of the production context, requirements, and constraints. It then systematically generates low, medium, and high automation scenarios for each production step, considering both cognitive and physical tasks. These scenarios are conceptualized using structured models (IDEF0 diagrams) and layout maps, followed by the development of dynamic models, to quantitatively assess system dynamics, throughput, and bottlenecks. Each scenario is evaluated across the five key factors using both model outcomes and expert review sessions, enabling a holistic comparison. The methodology is validated through a practical test case at Quooker, a company developing a new production line, and refined through feedback from industry experts and academic reviewers.

The final version of the designed automation evaluation methodology provides a structured and comprehensive approach for evaluating automation trade-offs in early stage production line design. Its main strengths include the integration of both quantitative and qualitative assessments, the explicit consideration of five key factors, and the ability to generate actionable insights that broaden perspectives beyond intuitive thinking. Application to the Quooker test case demonstrated that the methodology supports informed, data-driven discussions and enables the identification of balanced automation scenarios. Feedback from industry and academic experts confirmed its practical relevance and alignment with industry needs. However, limitations include the time-intensive nature of dynamic modeling, the dependency on expert input, and challenges in quantifying certain factors at early design stages. Despite these constraints, the methodology offers a robust foundation for making informed automation decisions and structuring early design discussions in manufacturing environment.