Print Email Facebook Twitter Methodologies for Assessing the Benefits of Open Standards: The Implications for the Public IT Procurement Title Methodologies for Assessing the Benefits of Open Standards: The Implications for the Public IT Procurement Author Yang, X. Contributor Egyedi, T.M. (mentor) Enserink, B. (mentor) Tan, Y.H. (mentor) Faculty Technology, Policy and Management Department Infrastructure Systems & Services Department Programme Information and Communication Technology Date 2012-07-25 Abstract From economic perspective, standards provide information (e.g. increasing market transparency and reducing transaction costs), foster compatibility (e.g. positive network externalities and increased competition) and reduce variety (facilitating economies of scale and building a critical mass). There are growing debates about whether using open standards in public sector will or will not bring the potential benefits as indicated in qualitative literature, while a common methodology for quantification of benefits of standards is not readily available. This paper will explore the possibility to adopt methodologies used in other field in the context of public IT procurement. We outline our main question as following: What methodologies are used to assess the benefit of open standards at micro-level and macro-level, and to what extent can these methodologies be applied in the context of public IT procurement? This report describes three methodologies: ISO Methodology, econometric modeling, and Cost-Benefit Analysis, for economic impact assessment of standards and gives numerous examples of their application. ISO Methodology ISO Methodology is developed to ex-post evaluate the micro impacts of application of open standards in private sector from organizational perspective. Overview of the pilot cases found main impacts for most companies are reduced transaction cost due to easy access to information, and economies of scale due to reduced number of suppliers/ raw material/ mass productions. From the analysis of ISO Methodology and cases, we can say that ISO Methodology indeed provides a set of methods to measure the impacts on organizational value creation; however it has limitations including difficulty to isolate the impacts of standards to other factors and to estimate accurately, which are inherent problem in quantification of impacts. Econometric Modeling Econometric modeling uses statistical methods to empirically test the relationship between macro economic performance, e.g. economic growth and trade, and stock of standards. Studies have found that standards have a positive influence on economic growth; international standards often have a positive relationship to exports and imports; and there is no single answer to relationship between national standards and trade. However, there are limitations conducting the econometric modeling to test the standards’ impact on macro performance. Firstly, patents and standards could turn out to be highly co-linear, which lead to the elasticity estimates of standards be the upper bound on the estimated contribution of standards to the growth. Econometric modeling does not present us a detailed picture of how standards impact on the macro economic performance; and thirdly the number of standards is taken as a legitimate proxy for the quality of standards without considering the importance and functionality of different standards. Cost-Benefit Analysis Cost-Benefit Analysis is initially developed to evaluate the social effects of infrastructure projects. Cost-Benefit Analysis is ‘social’ in nature and includes all types of effects. CBA is mostly used as ex-ante evaluation. The three standard implementation cases studied show that the most quantitative benefit is improved efficiency; some social impacts are not quantified, such as the main goals of implementation of open standards, i.e. to increase interoperability and reduce the supplier-dependence. Additional research presents us a classification of types of consumer switching cost in the IT field and a simple way to estimate the switching cost just using price and market share, which make it possible to investigate the switching costs as indication of vendor lock-in. Redundant transaction costs could be measured as an indication for imperfect interoperability. Ex ante CBA is always disputable. Firstly, it lies in the difficulties to measure the societal impacts, e.g. put price tag to put price tag to non-price effects and leave qualitative effects aside. This requires careful interpretation of result when benefits are not easily quantifiable in monetary terms. Secondly, as in ex ante CBA, results depend largely on assumption made about the future trend, which is highly complicated and subjectively. Thirdly, it arises from the evaluation process being as a ‘black box’, leaving room for strategic manipulations. Comparative overview Comparison of the three methodologies gives a picture of complementarity. The focus level is complementary. Econometric modeling focuses on macro-level, ISO Methodology studies in micro-level; CBA includes both the micro and/or macro-level. Secondly, the three methodologies together cover both private and public sector. Thirdly, the three methodologies measure the benefits of applying standards at different time points. Moreover, the three methodologies are complementary because they measure different dimensions of benefits of standards. By comparing all quantified benefits to potential benefits indicated in literature, we can find that to date the most observed and recognized benefits of using standards in monetary value are reduction of transaction cost and economies of scale. In addition, switching cost can be measured as indication of magnitude of vendor lock-in; redundant transaction costs can be measure as an indication for imperfect interoperability. For perception of stakeholders, such as trust due to improved quality, environment and policy compliance, it is hard to be quantified to monetary value. The three methodologies suffer from some limitations. For example, it is hard to isolate impacts of standards from other factors; and each methodology can only address part of the effects of standards. Moreover, there are also estimation errors. Partially the limitation is caused by cognitive limitations of analysts; partially it is caused by explicitly or implicitly strategic manipulation and misinterpretation, which should be avoided at best. The three methodologies are complementary and they together cover all fields. Theoretically, the combination of the three methodologies could assess benefits of standards in any field. Depending on the scope of the problem, the author of the study can choose a suitable methodology or combined methodologies to conduct in its specific context. Although certain benefits of standards are not easily quantifiable, the three methodologies provide ways to quantify the benefits of using standards in a systematic way. Qualitative methodologies, such as Multi-Criteria Analysis and Value Measuring Methodology, are able to compare all possible effects of projects, which is helpful to get the perception of stakeholders on different alternatives and support the political decision making. But qualitative methodologies bring more political controversial, as MCA and VMM depends on political weights on criteria. In summary, to choose which methodology depends on the aim and scope of the problem, keep in mind that each methodology has its limitations. Recommendation Firstly, current assessment reports have not quantified the most important objectives of using open standards in public sector, such as increased interoperability and vendor independence. Further studies should include quantification of these effects. These cases should try to quantify all impacts, or as many effects as possible, to give an actual picture on whether there are substantial savings due to increased using of open standards in public sector. More complete cases on value of standards implementation should be done. As studies accumulates, it is suggested to reach a standardized guideline including a list of impacts of standards and possible techniques to measure certain impacts. Secondly, to make it clear the position of quantification methodologies in the decision making process. Incomplete and inaccurate results could be interpreted in different ways. Moreover, in order to ensure the process of quantification be transparent (to make the assumptions and the estimation clear), peer review are needed. Subject Benefits of open standardsQuantification MethodsPublic IT ProcurementISO MethodologyCost Benefit AnalysisEconometric Model To reference this document use: http://resolver.tudelft.nl/uuid:a2ad8b6b-ffe8-406e-b89c-f288c071f6e2 Embargo date 2012-07-25 Access restriction Campus only Part of collection Student theses Document type master thesis Rights (c) 2012 Yang, X.