Face validity indicates to what extent participants are engaged in making choices; and understand and interpret the information presented to them in the study as intended by its designer. It is an important but often overlooked aspect of the overall validity of choice experiments and no comprehensive instruments for assessing it are available. Improving the design of choice experiments potentially improves the quality of participants’ responses, which increases the relevance and usability for policy and practice. In this study we developed and tested an instrument to assess the FAce validity of Choice Experiments (FACE) in a uniform, systematic manner. The instrument is based on nine components that are used to define face validity identified from literature: acceptance, clarity, completeness, familiarity, feasibility, legibility, relevance, sensitivity, and transparency. FACE covers these components in 14 questions with 5-point Likert scales on which participants can indicate their level of agreement. 1,020 participants completed the instrument following a discrete choice experiment on COVID-19 pandemic preparedness measures in the Netherlands. This first application of FACE showed that the face validity of a choice experiment was determined by whether participants considered its study design to be relevant, reliable and feasible. Moreover, we found that relevance and reliability were most strongly related to characteristics of the survey design, while feasibility was most strongly related to participants’ socio-demographic characteristics. Face validity was assessed high(er) by participants who were younger, male, lower educated, vaccinated against COVID-19, supportive of policy responses to a pandemic situation and sufficiently engaged in the experiment.

The importance of public participation in transport planning continues to increase. Participatory Value Evaluation (PVE) is a relatively new instrument to involve citizens in transport planning. In a PVE, citizens are put into the shoes of a policymaker and see which polices are considered, the impacts of the policies and the constraint that the policymaker faces. Subsequently, citizens are asked to advise the policymaker on which policy options should be chosen and why. So far PVE has been successfully applied in contexts in which citizens and stakeholders broadly agreed on the pursued policy goals. It is, however, unclear whether PVE is equally valuable in contexts in which stakeholders have conflicting interests. This paper investigates whether deploying PVE in an institutionalized transport conflict results in similar benefits and costs as in contexts where stakeholders pursue the same goals. We studied a PVE application in which 2466 participants provided recommendations to the government about decision-making regarding Schiphol Airport, the Netherlands. Stakeholders were involved in many phases of and decisions on the PVE design process. The aim of this research was twofold. First, to establish whether the benefits of applying PVE in an institutionalized transport conflict outweigh the costs. Second, to develop an instrument to assess the face validity of PVE as experienced by participants. We find that PVE can provide similar benefits in the context of an institutionalized conflict compared to cases in which such a conflict was absent. Citizens participated who normally do not participate and the PVE produced useful outcomes for decision-making. However, a notable difference is that the design phase of the PVE involved several feedback rounds which made it much more time-intensive than other cases. Hence, the benefit that PVE requires low time investment of civil servants that emerged in other studies was not applicable when applying PVE to an institutionalized transport conflict setting. Furthermore, stakeholders could not agree on various design choices of the PVE. To satisfy the diverging requests of stakeholders a very lengthy PVE was constructed. As a result, the satisfaction among participants about its face validity was relatively low.

Background: Vaccination is generally considered the most direct way to restoring normal life after the outbreak of COVID-19, but the available COVID-19 vaccines are simultaneously embraced and dismissed. Mapping factors for vaccine hesitancy may help the roll-out of COVID-19 vaccines and provide valuable insights for future pandemics. Objectives: We investigate how characteristics of a COVID-19 vaccine affect the preferences of adult citizens in the Netherlands to take the vaccine directly, to refuse it outright, or to wait a few months and first look at the experiences of others. Methods: An online sample of 895 respondents participated between November 4th and November 10th, 2020 in a discrete choice experiment including the attributes: percentage of vaccinated individuals protected against COVID-19, month in which the vaccine would become available and the number of cases of mild and severe side effects. The data was analysed by means of panel mixed logit models. Results: Respondents found it important that a safe and effective COVID-19 vaccine becomes available as soon as possible. However, the majority did not want to be the first in line and would rather wait for the experiences of others. The predicted uptake of a vaccine with the optimal combination of attributes was 87%, of whom 55% preferred to take the vaccine after a waiting period. This latter group tends to be lower-educated. Older respondents gave more weight to vaccine effectiveness than younger respondents. Conclusions: The willingness to take a COVID-19 vaccine is high among adults in the Netherlands, but a considerable proportion prefers to delay their decision to vaccinate until experiences of others are known. Offering this wait-and-see group the opportunity to accept the invitation at a later moment may stimulate vaccination uptake. Our results further suggest that vaccination campaigns targeted at older citizens should focus on the effectiveness of the vaccine.