User feedback plays a significant role in helping recommendation systems to make personalized and accurate predictions. Despite the fact that many methods of collecting user feedback have been proposed, little research exists that addresses both the breadth and depth of data collected. In this study, we incorporate personal reflection into traditional crowdsourcing tasks and investigate how it facilitates people's reflection on their usage of YouTube. We present a novel crowdsourcing approach with personal reflection integration based on several design principles, which allows participants to reflect on contextual factors and personal values of using YouTube through guided context recall and evaluations, therefore gathering deeper insights into people's preferences and behaviors on a large scale. We conducted a user study involving 20 participants and explored the insights generated from their textual answers and the role of design principles in the reflection process. This approach successfully enables multiple participants to conduct the study simultaneously, thereby reflecting on their watching behaviors and preferences. Based on the quantitative and qualitative analysis of the findings, we sum up the implications of this approach to provide guidance for the YouTube recommendation system and point to the directions for the design of similar studies in the future.

Recommender systems are widely used in modern lives and contribute to many industries. Therefore, methods to evaluate and improve them are important. Nowadays, much research has been done to improve the system aspects such as algorithms. However, user experiences are not only affected by the systems but heavily rely on the context when using the systems. Therefore, the research on user aspects to understand their experiences is as important. This study contributes an approach that uses collaborative reflection to find insights into users' experiences with recommender systems. Using this approach, this study presents the influences of context on user experiences with recommender systems. This study investigates the importance of situational and personal contexts like mood, time, and location in shaping user satisfaction with recommendations. The research adopts a method based on collaborative reflection, where participants engage in tasks using their YouTube watch history, paired with another individual for real-time discussion. By analyzing contextual influences and the values users wish to achieve, the study identifies key patterns in user behavior and insights into personal preferences. This research not only contributes to the evaluation of recommender systems but also highlights the need for systems to align with both the goals of users and broader societal values. The usability of the proposed method was tested to be successful in crowdsourcing, yielding practical implications for future evaluation and improvements of recommender systems.

Language is one of the most important factors that international citizens need to acquire when stablishing in a new country, it enables communication, widens opportunities in relation to academic and professional development (Geurts & Lubbers, 2017b), and provides a sense of belonging. Therefore, it is important to understand what enables and inhibits the language learning process to make the integration process more user centered, propose actionable ideas and design effective interventions in order to optimize current language services, which in the end will result in cultural-societal impact (Sosa, 2019).

This project aims to contribute to such language learning services optimization, working together with NLtraining, who is a specialized Dutch school providing language courses for expats and immigrants.

A service design methodology is proposed to help bringing a user centric vision based on students’ real context but also provides a holistic scope including not just students and professors but also content making professionals and administrative departments.
The initial phase of the project explores existing literature from Second Language Acquisition and service design, where it was found that SLA research is centered on designing the ideal learning environment, however, in language learning, interactions in real life represent a great space for learning, therefore a gap between real context and learning environment was found.

Afterwards, exploratory research is carried out, interviewing students from NLtraining coming from different levels, where the main insights show that even though students have great motivation to learn the language and achieve academic and professional goals, their language use in real life is inhibited by different factors: cultural barriers, self-esteem, and deep complex reflective processes. It was found that professors who are the course facilitators have limited access to these kinds of insights, since they make their decisions based on assumptions from what they observe from the course, this, seen from a service design perspective, can become an opportunity to optimize current strategies.

The goal of the project is, then to design a tool that will help bridge students’ experiences through a reflective practice, which will support their learning process, as well as support professors with real insights of their students’ language use. Three exploratory concepts were developed, tested with students, and evaluated from a design perspective, then a final concept is presented, where a digital prototype is built, given its versatility and accessibility, it has the potential to be integrated into NLtraining’s current workload and bridge students’ context to the learning environment.

This tool helps collecting data such as: activity performed (listening, speaking, reading, or writing), description of the experience (text input), related class resources and emotion (good, neutral or not so good).

The prototype is validated in the final stage of the project, where the main findings conclude that enabling such reflective tool can promote students engagement with the course, in first instance because it provides a structured space to share and vent their (positive or negative) experiences, identify elements from the course that are working well or not, and finally allow professors to learn, empathize and best prepare the content of the course, based on a loyal portrait of students’ needs.

This project presents the impact of service design practices in an educational setting, which includes supporting stakeholder communication and collaboration, helping scope gaps in the service based on user insights and enabling an iterative approach to fostering optimizations.