Artificial intelligence (AI) has become a widely discussed and transformative technology, with its adoption growing across industries to drive insights and impact. In this thesis, we explore how AI methods and algorithms can facilitate the operation of soft-fruit supply chains, u
...
Artificial intelligence (AI) has become a widely discussed and transformative technology, with its adoption growing across industries to drive insights and impact. In this thesis, we explore how AI methods and algorithms can facilitate the operation of soft-fruit supply chains, using strawberries as a case study.
The thesis begins by presenting the general background and various perspectives from related works on how AI and machine learning (ML) have been applied to address problems in agricultural or horticultural practices. This includes tasks that, while not directly optimizing supply strategies, still contribute to solving broader challenges. In a nutshell, this thesis categorizes the scope of study into three scales: the single-fruit scale, the greenhouse scale, and the market scale. Within each scale, we review the existing research, identify knowledge gaps, and introduce robust and applicable methodologies capable of dealing with real-world conditions.
Since no publicly available datasets met the requirements of the research plan, we established several datasets for research on the soft-fruit supply chain through collecting, annotating, and (pre-)processing data. These newly curated datasets not only support the research presented in this thesis but also lay a foundation for future research from various perspectives. Details about these datasets are introduced in Chapter 2. Moreover, we conceptualize the process of gathering longitudinal observations from growth monitoring images as a multiple object tracking (MOT) task. We named the image collection and their MOT annotations as ``The Growing Strawberries (GSD)''. The computer vision challenge that GSD brings are further benchmarked and discussed in Chapter 3. Following this, the core contributions of the thesis is presented from Chapter 3 to 6, each corresponding to a published paper or one currently under review. Finally, Chapter 7 summarizes the research findings, answering the research questions proposed in Chapter 1 and discussing the overall work of the thesis.
We discuss these contributions for each of the three mentioned scales separately:
At the fruit scale, we designed and analyzed novel methodologies to keep track of the fruit growths and to predict key properties, including both external characteristics like ripeness and internal qualities such as sweetness. For the ripeness, we propose to use appearance properties, mainly the hue, as an objective metric to quantify it. For the sweetness, we trained deep neural networks to perform non-destructive prediction using environmental and image data, individually and integrally.
Our employment of color analysis and ML models provides a non-destructive and generalizable manner that ensures consistency when upstream and downstream parties in a supply chain estimate the properties of fruits. Meanwhile, the models perform comparatively with laboratory benchmarks even under imperfect, outdoor data collection. We further demonstrated the model in a mobile app to further facilitate adoption in the field.
By benchmarking state-of-the-art MOT algorithms on GSD, we illustrated the new challenges that are brought by this use case: first, the MOT objects change appearance during the tracking due to their biological development, and second, sparse frame rates introduce irregular movements from image to image. We showcased how fruit properties, such as ripeness, change over its life cycle. The results not only provide quantitative measurements that describe the fruit's biological development, but also depict the pain points of current MOT algorithms' predictions. In the meantime, by quantifying these changes over the biological development, we also retrieval relevant information and datasets to support predictions of the changes.
At the greenhouse scale, we designed a framework that optimizes the timing of fruit harvesting by integrating the aforementioned quantified changes over biological development, based on sequential demands about the desired quantities to be harvested. Essentially, the framework makes fruit-specific decisions on dates of harvests by leveraging the monitoring data. The decisions are thus made to enhance both current and future demand-fulfillment capabilities. At each stage of this framework, we evaluated various methods and discussed their effectiveness in achieving the stage targets. For example, how to process the infield data to achieve coherent functions about the ripeness development, how to predict future changes, how to include different perspectives in the optimization model, and etc. As the decisions are made for each specific fruit, the work also demonstrates significant potential for integration with mobile apps and harvesting robots. On top of that, the information retrieval function can also serve as a standalone application to provide objective fruit-level quality assessment.
At the market scale, we focus on the portfolio optimization of a grower under a widely applied mechanism of the market system: the majority of demands for harvests are predetermined through advance contracts, which also serves as an a priori condition of the solution proposed at the greenhouse level. The local market, with dynamic prices and demands, can be used to save losses from the difference in contracted demands and the actual yield. To mitigate outlying decision failures, we introduced the ``smart predict-then-optimize (SPO)'' method, which trains models to predict future yield and local market prices.
Our results illustrate that SPO loss primarily affects the bias layer in neural networks, contrasting with models trained using mean squared error (MSE). This difference essentially leads to more conservative estimations in decision-making scenarios, and also motivates and highlights the importance of effective MSE-based pre-training. Additionally, our study reveals how SPO loss makes models interact when multiple neural networks are trained to predict decision parameters with diverse functions. This insight expands the applicability of SPO loss across a broader range of use cases and model architectures, underscoring its contribution to the field of decision-focused learning.
In conclusion, this thesis introduces diverse data-driven methodologies to tackle the distinct tasks involved in optimizing fruit supply, using strawberries as a case study. Central to our approach is the effective utilization of data, which serves as the foundation for solutions that span from fruit-level evaluations to market-level planning. By leveraging analytics of non-destructive data, our solutions provide objective estimations of fruit quality, fostering a more consistent shared understanding between sellers and buyers while reducing potential food waste. Overall, these advancements push the boundaries of AI in supporting decision-making during the supply of soft fruits, particularly for smaller growers. The findings not only empower more efficient and sustainable supply chain operations but also highlight the strong potential for many practical real-world applications.