Large Language Models (LLMs) such as GPT-4 and LLaMA have demonstrated promising performance in fact-checking tasks, particularly in labeling the veracity of claims. However, the real-world utility of such fact-checking systems depends not only on label accuracy but also on the faithfulness of the justifications they provide. Prior work has explored various prompting strategies to elicit reasoning from LLMs, but most studies evaluate these styles in isolation or focus solely on veracity classification, neglecting the impact on explanation quality. This study addresses that gap by investigating how different prompt styles affect both the accuracy and the faithfulness of LLM-generated claim labelling and justifications. Seven established prompting strategies such as Chain-of-Thought, Role-Based, or Decompose-and-Verify, were tested across two datasets (QuanTemp and HoVer) using two efficient models: LLaMA 3.1:8B and GPT-4o-mini. Additionally, two novel prompt variants were introduced and all styles were tested under three label conditions to assess bias and explanation drift.

Large Language Models (LLMs) are becoming more commonplace in today's society. However their adoption rate, especially in the fact checking field, is being slowed down by the distrust in their thinking process and the rationales leading to the results. In crucial moments the justifications behind a verdict are more important than the verdict itself. However, LLMs often produce explanations that are not grounded in the provided evidence, leading to hallucinations and reduced trust in their outputs. This paper aims to show exactly the level the LLMs have reached in both the faithfulness of their explanations, based on some provided facts, and the correctness of their explanations. To investigate this, multiple LLMs are asked to assign a label to a claim based on some evidence provided from two datasets of varying complexity: HoVer and QuanTemp. The outputs are then evaluated both manually and by another LLM to evaluate how well the LLM relates to the evidence and if the LLM hallucinates in some parts of its responses. The results reveal that while some models demonstrate high correctness in label assignment, faithfulness in explanations varies significantly across models and evidence types. The outcomes of this experiment aim to inform both LLM developers and fact-checking researchers about the current limitations of LLMs in response quality while also showing which areas require further improvements to become mainstream.

Large Language Models (LLMs) are increasingly used to generate fact-checking explanations, but evaluating how faithful these justifications are remains a major challenge. In this paper, we examine how well four popular automatic metrics—G-Eval, UniEval, FactCC, and QAGs—capture faithfulness compared to expert-written explanations. We look at how these metrics agree with each other, how they correlate with explanation similarity, and how they respond to controlled errors.Our findings show that while some metrics like UniEval and FactCC show some sensitivity to noise and partial alignment with expert reasoning, none of them reliably catch hallucinations or consistently reflect true faithfulness. Manual analysis also reveals that metric behavior varies depending on the type and structure of the claim. Overall, current metrics are only moderately effective and often biased toward the style of LLM-generated text. This study points to the need for more reliable, context-aware evaluation methods and offers practical insights for improving how we measure faithfulness in fact-checking tasks.

Large Language Models (LLMs) are increasingly transforming how scientists approach research, with emerging tools supporting ideation, experimentation, and publication in attempts to expedite the research process. This work focuses on the foundational first step: generating novel, high-quality research ideas. Building on an existing LLM idea generation method, we introduce a system for personalised research idea generation that generates ideas by analysing a researcher’s publication history, providing tailored research recommendations. We evaluate the generated ideas via expert surveys across predefined metrics, offering insights into the system’s effectiveness and LLMs’ role in ideation in general. Findings show that personalised generation produces novel, interesting, and potentially impactful ideas comparable to baseline methods. Clarity remains a common limitation, while feasibility improves in the personalised variant, highlighting the potential of personalised LLM ideation.

We investigate the application of Retrieval-Augmented Generation (RAG) for enhancing the analysis of corporate sustainability disclosures. We introduce CorSus, a novel dataset for evaluating RAG models in answering corporate sustainability-focused claims, using data from the Transition Pathway Initiative for over 100 companies. Further, we develop a subset of this dataset with reference documentation and fully explained answers. Finally, in a systematic framework, we optimise and benchmark state-of-the-art RAG approaches using the CorSus dataset. With this work, we aim to empower stakeholders with a tool for informed evaluations of corporate sustainability practices, thereby encouraging a greater commitment to environmental responsibility.