Video annotation is the gold-standard method to assess Freezing of Gait (FOG) in Parkinsonian disorders, but it is time-consuming. Deep learning (DL)-based assessment of FOG using inertial measurement units ameliorates these problems but poses challenges. Particularly, the large heterogeneity between patients and assessment methods potentially affects detection performance between independent cohorts. To evaluate heterogeneity effects, we developed a DL model on a local cohort (85 participants; 2043 trials) and validated it across six external cohorts (256 participants; 1058 trials). Model-expert agreement on the percentage-of-time-frozen was strong locally (ICC=0.886 [0.79,0.90]) but reduced in external cohorts (ICC=0.562±0.141). Fine-tuning the DL model with just 50 minutes of external cohort data improved the ICC to 0.732±0.138, falling within the borderline of the inter-rater agreement (ICC=0.73-0.99). Therefore, while unified standards are still being developed, we propose an expert-in-the-loop workflow as an effective intermediary and present a proof-of-concept web-based platform for fine-tuning and expert review (aidfog.be).