The last two decades have witnessed increasing awareness of the potential for terrestrial laser scanning (TLS) in forest applications in both public and commercial sectors, along with tremendous research efforts and progress. It is time to inspect the achievements of and the remaining barriers to TLS-based forest investigations, so further research and application are clearly orientated in operational uses of TLS. In such context, the international TLS benchmarking project was launched in 2014 by the European Spatial Data Research Organization and coordinated by the Finnish Geospatial Research Institute. The main objectives of this benchmarking study are to evaluate the potential of applying TLS in characterizing sample plots, to clarify the strengths and the weaknesses of TLS as a measure of forest digitization, and to reveal the capability of recent algorithms for tree-attribute extraction. The project was designed to benchmark the TLS algorithms by processing identical TLS datasets for a standardized set of forest attribute criteria and by evaluating the results through a common procedure respecting reliable references. Benchmarking results reflect large variances in estimating accuracies, which were unveiled through the 18 compared algorithms and through the evaluation framework, i.e., forest complexity categories, TLS data acquisition approaches, tree attributes and evaluating procedures. The results also reveal some best available forest attribute estimates at this time achieved by a couple of groups using their algorithms, which hints at the potential of TLS in forest environments with the hardware currently available. Some results are well expected, while some are new, e.g., the variances of estimating accuracies between single-/multi-scan, the principle of the algorithm designs and the possibility of a computer outperforming human operation. This paper focuses on the conceptual schema to promote the understanding of the benchmarking results, the potential of TLS in forest modelling, and the fundamental components, i.e., the selection of sample plots, the collection of TLS data, the acquisition of reference datasets, the definition of evaluation criteria (including three criteria proposed in this benchmarking), and the development of evaluation structures (e.g., the algorithm performances are investigated through combining two or more criteria). The TLS datasets are set to open data for further research purposes. New developments can be linked to the eighteen algorithms reported in this benchmarking. ... Read more

The last two decades have witnessed increasing awareness of the potential of terrestrial laser scanning (TLS) in forest applications in both public and commercial sectors, along with tremendous research efforts and progress. It is time to inspect the achievements of and the remaining barriers to TLS-based forest investigations, so further research and application are clearly orientated in operational uses of TLS. In such context, the international TLS benchmarking project was launched in 2014 by the European Spatial Data Research Organization and coordinated by the Finnish Geospatial Research Institute. The main objectives of this benchmarking study are to evaluate the potential of applying TLS in characterizing forests, to clarify the strengths and the weaknesses of TLS as a measure of forest digitization, and to reveal the capability of recent algorithms for tree-attribute extraction. The project is designed to benchmark the TLS algorithms by processing identical TLS datasets for a standardized set of forest attribute criteria and by evaluating the results through a common procedure respecting reliable references. Benchmarking results reflect large variances in estimating accuracies, which were unveiled through the 18 compared algorithms and through the evaluation framework, i.e., forest complexity categories, TLS data acquisition approaches, tree attributes and evaluation procedures. The evaluation framework includes three new criteria proposed in this benchmarking and the algorithm performances are investigated through combining two or more criteria (e.g., the accuracy of the individual tree attributes are inspected in conjunction with plot-level completeness) in order to reveal algorithms’ overall performance. The results also reveal some best available forest attribute estimates at this time, which clarify the status quo of TLS-based forest investigations. Some results are well expected, while some are new, e.g., the variances of estimating accuracies between single-/multi-scan, the principle of the algorithm designs and the possibility of a computer outperforming human operation. With single-scan data, i.e., one hemispherical scan per plot, most of the recent algorithms are capable of achieving stem detection with approximately 75% completeness and 90% correctness in the easy forest stands (easy plots: 600 stems/ha, 20 cm mean DBH). The detection rate decreases when the stem density increases and the average DBH decreases, i.e., 60% completeness with 90% correctness (medium plots: 1000 stem/ha, 15 cm mean DBH) and 30% completeness with 90% correctness (difficult plots: 2000 stems/ha, 10 cm mean DBH). The application of the multi-scan approach, i.e., five scans per plot at the center and four quadrant angles, is more effective in complex stands, increasing the completeness to approximately 90% for medium plots and to approximately 70% for difficult plots, with almost 100% correctness. The results of this benchmarking also show that the TLS-based approaches can provide the estimates of the DBH and the stem curve at a 1–2 cm accuracy that are close to what is required in practical applications, e.g., national forest inventories (NFIs). In terms of algorithm development, a high level of automation is a commonly shared standard, but a bottleneck occurs at stem detection and tree height estimation, especially in multilayer and dense forest stands. The greatest challenge is that even with the multi-scan approach, it is still hard to completely and accurately record stems of all trees in a plot due to the occlusion effects of the trees and bushes in forests. Future development must address the redundant yet incomplete point clouds of forest sample plots and recognize trees more accurately and efficiently. It is worth noting that TLS currently provides the best quality terrestrial point clouds in comparison with all other technologies, meaning that all the benchmarks labeled in this paper can also serve as a reference for other terrestrial point clouds sources. ... Read more