TY - JOUR
T1 - Taxonomic hierarchical loss function for enhanced crop and weed phenotyping in multi-task semantic segmentation
AU - Picon, Artzai
AU - Mugica, Daniel
AU - Eguskiza, Itziar
AU - Bereciartua-Perez, Arantza
AU - Romero, Javier
AU - Jimenez, Carlos Javier
AU - Klukas, Christian
AU - Gomez-Zamanillo, Laura
AU - Eggers, Till
AU - Navarra-Mestre, Ramon
N1 - Publisher Copyright:
© 2025 The Author(s)
PY - 2025/3
Y1 - 2025/3
N2 - Herbicide research and development necessitate specific trials to monitor the effects of various herbicide formulations, quantities, and protocols on different plant species and growth stages. These trials are necessary to ensure the safety and efficacy of the developed products. Currently, these tests are conducted manually and assessed visually, making the process time-consuming and labor-intensive. Developing a computer model to characterize species, damage, and growth stages is challenging due to the fine-grained differences between species and damage, significant intra-class variability, and difficulties in manual annotations. Additionally, manually annotated datasets for semantic segmentation are often imperfect. The presence of non-target or unknown species, where only the genus or family is known, complicates the management and scalability of these datasets. In this work, we propose a new hierarchical loss function, suitable for semantic segmentation tasks, capable to take advantage for the hierarchical taxonomy relationships between species, plant damages and other relationships and thus, reduce the need for annotated data. The proposed loss function support datasets with varying granularity and annotation heterogeneity, including for partial annotations at the pixel level. We validated this loss function using a multi-task semantic segmentation neural network to simultaneously detect plant species and quantify the damage of each species. The proposed hierarchical loss function improves model performance, increasing the F1-Score for species detection from 0.41 to 0.52, for damage detection from 0.23 to 0.28. This enhancement forces the model to learn richer hierarchical representations, enabling the support of heterogeneous and partially annotated scalable datasets, which are common in real-world AI applications.
AB - Herbicide research and development necessitate specific trials to monitor the effects of various herbicide formulations, quantities, and protocols on different plant species and growth stages. These trials are necessary to ensure the safety and efficacy of the developed products. Currently, these tests are conducted manually and assessed visually, making the process time-consuming and labor-intensive. Developing a computer model to characterize species, damage, and growth stages is challenging due to the fine-grained differences between species and damage, significant intra-class variability, and difficulties in manual annotations. Additionally, manually annotated datasets for semantic segmentation are often imperfect. The presence of non-target or unknown species, where only the genus or family is known, complicates the management and scalability of these datasets. In this work, we propose a new hierarchical loss function, suitable for semantic segmentation tasks, capable to take advantage for the hierarchical taxonomy relationships between species, plant damages and other relationships and thus, reduce the need for annotated data. The proposed loss function support datasets with varying granularity and annotation heterogeneity, including for partial annotations at the pixel level. We validated this loss function using a multi-task semantic segmentation neural network to simultaneously detect plant species and quantify the damage of each species. The proposed hierarchical loss function improves model performance, increasing the F1-Score for species detection from 0.41 to 0.52, for damage detection from 0.23 to 0.28. This enhancement forces the model to learn richer hierarchical representations, enabling the support of heterogeneous and partially annotated scalable datasets, which are common in real-world AI applications.
KW - Deep learning
KW - Plant species and damage segmentation
KW - Precise phenotyping
KW - Precision agriculture
KW - Taxonomic hierarchical loss
UR - http://www.scopus.com/inward/record.url?scp=85214090422&partnerID=8YFLogxK
U2 - 10.1016/j.atech.2024.100761
DO - 10.1016/j.atech.2024.100761
M3 - Article
AN - SCOPUS:85214090422
SN - 2772-3755
VL - 10
JO - Smart Agricultural Technology
JF - Smart Agricultural Technology
M1 - 100761
ER -