Taxonomic hierarchical loss function for enhanced crop and weed phenotyping in multi-task semantic segmentation

Artzai Picon*, Daniel Mugica, Itziar Eguskiza, Arantza Bereciartua-Perez, Javier Romero, Carlos Javier Jimenez, Christian Klukas, Laura Gomez-Zamanillo, Till Eggers, Ramon Navarra-Mestre

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

Herbicide research and development necessitate specific trials to monitor the effects of various herbicide formulations, quantities, and protocols on different plant species and growth stages. These trials are necessary to ensure the safety and efficacy of the developed products. Currently, these tests are conducted manually and assessed visually, making the process time-consuming and labor-intensive. Developing a computer model to characterize species, damage, and growth stages is challenging due to the fine-grained differences between species and damage, significant intra-class variability, and difficulties in manual annotations. Additionally, manually annotated datasets for semantic segmentation are often imperfect. The presence of non-target or unknown species, where only the genus or family is known, complicates the management and scalability of these datasets. In this work, we propose a new hierarchical loss function, suitable for semantic segmentation tasks, capable to take advantage for the hierarchical taxonomy relationships between species, plant damages and other relationships and thus, reduce the need for annotated data. The proposed loss function support datasets with varying granularity and annotation heterogeneity, including for partial annotations at the pixel level. We validated this loss function using a multi-task semantic segmentation neural network to simultaneously detect plant species and quantify the damage of each species. The proposed hierarchical loss function improves model performance, increasing the F1-Score for species detection from 0.41 to 0.52, for damage detection from 0.23 to 0.28. This enhancement forces the model to learn richer hierarchical representations, enabling the support of heterogeneous and partially annotated scalable datasets, which are common in real-world AI applications.

Original languageEnglish
Article number100761
JournalSmart Agricultural Technology
Volume10
DOIs
Publication statusPublished - Mar 2025

Keywords

  • Deep learning
  • Plant species and damage segmentation
  • Precise phenotyping
  • Precision agriculture
  • Taxonomic hierarchical loss

Fingerprint

Dive into the research topics of 'Taxonomic hierarchical loss function for enhanced crop and weed phenotyping in multi-task semantic segmentation'. Together they form a unique fingerprint.

Cite this