Addressing the "open world": detecting and segmenting pollen on palynological slides with deep learning

Abstract

Fossil pollen analysis is an "open-world"problem in paleontology for which there is a long-standing need for automated identification and classification. In the open world, categorical classes are imbalanced, test classes are not known a priori, and test data are captured across different domains. Pollen samples capture large numbers of specimens that include both common and abundant types and rare and sometimes novel taxa. Pollen is diverse morphologically and features can be altered during fossilization. Additionally, there is little standardization in the imaging of pollen samples. Therefore, generalized workflows for automated pollen analysis require techniques that are robust to these differences and can work with microscope images. We focus on a critical first step, the initial detection of pollen specimens on a palynological slide and review how existing methods can be employed to build robust and generalizable analysis pipelines. First, we demonstrate how a mixture-of-experts approach - the fusion of a general pollen detector with an expert model trained on minority classes - can be used to address taxonomic biases in detections, particularly the missed detections of rarer pollen types. Second, we demonstrate the efficiency of domain fine-tuning in addressing domain gaps - differences in image magnification and resolution across microscopes and of taxa across different sample sources. Third, we demonstrate the importance of continual learning workflows, which integrate expert feedback, in training detection models from incomplete data. Finally, we demonstrate how cutting-edge segmentation models can be used to refine and clean detections for downstream deep learning classification models.