As written in the coverletter, the proposed work is an extension of the recently granted CTI project
27395.1 on brain shift correction for Neurosurgical interventions. The aim of the CTI project is to
develop techniques to determine the brain shift and then to overlay the tumour and other critical
structures onto the surgical microscope’s image. This, however, implies that segmentation of the
tumour, brain surface, vascular tree and the critical structures are available. Segmenting these
often requires substantial manual input for training which is a tedious and time consuming task.
With the research described herein (partially funded by a Novartis FreeNovation Project) we try
to close this gap and go one step beyond. In particular we propose an approach able to learn on
its own how to segment a pathology only on weakly labelled data. In other words our approach is
capable to learn how to segment pathologies from a training set of images with the pathology and
a second set of images without the pathology (i.e. healthy subjects). Such data sets are easy to get
in contrast to the manually labelled data sets required for the state-of-the-art approaches.
The proposed approaches is the first of its kind and inspired by CycleGAN (a DeepLearning
domain transfer approach) . Our approach can model pathologies in medical data trained only
with data labelled on the image level (i.e. healthy vs. diseased). Not only can the model create
pixelwise semantic segmentations of the pathologies it can also create inpaintings (i.e. heal) to
render the pathological image healthy. As a side effect, we can also create new unseen pathological
samples useful for example in training of medical personnel.
In a proof-of-principle study we could recently show that the idea has great potential and might
even be a disruptive technology in image segmentation.
The significance of the proposed project is very high as it might render manual segmentation
unnecessary in the near future. Training the algorithm to recognise and segment new pathologies
would be simple and fast. Imaging CROs could evaluate their drug studies more cost effective and
also faster speeding up the development cycle of new drugs.
In this research proposal we will first give an overview on the principles and limitations of
current in segmentation concepts, followed by an overview of our own research directions in this
field and the detailed research plan. After the project plan and risk analysis the significance of the
planned work is shown. Lastly, the budget for the planned research is detailed.