The development of neuroimaging software requires strict validation before it can be widely used by the community. However, there is no established framework to carry out this validation and it is up to the developers to find an adequate validation scheme. Garikoitz will be presenting his latest bioRxiv preprint at Brainhack Donostia 2020, where he proposes a validation framework for neuroimaging software with reproducibility in mind.
A bit about Dr Garikoitz Lerma-Usabiaga
Dr Lerma-Usabiaga’s research focuses on 1) behavioral as well as functional and structural MRI techniques to investigate the neural basis of vision and reading and 2) developing MRI methods–both functional and structural–to further examine cognitive functions and enhance neuroimaging reproducibility, validity, and generalizability. He is an Electrical Engineer with 5 years of industry experience as a management consultant and 7 years of experience as a tech entrepreneur. He obtained his PhD at the BCBL on characterizing the involvement of ventral occipitotemporal cortex in word recognition using multimodal MRI techniques. After his PhD, he joined Prof Brian Wandell at Stanford University for 3 years of postdoctoral studies, to work on advanced diffusion MRI methods, population receptive fields (pRF), MRI biomarkers, and single-subject quantification. Dr Lerma-Usabiaga is currently affiliated to the BCBL in his third year of the Marie Skłodowska-Curie Global Fellowship.
There’s a bit more about Gari’ talk in Brainhack Donostia in his abstract below!
Neuroimaging software methods are complex, making it a near certainty that some implementations will contain errors. Modern computational techniques (e.g., public code and data repositories, continuous integration, and containerization) enable the reproducibility of analyses and reduce coding errors, but they do not guarantee the scientific validity of the results.
We describe a framework for validating and sharing software implementations. We apply the framework to an application: population receptive field (pRF) methods for functional MRI data. Using this framework, we identified realistic conditions that lead to imperfect parameter recovery in four public pRF implementations, and we provide a means to reduce this problem in real experimental settings. Additionally, I will show new results comparing circular and elliptical pRF fits.
The computational validity framework supports scientific rigor and creativity, as opposed to the oft-repeated suggestion that investigators rely on a few agreed upon packages. Having validation frameworks helps (1) developers to build new software, (2) research scientists to verify the software’s accuracy, and (3) reviewers to evaluate the methods used in publications and grants.