"Linking convolutional neural networks with graph convolutional networks: application in pulmonary artery-vein separation"

Zhiwei Zhai, Marius Staring, Xuhui Zhou, Qiuxia Xie, Xiaojuan Xiao, M. Els Bakker, Lucia J. Kroft, Boudewijn P.F. Lelieveldt, Duliette Boon, Frederikus A. Klok and Berend C. Stoel


Graph Convolutional Networks (GCNs) are a novel and powerful method for dealing with non-Euclidean data, while Convolutional Neural Networks (CNNs) can learn features from Euclidean data such as images. In this work, we propose a novel method to combine CNNs with GCNs (CNN-GCN), that can consider both Euclidean and non-Euclidean features and can be trained end-to-end. We applied this method to separate the pulmonary vascular trees into arteries and veins (A/V). Chest CT scans were pre-processed by vessel segmentation and skeletonization, from which a graph was constructed: voxels on the skeletons resulting in a vertex set and their connections in an adjacency matrix. 3D patches centered around each vertex were extracted from the CT scans, oriented perpendicularly to the vessel. The proposed CNN-GCN classifier was trained and applied on the constructed vessel graphs, where each node is then labeled as artery or vein. The proposed method was trained and validated on data from one hospital (11 patient, 22 lungs), and tested on independent data from a different hospital (10 patients, 10 lungs). A baseline CNN method and human observer performance were used for comparison. The CNN-GCN method obtained a median accuracy of 0.773 (0.738) in the validation (test) set, compared to a median accuracy of 0.817 by the observers, and 0.727 (0.693) by the CNN. In conclusion, the proposed CNN-GCN method combines local image information with graph connectivity information, improving pulmonary A/V separation over a baseline CNN method, approaching the performance of human observers.



PDF (8 pages, 694 kB) click to start download
From publisher link

Copyright © 2019 by the authors. Published version © 2019 by Springer Lecture Notes in Computer Science. Personal use of this material is permitted. However, permission to reprint or republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works, must be obtained from the copyright holder.


BibTeX entry

author = "{Zhiwei Zhai and Marius Staring and Xuhui Zhou and Qiuxia Xie and Xiaojuan Xiao and M. Els Bakker and Lucia J. Kroft and Boudewijn P.F. Lelieveldt and Duliette Boon and Frederikus A. Klok and Berend C. Stoel}",
title = "{Linking convolutional neural networks with graph convolutional networks: application in pulmonary artery-vein separation}",
booktitle = "{Graph Learning in Medical Imaging, MICCAI workshop}",
editor = "{D. Zhang and L. Zhou and B. Jie and M. Liu}",
address = "{Shenzhen, China}",
series = "{Lecture Notes in Computer Science}",
volume = "{11849}",
pages = "{36 - 43}",
month = "{October}",
year = "{2019}",

last modified: 04-02-2020 |webmaster |Copyright 2004-2020 © by Marius Staring