Evaluation

Among the methods considered in this work, the synthetic OCT-A is the only one trained with the raw OCT-A images. Therefore, we computed image synthesis metrics (mean absolute error (MAE) and structural similarity index (SSIM)) on the synthetic OCT-A images with respect to the OCT-A ground truth. For the test dataset, the MAE was 40.78 (SD = 3.53) and the SSIM was 0.08 (SD = 0.04).

Further experiments evaluated four distinct vessel segmentation methods. Two of them relying on manual vessels segmentation from fundus images (IterNet and SA-UNet), and the other two relying on vessels delineated from OCT-A using a vessel segmentation algorithm (Synthetic OCT-A Segmented and IterNet w/ OCT-A).

Estimating the vessel density at the optic disc and macular regions is a standard process in the quantification of vascular morphology in OCT-A data^{8, 24, 25}. We used this assessment to evaluate how well the segmentations from the different methods approximated the ground truth OCT-A grade vessels based on a biomarker (density) computed from them. We computed precision-recall curves and Areas Under the Curve (AUCs) to assess segmentation performance at the pixel level.

In our evaluation, we considered versions with vessels segmented from both ground truth and synthetic OCT-A to highlight only the pixels belonging to blood vessels. To segment the vessels from OCT-A images, we used the OCT-A vessel segmentation methodology proposed by Ma et al.²⁶. The algorithm uses OCTA-Net, a model encompassing different stages of neural networks processing coarse and fine features in en face OCT-A images to classify pixels into vessel and non-vessel. The model was trained on 229 OCT-A images each with a corresponding manual vessel delineation provided by an ophthalmologist. The ROSE dataset is a fully independent dataset to our own, provided by the authors²⁶.

We first computed Pearson and Spearman correlations for the vessel density measures; specifically, we explored the correlation of vessel density for the optic disc and macula between synthetic and real OCT-A images with the quadrants defined by the Early Treatment Diabetic Retinopathy Study (ETDRS) grid. This grid breaks down the images into four quadrants corresponding to nasal, temporal, superior, and inferior regions. Our experiments used an ETDRS grid with an outer diameter of 3 mm and inner diameter of 1 mm.

Software in commercial cameras employ similar strategies for the extraction of vasculature density measures (sometimes without any segmentation) by not relying on manual annotations, given that if the image is correctly acquired, any pixel in the en face OCT-A is a representation of blood perfusion. While vascular density can be computed from non-binarized, raw OCT-A en face images, we did not conduct this experiment as the synthetic OCT-A would have had an unfair advantage over the vessel segmentation approaches trained on the binarized vasculature.

In addition, using the target OCT-A pixel-level labels and the probability scores from the different models, we computed precision-recall (PR) curves for macular and optic disc regions. We also provide visualizations for the segmentations produced by both proposed and established methods considered.