Inter-vendor Comparison of EchoGo Core v1.0 with Competitor1A using the UK EVAREST Study
- | By Ultromics
Upton et al, EuroEcho Dec, 2019.
This study comprised 311 previously unseen echocardiographic images from the multicentre prospective EVAREST study (ClinicalTrials.gov Identifier: NCT03674255), which included 31 NHS hospitals. Routine transthoracic echocardiographic examinations were acquired according to local clinical protocols using GE (Vivid 7, Vivid E9, Vivid E95) or Philips (iE33, EPIQ 5C, EPIC 7G, EPIQ 7C, EPIC CVx) ultrasound scanners. Echocardiography was performed between 2014 and 2019.
To generate comparative semi-automated quantitative measures of strain and left ventricular volumes, for the purposes of this study, vendor neutral analysis software (Competitor1A Arena TTA2) was used to perform measurements of cardiac left ventricular volumes, ejection fraction (Simpson’s Bi-plane method) and global longitudinal strain, representing “manual” clinical image processing.
Global strain measures were performed using two apical views (LV focussed four-chamber and standard two-chamber). For image processing through Competitor1A , three endocardial points were applied in each apical view, with two points at each side of the mitral annulus and a third point at the apex of the left ventricle in the end-systolic period, as determined by five observers (three BSE accredited echocardiographers and two board-certified cardiologist).
Figure 1. Overview of inter-vendor comparison testing. All patient imaging studies were processed using both EchoGo Core v1.0 and Competitor1A .
The software automatically provided lines along with endocardial layer, which followed each myocardial layer by speckle tracking. The endocardial layer region of interest required manual adjustment at end-diastole and end-systole in all cases by all observers to include the entire LV segmentation.
All images collected were also processed through the automated Ultromics software, EchoGo Core v1.0 by five observers (three BSE accredited echocardiographers and two board-certified cardiologist). All observers were blinded to the patient information and output from each respective analysis (i.e. Competitor1A or EchoGo Core results).
Reading instances were respectively randomised before each instance of image processing for each observer and vendor (fully-crossed randomised reader design). Each application was used to compute ED and ES volumes, EF and GLS.
Table 1. Summary statistics for inter-vendor comparison of EchoGo Core v1.0 to Competitor1A using UK EVAREST data.
Figure 2. Inter-vendor comparison of EchoGo Core v1.0 to Competitor1A volumes using UK EVAREST data.
Figure 3. Summary results for inter-vendor comparison of EchoGo Core 1.0 to Competitor1A using UK EVAREST data.
The inter-vendor comparison to Competitor1A was summarised as the mean value produced across observers during echocardiographic examination.
Since each measurement was made by a difference independent observer, mean values (EchoGo Core vs Competitor1A ) as an overall assessment of performance was appropriate using standard distribution analysis, Bland-Altman analysis and Deming regressions, since repeated measures was not observed in this instance (i.e. same observer producing multiple measurements).
Mean ES volumes, ED volumes, GLS and EF RMSE values of 7.8 ml, 9.9 ml, 2.6% and 5.3%, respectively for Philips and 8.0 ml, 12.2 ml, 2.56% and 4.75%, respectively for GE were observed compared to Competitor1A.
Inter-operator variability of continuous variables (EF, GLS, and volumes) when processing the same image clips with EchoGo Core demonstrated no variation was observed (0 RMSE, Bias and LoA for EF, GLS, and Volumes, respectively). This observation was not dependent on manufacturer.