Alya Red CCM: HPC-Based Cardiac Computational Modelling - Selected Topics of Computational and Experimental Fluid Mechanics - page 192

Among other coupled problems, a cardiac electro-mechanical simulation is run for a
high-resolution bi-ventricular geometry. Figure 4 shows the geometry and the strong
scalability plot. The test is performed in the following way. A 6M tetrahedral mesh
is generated for the bi-ventricular geometry provided by Dr. A. Berruezo (Hospital
Clinic de Barcelona) in collaboration with R. Sebastian (UVEG) and O. Camara
(UPF). Following Houzeaux et al. ( 2013 ), the original mesh is progressively sub-
divided in smaller elements, creating a hierarchy of larger meshes. To perform the
scalability test, two meshes coming from the subdivision cycle, 427M and 3.4B
elements, are used. They are respectively labelled “DIV2” and “DIV3” in Fig. 4 .
Scalability is measured upon the time needed to solve one time step.
The smaller mesh “DIV2” shows linear scalability up to 65K processors, being
normalized with the 1,024 cores run. At 65K “DIV2” has a mean of 6,500 tetrahedra
per core. At this end, communications' time starts to be noticeable with respect to
computing time. However, scalability figures for the larger mesh “DIV3” are linear
up to the top: it is 8 times larger, with a mean of 52K elements per core at 65K cores.
It is worth mentioning that after several tests we have established a practical
rule of thumb for the wall clock time of coupled electro-mechanical problems. With
5,000-10,000 elements per core for a rabbit size heart with millions of elements, a
coupled problem for 1 s of real time runs in approximately 10-15min wall clock time
in BSC's Marenostrum III if no special optimization option is used and compiling
the code with the Intel Fortran Compiler. Depending on the electrophysiology model
used (FHN or TT) this time could vary around 20-30%. We believe that after a
heavy code optimization and cleaning, these figures should go down to 1 s of real
time solved in 1min of wall clock time.
4 Examples
Arís ( 2014 ) and Arís et al. ( 2014 ) have shown the potential of Alya Red CCM by
performing a sensitivity analysis for several conditions. The goal is to investigate the
effect of initial conditions in the resultant simulation, determining how a change in
an input will affect the output. In particular, we study the influence of the electrical
activation in the contraction of the tissue. The analysis is performed in a bi-ventricular
geometry, where 14 different activation protocols are tested. In addition, two different
fiber field orientations are interpolated in the geometry, and each activation protocol
is combined with each fiber field description. For each case, simulations are evaluated
by quantifying the variation of epicardial breakthrough, total activation time, ejection
fraction and time of maximal contraction. Considering the complexity of cardiac
models, the sensitivity analysis carried out by Arís ( 2014 ) and Arís et al. ( 2014 )
emphasizes the possibility to optimise the cardiac activation protocols by using the
present method. The results presented show that the proposed CCM tool is capable
of capturing variations to small input parameter changes.
In this paper we briefly present some of the results.