[cig-commits] commit: Paragraph on scalability

[Mercurial]

changeset:   127:739270e078b9
tag:         tip
user:        Matthew G Knepley <knepley at gmail.com>
date:        Wed May 16 14:40:28 2012 -0400
files:       faultRup.tex
description:
Paragraph on scalability


diff -r 7d92bb1e50a7 -r 739270e078b9 faultRup.tex
--- a/faultRup.tex	Tue May 15 18:56:55 2012 -0400
+++ b/faultRup.tex	Wed May 16 14:40:28 2012 -0400
@@ -1213,15 +1213,22 @@ number of iterations required for conver
 number of iterations required for convergence.
 
 \matt{Add additional comments, explanation of performance}
-Sources of load imbalance:
-  unstructured partition (1.3)
-  Dirichlet conditions (2.2)
-  fault unknowns (3)
-Reasons to eliminate unknowns
+The underlying PETSc solver infrastructure has dmeonstrated optimal scalability
+on the largest machines available today. However, very often computer science
+scalability results are bsed upon unrelaistically simple problems which do not
+advance the scientific state-of-the-art. We will concentrate on explaining the
+sources of reduced scalability, and propose possible algorithmic mitigation.
 
-
-Competition between integration and solve for balance
-
+The main impediment to scalability in PyLith is load imbalance in the solver stage.
+This imbalance is the combination of three effects: the inherent imbalance in the
+partition of an unstructured mesh, the use of a cell partition, and lack of
+incorporation of cohesive cells in the partition. In our full test case, the unstructured
+partition calculated with both ParMetis could have a load imbalance of up to 30\%
+on 128 processors. On top of this, the cell partition, which is necessary in order
+to achieve good balance for the finite element integration, does not take into
+account Dirichlet boundary conditions or unknowns on the fault, which can exacerbate
+the imbalance. However, elimination of constrained unknowns preserves the symmetry
+of the overall systems, and can result in better conditioned linear systems.
 
 \subsection{Parallel Scaling Performance}