[CIG-LONG] Error: The side is moving in the wrong direction

[Mark Brandon]

I am having a problem with job that runs nearly to completion but then 
aborts with the following error:

The side is moving in the wrong direction.
750 747.272 750 739.772 750
p0_30304:  p4_error: interrupt SIGx: 6

I included an expanded listing (below) of the log information where this 
error was generated. The full versions of the input.xml and log files 
are attched. As you can see from the log below, the program has already 
converged to a solution (after 1237 iterations), so it is unclear why it 
would abort at this point.

Note that I have found this error with a number of input files I have 
been running. As a check, I went back to an older input file that run to 
completion in November 2008 using the version of Gale current at that 
time. That input.xml file also aborts so I suspect that there is 
something that has changed with Gale. I did have to make one change to 
the old input.xml file before running it anew: I removed the following 
line: <param>Underworld_DumpSwarm</param>. It seems that this option is 
no longer supported.

I am running this job on a Linux workstation using a x86_64 cpu and GNU 
Linux, with the 64bit binary for Gale 1.4.1. I have run one of the 
benchmarks divergence3D.xml to completion with no problems, so the 
installation appears to be correct.
Thanks in advance for your help,
Mark Brandon


TAIL OF LOG FILE:
....
Loop = 1237, absResidual = 1.02870367e-05, relResidual = 9.84524890e-06
Summary: Uzawa_its = 1238; Uzawa Residual  = 9.84524890e-06
Summary: || Div ( u ) || / || u ||         = 8.14044912e+00
Summary: || F - Ku - Gp || / || F ||       = 3.14036853e-07
Summary: || F - Ku - Gp ||_w / || F ||_w   = 1.00000000e+00
Summary: Max velocity component = 1.33224002e+00; Velocity magnitude = 
4.77087969e-01
Summary: Max pressure value     = 2.65297991e+00; Pressure magnitude = 
2.11917203e-01
Linear solution time 127591 (secs)
Time Integration
Time Integration
            EP:        EulerDeform_IntegrationSetup -    0.0742 (secs)
            EP:        SwarmAdvector_AdvectionSetup -    0.0000 (secs)
     1st order:               materialSwarmAdvector -  629.6562 (secs)
     1st order:                EulerDeform_Velocity -   21.5039 (secs)
     1st order:                EulerDeform_Velocity -   16.9062 (secs)
The side is moving in the wrong direction.
750 747.272 750 739.772 750
p0_30304:  p4_error: interrupt SIGx: 6
-------------- next part --------------
A non-text attachment was scrubbed...
Name: inputStolar.xml
Type: text/xml
Size: 21042 bytes
Desc: not available
Url : http://geodynamics.org/pipermail/cig-long/attachments/20100528/efa1b13c/attachment-0001.bin 
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: runLog_inputStolar.txt
Url: http://geodynamics.org/pipermail/cig-long/attachments/20100528/efa1b13c/attachment-0001.txt