<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Hi Eric,<div><br></div><div>The error occurs after the second time step. I'll try running the model again to see if I can reproduce the problem with the same number of processors (32).</div><div><br></div><div>John</div><div><br></div><div><div><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><br class="Apple-interchange-newline"></div></span></span>
</div>
<br><div><div>On Jun 25, 2012, at 10:03 AM, Eric Heien wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div>Hi John,<br><br>I can't seem to reproduce this problem with your parameter file. Does the error occur at the beginning of the simulation or after it has run for a while? If it happens after a while, can you do a visualization of the particle movements until it crashes?<br><br>Thanks,<br><br>-Eric<br><br>On Jun 23, 2012, at 3:30 PM, John Naliboff wrote:<br><br><blockquote type="cite">Hi Eric,<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Gotcha. The simulation (.prm file attached) is pretty straightforward - 2D box (1x1 dimensions) with cooling at the top and heating at the bottom. The Rayleigh number is pretty high (1e8), but the model seems to be converging towards the predicted Nusselt number (i.e. running on single node).<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">For the even higher Ra number simulations (5e8, 1e9) I'll comment the out the check_particle_count() line, as the computation time is noticeably increasing as the Rayleigh number goes up.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">John<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><bkb1Ra1e8.prm><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">On Jun 22, 2012, at 10:49 PM, Eric Heien wrote:<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><blockquote type="cite">This means a particle fell out of the mesh and was lost - I haven't yet written code to put these particles back in the mesh. It may be indicative of something non-physical about the simulation, but since it happens based on the number of cores it is more likely a bug I need to fix. Can you send me the parameter file you're using?<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">If you want to ensure the run works anyway, you can try commenting out the call to check_particle_count() in tracer.cc.<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">-Eric<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">On Jun 23, 2012, at 2:53 AM, John Naliboff wrote:<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">Hi all,<br></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">I came across an error that appears to be associated with tracking of particles across multiple nodes:<br></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><span class="Apple-tab-span" style="white-space:pre"> </span>Exception on MPI process <0> while running postprocessor <N6aspect11Postprocess11ParticleSetILi2EEE>: <br></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><span class="Apple-tab-span" style="white-space:pre"> </span>-------------------------------------------------------- <br></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"> An error occurred in line <484> of file <source/postprocess/tracer.cc> in function <br></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><span class="Apple-tab-span" style="white-space:pre"> </span> void aspect::Postprocess::ParticleSet<dim>::check_particle_count() [with int dim = 2] <br></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"> The violated condition was: <br></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><span class="Apple-tab-span" style="white-space:pre"> </span> global_particles==global_sum_particles <br></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"> The name and call sequence of the exception was: <br></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"> ExcMessage ("Particle count changed.") Additional Information: Particle count changed. <br></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"> -------------------------------------------------------- <br></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"> Aborting!<br></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">I received this error when I tried to run a model on 32 threads across 2 nodes. I am currently running the same model successfully on 16 threads across 1 node. The attached file (bkb1Ra1e8.o6876) contains the full error output. The error also generated a number of "core" files in the model directory. I last updated aspect through svn update two weeks ago, but this is perhaps an issue specific to deal.ii?<br></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">Cheers,<br></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">John<br></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><bkb1Ra1e8.o6876><br></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">_______________________________________________<br></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">Aspect-devel mailing list<br></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><a href="mailto:Aspect-devel@geodynamics.org">Aspect-devel@geodynamics.org</a><br></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><a href="http://geodynamics.org/cgi-bin/mailman/listinfo/aspect-devel">http://geodynamics.org/cgi-bin/mailman/listinfo/aspect-devel</a><br></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">_______________________________________________<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">Aspect-devel mailing list<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><a href="mailto:Aspect-devel@geodynamics.org">Aspect-devel@geodynamics.org</a><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><a href="http://geodynamics.org/cgi-bin/mailman/listinfo/aspect-devel">http://geodynamics.org/cgi-bin/mailman/listinfo/aspect-devel</a><br></blockquote></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">_______________________________________________<br></blockquote><blockquote type="cite">Aspect-devel mailing list<br></blockquote><blockquote type="cite"><a href="mailto:Aspect-devel@geodynamics.org">Aspect-devel@geodynamics.org</a><br></blockquote><blockquote type="cite"><a href="http://geodynamics.org/cgi-bin/mailman/listinfo/aspect-devel">http://geodynamics.org/cgi-bin/mailman/listinfo/aspect-devel</a><br></blockquote><br>_______________________________________________<br>Aspect-devel mailing list<br><a href="mailto:Aspect-devel@geodynamics.org">Aspect-devel@geodynamics.org</a><br>http://geodynamics.org/cgi-bin/mailman/listinfo/aspect-devel<br></div></blockquote></div><br></div></body></html>