[aspect-devel] Aspect 2.0 tests failure question

[Wolfgang Bangerth]

Hi Bob,

> Wolfgang, do you think it might be valuable to add a relaxed tolerance to the 
> tests for end users, to suppress the non critical failures? We could always 
> keep the pure diff when testing on Timo's server.
> 
> I could play with this at the start of the Hackathon if you wanted, in lieu of 
> the annual typo hunt? My OCD always plays up when I see failed tests!
I think we all feel that way :-)

We already do allow for a tolerance if you have the 'numdiff' program 
installed, which compares numbers based on their relative size. (If you don't 
have 'numdiff', it just uses 'diff' that does a stupid text match and doesn't 
know anything about numbers at all. So at the very least one would need to 
have 'numdiff' installed to make this useful.

I don't know by how much one would need to relax the tolerance. I think it 
would make for an interesting experiment to find that out. It may be that 
there are some tests where small floating point differences lead to different 
refinement decisions -- in which case we enter the realm of O(1) differences. 
Either way, it would be interesting to play with it where cmake could set a 
default (loose) tolerance unless one passes a stricter tolerance requirement 
to cmake -- which is what one would do on the "official" tester.

Nice project for sure!

Cheers
  W.

-- 
------------------------------------------------------------------------
Wolfgang Bangerth          email:                 bangerth at colostate.edu
                            www: http://www.math.colostate.edu/~bangerth/