32bit vs. 64bit

[Jason Riedy]

And Michael D. Godfrey writes:
> In addition, the "solution" of just using higher precision does little
> good.

Actually, trying in higher precision does a lot of good, but it
isn't perfect.  There are examples where just about every
non-exact solution fails, see below.

> The position taken in Forsythe and Moler that computations should
> evaluate the condition of the result seems safer.  However, they also
> make the point that it is not possible to evaluate the condition of a
> result in the precision used to compute it.

In general, the condition number of a condition number is the condition
number.  Thus accurately determining a large condition number may
be problematic, but often ends up large enough.  You do lose
"easy" problems when you allow for a safety region. For more
background, see
  Demmel, J. W. 1987. The geometry of ill-conditioning.
  J. Complex.  3, 2 (Jun. 1987), 201-229.
  http://dx.doi.org/10.1016/0885-064X(87)90027-6
(BTW, Dr. Demmel is my advisor at UCB.  And I worked on 754r with
Prof. Kahan for as long as I could take it. whee.)

There are other great examples in Prof. Kahan's write-ups and
slides:
  http://www.cs.berkeley.edu/~wkahan/
In particular,
 * "How Futile are Mindless Assessments of Roundoff in
   Floating-Point Computation?", and
 * "Mathematics Written in Sand".

The former has Jean-Michel Muller's recurrence on p14 of the
PDF.  That is the example to foil most schemes.

> Is this [lack of a guard bit] really true?

Nope.  x86-64 pretty much makes sense and even fixes one
outstanding bug in the x87 unit (invalid being overloaded also to
mean stack overflow).  There is the outstanding issue of how to
interpret & use the exceptional flags for vectors, but that's not
a trivial issue.

> (By the way, I was around when Kahan forced IBM, at very large
> cost, to retrofit guard digits into the original 360 systems.
> It was fun to watch IBM trying sneak out of having to do that.)

I don't suppose you were a witness to lightning striking and the
lights going out at a presentation when Prof. Kahan said "and now
on to IBM"?  ;) I've been told that was at a conference in
Toronto.

> I would be very surprised if Cleve had a say in this.

He's the one who I heard gripe about ATLAS once, although he's
not the official decider.  It's the support cost.

However, I don't think they produce exactly the same data for
everything but rather try in most circumstances.  I know they
don't produce exactly the same result between *versions*.  They
adopt improved LAPACK routines, and the eigenvalue routines now
have better accuracy qualities.

There is a relatively easy test.  Generate long vectors v and w
whose dot products destructively cancel at different block sizes;
I think there may be such a generator in the XBLAS testers.  Fill
one matrix with v and another with w.', then multiply the
matrices.  I don't have access to Matlab(TM) at the moment, and
it may have changed since I last played with it / griped at
MathWorks folks about it.

> that computers always produce "correct" answers for any inputs.

Yeah, see my mini-rant in the other message about the ACM
suggested curriculum.  sigh.  And Prof. Kahan's writings above.

Jason