pi is wrong at the 16th digit
John W. Eaton
jwe at octave.org
Tue May 5 12:14:52 CDT 2009
On 29-Mar-2009, Rob Mahurin wrote:
| On Mar 28, 2009, at 7:03 PM, Rafael Laboissiere wrote:
| > I am wondering whether output_precision() should impose an upper
| > limit,
| > something like ceil(-log10(eps)), just to prevent erroneous
| > interpretations like the above.
|
|
| Right now output_precision is
|
| DEFUN(output_precision,...)
| {
| return SET_INTERNAL_VARIABLE_WITH_LIMITS (output_precision, -1,
| INT_MAX);
| }
|
| so that should be easy (if it's worth doing). But with unchanged
| sources (stable and dev) I get
|
| $ ./run-octave -q
| octave:1> output_precision
| ans = 5
| octave:2> output_precision(-1)
| octave:3> 1+eps
| ans = 1.00
| octave:4> output_precision(-2)
| panic: Segmentation fault -- stopping myself...
| attempting to save variables to `octave-core'...
| save to `octave-core' complete
| Segmentation fault
|
| which is a bug.
I checked in the following fix for the segfault:
http://hg.savannah.gnu.org/hgweb/octave/rev/2669527e0ce5
jwe
More information about the Bug-octave
mailing list