Format rat and "-1"

[David Bateman]

Eduardo Garcia wrote:
> Cc: Eduardo Garcia
> --------
> Bug report for Octave 3.2.0 configured for i386-apple-darwin8.11.1
>
> Description:
> -----------
>
>    When using "format rat" sometimes "-2147483648/-2147483648" is  
> displayed instead of "-1".
>    (Same bug is present in Octave 3.0.5 configured for i386-apple- 
> darwin8.9.1)
>
> Repeat-By:
> ---------
> BUG:
> octave-3.2.0:23> a=[1 2;3 5]
> octave-3.2.0:24> det(a)
> ans = -1.0000
> octave-3.2.0:25> format rat
> octave-3.2.0:26> det(a)
> ans = -2147483648/-2147483648
>
> BUT:
> octave-3.2.0:29> a=[1 2;1 1]
> octave-3.2.0:30> format
> octave-3.2.0:31> det(a)
> ans = -1
> octave-3.2.0:32> format rat
> octave-3.2.0:33> det(a)
> ans = -1
>   
Interesting problem.... What to do for the format rat when the numerator 
and/or the denominator is outside the valid integer range.. A simpler 
test example is

octave:1> format rat
octave:2> 1 - eps
ans = -2147483648/-2147483648

It would be interesting to know what matlab display for the cases 1 - 
eps,  1 - 10*eps, 1 - 1e3*eps and maybe a couple of other similar values 
before looking at a proper fix.

D.


-- 
David Bateman                                dbateman at dbateman.org
35 rue Gambetta                              +33 1 46 04 02 18 (Home)
92100 Boulogne-Billancourt FRANCE            +33 6 72 01 06 33 (Mob)