integer arithmetics

[Jaroslav Hajek]

On Thu, Oct 2, 2008 at 9:30 AM, Jaroslav Hajek <highegg at gmail.com> wrote:
> On Thu, Oct 2, 2008 at 7:47 AM, Jaroslav Hajek <highegg at gmail.com> wrote:
>> On Wed, Oct 1, 2008 at 10:36 PM, John W. Eaton <jwe at bevo.che.wisc.edu> wrote:
>
>>
>> I'll resend an update.
>>
>
>
> And here it is.
>

Sorry, here it is. It appears to work reasonably well even if I
undefine HAVE_FAST_INT_OPS and SIZEOF_LONG_DOUBLE. I fixed a few
places and modified multiplication and division to get correct
behavior in the common cases like 2*a or a/3.
If you want the illustration of the remaining problems, try:
((int64(2**62) + 2) * 0.5) - (int64(2**61) + 1)
which should, in theory, give zero. These are really corner cases, as
nobody would probably do a*0.5 and intend integer arithmetics, but
still an inconsistency. I think we may warn about it in the manual. If
you replace * 0.5 with / 2, it should work (though it's still division
by a double).

Nothing like this should happen for addition and subtraction - try
(3*2**62 + (1+intmin('int64'))) == (int64(2**62)+1)
or
(3*2**63 - intmax('uint64')) == (uint64(2**63)+1)

These should both be true (note that in both cases the first operand
is outside the integer range).

OK to push?
I'll contribute some tests later.

regards

-- 
RNDr. Jaroslav Hajek
computing expert
Aeronautical Research and Test Institute (VZLU)
Prague, Czech Republic
url: www.highegg.matfyz.cz
-------------- next part --------------
A non-text attachment was scrubbed...
Name: int_patch2.diff
Type: text/x-patch
Size: 38443 bytes
Desc: not available
Url : https://www-old.cae.wisc.edu/pipermail/octave-maintainers/attachments/20081002/0df424cf/attachment-0001.bin