integer arithmetics

John W. Eaton jwe at bevo.che.wisc.edu
Thu Oct 2 14:12:06 CDT 2008


On  2-Oct-2008, Jaroslav Hajek wrote:

| On Thu, Oct 2, 2008 at 9:30 AM, Jaroslav Hajek <highegg at gmail.com> wrote:
| > On Thu, Oct 2, 2008 at 7:47 AM, Jaroslav Hajek <highegg at gmail.com> wrote:
| >> On Wed, Oct 1, 2008 at 10:36 PM, John W. Eaton <jwe at bevo.che.wisc.edu> wrote:
| >
| >>
| >> I'll resend an update.
| >>
| >
| >
| > And here it is.
| >
| 
| Sorry, here it is. It appears to work reasonably well even if I
| undefine HAVE_FAST_INT_OPS and SIZEOF_LONG_DOUBLE. I fixed a few
| places and modified multiplication and division to get correct
| behavior in the common cases like 2*a or a/3.
| If you want the illustration of the remaining problems, try:
| ((int64(2**62) + 2) * 0.5) - (int64(2**61) + 1)
| which should, in theory, give zero. These are really corner cases, as
| nobody would probably do a*0.5 and intend integer arithmetics, but
| still an inconsistency. I think we may warn about it in the manual. If
| you replace * 0.5 with / 2, it should work (though it's still division
| by a double).
| 
| Nothing like this should happen for addition and subtraction - try
| (3*2**62 + (1+intmin('int64'))) == (int64(2**62)+1)
| or
| (3*2**63 - intmax('uint64')) == (uint64(2**63)+1)
| 
| These should both be true (note that in both cases the first operand
| is outside the integer range).

It seems bad to get incorrect answers.  I'd guess people are likely to
think that multiplying by 0.5 is the same as dividing by 2.

Would it be better to skip mixed-type operations for int64?  Or at
least skip the ones that can cause trouble?

jwe


More information about the Octave-maintainers mailing list