Tue, Jul 18 2006
14:38:02
|
|
Request created by hbowman
|
|
Subject: integer division errors
Hi,
I've just fixed a bug in the symbol library where:
float_red = int_red / 255;
int/int -> int, so the answer was always 0.0 or 1.0. no good.
grep finds these ones too, maybe some are wrong, some are ok?
display/d.his/his.c: R = R * I / 255;
display/d.his/his.c: G = G * I / 255;
display/d.his/his.c: B = B * I / 255;
display/d.his/his.c: R = 127 + (R - 127) * S / 255 ;
display/d.his/his.c: G = 127 + (G - 127) * S / 255 ;
display/d.his/his.c: B = 127 + (B - 127) * S / 255 ;
general/g.pnmcomp/main.c: *r = (*r * c0 + *p * c1) / 255;
raster/r.his/his.c: R = R * I / 255;
raster/r.his/his.c: G = G * I / 255;
raster/r.his/his.c: B = B * I / 255;
raster/r.his/his.c: R = 127 + (R - 127) * S / 255 ;
raster/r.his/his.c: G = 127 + (G - 127) * S / 255 ;
raster/r.his/his.c: B = 127 + (B - 127) * S / 255 ;
raster/r.out.vtk/writeascii.c: fprintf(fp, "%lf %lf %lf \n", r / 255, g /
255, b / 255);
raster3d/r3.out.vtk/writeVTKData.c: fprintf(fp, "%.*f ", dp, (value / 255));
raster3d/r3.out.vtk/writeVTKData.c: fprintf(fp, "%.*f ", dp, (value / 255));
raster/r.out.tiff/r.out.tiff.c:#define SCALE(x) (((x)*((1L<<16)-1))/255)
raster3d/r3.showdspf/togif.c: val = (nshades*k)/255;
Hamish
|
|
Wed, Jul 19 2006
09:37:45
|
|
Mail sent by glynn@gclements.plus.com
|
|
Return-Path |
<glynn@gclements.plus.com>
|
Delivered-To |
grass-bugs@lists.intevation.de
|
From |
Glynn Clements <glynn@gclements.plus.com>
|
MIME-Version |
1.0
|
Content-Type |
text/plain; charset=us-ascii
|
Content-Transfer-Encoding |
7bit
|
Message-ID |
<17597.57665.894837.849966@cerise.gclements.plus.com>
|
Date |
Wed, 19 Jul 2006 08:37:37 +0100
|
To |
Request Tracker <grass-bugs@intevation.de>
|
Cc |
grass-dev@grass.itc.it
|
Subject |
Re: [GRASS-dev] [bug #4864] (grass) integer division errors
|
In-Reply-To |
<20060718123802.EDB141005C9@lists.intevation.de>
|
References |
<20060718123802.EDB141005C9@lists.intevation.de>
|
X-Mailer |
VM 7.07 under 21.4 (patch 15) "Security Through Obscurity" XEmacs Lucid
|
X-Virus-Scanned |
by amavisd-new at intevation.de
|
X-Spam-Status |
No, hits=-3.441 tagged_above=-999 required=3 tests=[AWL=1.293, BAYES_00=-5, FORGED_RCVD_HELO=0.266]
|
X-Spam-Level |
|
Request Tracker wrote:
> this bug's URL: http://intevation.de/rt/webrt?serial_num=4864
> Subject: integer division errors
>
> Hi,
>
> I've just fixed a bug in the symbol library where:
>
> float_red = int_red / 255;
>
> int/int -> int, so the answer was always 0.0 or 1.0. no good.
>
> grep finds these ones too, maybe some are wrong, some are ok?
They're all okay.
> display/d.his/his.c: R = R * I / 255;
> display/d.his/his.c: G = G * I / 255;
> display/d.his/his.c: B = B * I / 255;
> display/d.his/his.c: R = 127 + (R - 127) * S / 255 ;
> display/d.his/his.c: G = 127 + (G - 127) * S / 255 ;
> display/d.his/his.c: B = 127 + (B - 127) * S / 255 ;
> general/g.pnmcomp/main.c: *r = (*r * c0 + *p * c1) / 255;
> raster/r.his/his.c: R = R * I / 255;
> raster/r.his/his.c: G = G * I / 255;
> raster/r.his/his.c: B = B * I / 255;
> raster/r.his/his.c: R = 127 + (R - 127) * S / 255 ;
> raster/r.his/his.c: G = 127 + (G - 127) * S / 255 ;
> raster/r.his/his.c: B = 127 + (B - 127) * S / 255 ;
> raster3d/r3.showdspf/togif.c: val = (nshades*k)/255;
> raster/r.out.tiff/r.out.tiff.c:#define SCALE(x) (((x)*((1L<<16)-1))/255)
These are all supposed to produce integer results.
> raster/r.out.vtk/writeascii.c: fprintf(fp, "%lf %lf %lf \n", r / 255, g /
255, b / 255);
> raster3d/r3.out.vtk/writeVTKData.c: fprintf(fp, "%.*f ", dp, (value / 255));
> raster3d/r3.out.vtk/writeVTKData.c: fprintf(fp, "%.*f ", dp, (value / 255));
These all have FP numerators.
--
Glynn Clements <glynn@gclements.plus.com>
|
|
Wed, Jul 19 2006
11:09:13
|
|
Status changed to resolved by hbowman
|
|