Thu, Jan 9 2003
09:21:17
|
|
Request created by guest
|
|
Subject: r.mapcalc error
Platform: GNU/Linux/i386
grass obtained from: Trento Italy site
grass binary for platform: Compiled from Sources
GRASS 5.0.1(cvs:2003/1/3) compiled under Redhat Linux 7.2
I found the following bugs(?):
Error Message: Segmentation Error
Command : r.mapcalc "crange=@cmax-@cmin"
cmax(raster) : created by r.statistics command
base(integer) : include 12737 categoriws
cover(integer): DEM
method : max
cmin(raster) : created by r.statistics command
base(integer) : include 12737 categoriws(same as cmax)
cover(integer): DEM
method : min
All raster data have 2400x3600 cells with 0.5m resolution under UTM corrdinates.
Kazukiyo Yamamoto:
Graduate School of Bioagricultural Sciences
School of agricultural Sciences
Nagoya University |
|
Fri, Jan 10 2003
18:32:00
|
|
Mail sent by neteler@itc.it
|
|
Return-Path |
<neteler@itc.it>
|
Delivered-To |
grass-bugs@lists.intevation.de
|
Date |
Fri, 10 Jan 2003 18:31:53 +0100
|
From |
Markus Neteler <neteler@itc.it>
|
To |
Request Tracker <grass-bugs@intevation.de>
|
Cc |
grass5@grass.itc.it
|
Subject |
Re: [GRASS5] [bug #1502] (grass) r.mapcalc error
|
Message-ID |
<20030110183153.A18566@itc.it>
|
Mail-Followup-To |
Request Tracker <grass-bugs@intevation.de>, grass5@grass.itc.it
|
References |
<20030109082117.D943213B01@lists.intevation.de>
|
Mime-Version |
1.0
|
Content-Type |
text/plain; charset=us-ascii
|
Content-Disposition |
inline
|
User-Agent |
Mutt/1.2.5.1i
|
In-Reply-To |
<20030109082117.D943213B01@lists.intevation.de>; from grass-bugs@intevation.de on Thu, Jan 09, 2003 at 09:21:17AM +0100
|
X-Spam-Status |
No, hits=-3.0 required=5.0 tests=IN_REP_TO,QUOTED_EMAIL_TEXT,REFERENCES,SPAM_PHRASE_00_01, USER_AGENT,USER_AGENT_MUTT version=2.43
|
X-Spam-Level |
|
On Thu, Jan 09, 2003 at 09:21:17AM +0100, Request Tracker wrote:
> this bug's URL: http://intevation.de/rt/webrt?serial_num=1502
> -------------------------------------------------------------------------
>
> Subject: r.mapcalc error
>
> Platform: GNU/Linux/i386
> grass obtained from: Trento Italy site
> grass binary for platform: Compiled from Sources
>
> GRASS 5.0.1(cvs:2003/1/3) compiled under Redhat Linux 7.2
>
> I found the following bugs(?):
>
> Error Message: Segmentation Error
> Command : r.mapcalc "crange=@cmax-@cmin"
> cmax(raster) : created by r.statistics command
> base(integer) : include 12737 categoriws
> cover(integer): DEM
> method : max
> cmin(raster) : created by r.statistics command
> base(integer) : include 12737 categoriws(same as cmax)
> cover(integer): DEM
> method : min
> All raster data have 2400x3600 cells with 0.5m resolution under UTM corrdinates.
>
> Kazukiyo Yamamoto:
> Graduate School of Bioagricultural Sciences
> School of agricultural Sciences
> Nagoya University
The problem appears also for me.
r.mapcalc "test=@map"
Segmentation fault
A fix were great :-)
Markus
|
|
Sat, Jan 11 2003
03:36:25
|
|
Mail sent by glynn.clements@virgin.net
|
|
Return-Path |
<glynn.clements@virgin.net>
|
Delivered-To |
grass-bugs@lists.intevation.de
|
From |
Glynn Clements <glynn.clements@virgin.net>
|
MIME-Version |
1.0
|
Content-Type |
text/plain; charset=us-ascii
|
Content-Transfer-Encoding |
7bit
|
Message-ID |
<15903.21591.591195.116054@cerise.nosuchdomain.co.uk>
|
Date |
Fri, 10 Jan 2003 23:16:39 +0000
|
To |
Markus Neteler <neteler@itc.it>
|
Cc |
Request Tracker <grass-bugs@intevation.de>, grass5@grass.itc.it
|
Subject |
Re: [GRASS5] [bug #1502] (grass) r.mapcalc error
|
In-Reply-To |
<20030110183153.A18566@itc.it>
|
References |
<20030109082117.D943213B01@lists.intevation.de> <20030110183153.A18566@itc.it>
|
X-Mailer |
VM 6.94 under 21.4 (patch 10) "Military Intelligence" XEmacs Lucid
|
X-Spam-Status |
No, hits=-2.4 required=5.0 tests=DATE_IN_PAST_03_06,EMAIL_ATTRIBUTION,IN_REP_TO,REFERENCES, SIGNATURE_SHORT_DENSE,SPAM_PHRASE_01_02 version=2.43
|
X-Spam-Level |
|
Markus Neteler wrote:
> > I found the following bugs(?):
> >
> > Error Message: Segmentation Error
> > Command : r.mapcalc "crange=@cmax-@cmin"
> The problem appears also for me.
>
> r.mapcalc "test=@map"
> Segmentation fault
Not for me, unfortunately. Can you reproduce the problem with a map
from one of the sample locations (spearfish, global, etopo5 etc)? If
not, can you provide a *small* location (in terms of file size) which
demonstrates the problem?
Failing that, can you get a backtrace? Either enable core files then
use "gdb -c core" on the resulting core file, or run r.mapcalc under
gdb, e.g.
$ gdb /usr/local/grass5/etc/bin/cmd/r.mapcalc
...
> run test=@map
...
> where
--
Glynn Clements <glynn.clements@virgin.net>
|
|
Mon, Jan 13 2003
15:23:03
|
|
Mail sent by neteler@itc.it
|
|
Return-Path |
<neteler@itc.it>
|
Delivered-To |
grass-bugs@lists.intevation.de
|
Date |
Mon, 13 Jan 2003 15:22:53 +0100
|
From |
Markus Neteler <neteler@itc.it>
|
To |
Glynn Clements <glynn.clements@virgin.net>
|
Cc |
Request Tracker <grass-bugs@intevation.de>, grass5@grass.itc.it
|
Subject |
Re: [GRASS5] [bug #1502] (grass) r.mapcalc error
|
Message-ID |
<20030113152253.G7614@itc.it>
|
Mail-Followup-To |
Glynn Clements <glynn.clements@virgin.net>, Request Tracker <grass-bugs@intevation.de>, grass5@grass.itc.it
|
References |
<20030109082117.D943213B01@lists.intevation.de> <20030110183153.A18566@itc.it> <15903.21591.591195.116054@cerise.nosuchdomain.co.uk>
|
Mime-Version |
1.0
|
Content-Type |
text/plain; charset=us-ascii
|
Content-Disposition |
inline
|
User-Agent |
Mutt/1.2.5.1i
|
In-Reply-To |
<15903.21591.591195.116054@cerise.nosuchdomain.co.uk>; from glynn.clements@virgin.net on Fri, Jan 10, 2003 at 11:16:39PM +0000
|
X-Spam-Status |
No, hits=-3.3 required=5.0 tests=IN_REP_TO,QUOTED_EMAIL_TEXT,REFERENCES,SPAM_PHRASE_01_02, USER_AGENT,USER_AGENT_MUTT version=2.43
|
X-Spam-Level |
|
On Fri, Jan 10, 2003 at 11:16:39PM +0000, Glynn Clements wrote:
[...]
> > r.mapcalc "test=@map"
> > Segmentation fault
>
> Not for me, unfortunately. Can you reproduce the problem with a map
> from one of the sample locations (spearfish, global, etopo5 etc)? If
> not, can you provide a *small* location (in terms of file size) which
> demonstrates the problem?
>
> Failing that, can you get a backtrace? Either enable core files then
> use "gdb -c core" on the resulting core file, or run r.mapcalc under
> gdb, e.g.
Here we are:
Program received signal SIGSEGV, Segmentation fault.
G_quant_get_cell_value (q=0x80a9d28, dcellVal=0) at quant.c:700
/hardmnt/thuille1/ssi/cvsgrass_exp/src/libes/gis/quant.c:700:19329:beg:0x80641ad
(gdb)
I have stored the location (320kb) here:
http://mpa.itc.it/markus/tmp/rmapcalcUTM.tar.gz
The command crashing is:
r.mapcalc "test=@census1994dryUTM34S"
6%
Segmentation fault
Thanks for looking into this. The map was imported with r.in.shape.
Markus
|
|
Tue, Jan 14 2003
03:04:31
|
|
Mail sent by glynn.clements@virgin.net
|
|
Return-Path |
<glynn.clements@virgin.net>
|
Delivered-To |
grass-bugs@lists.intevation.de
|
From |
Glynn Clements <glynn.clements@virgin.net>
|
MIME-Version |
1.0
|
Content-Type |
text/plain; charset=us-ascii
|
Content-Transfer-Encoding |
7bit
|
Message-ID |
<15907.28437.650858.933396@cerise.nosuchdomain.co.uk>
|
Date |
Tue, 14 Jan 2003 01:59:49 +0000
|
To |
Markus Neteler <neteler@itc.it>
|
Cc |
Request Tracker <grass-bugs@intevation.de>, grass5@grass.itc.it
|
Subject |
Re: [GRASS5] [bug #1502] (grass) r.mapcalc error
|
In-Reply-To |
<20030113152253.G7614@itc.it>
|
References |
<20030109082117.D943213B01@lists.intevation.de> <20030110183153.A18566@itc.it> <15903.21591.591195.116054@cerise.nosuchdomain.co.uk> <20030113152253.G7614@itc.it>
|
X-Mailer |
VM 6.94 under 21.4 (patch 10) "Military Intelligence" XEmacs Lucid
|
X-Spam-Status |
No, hits=-3.2 required=5.0 tests=EMAIL_ATTRIBUTION,IN_REP_TO,QUOTED_EMAIL_TEXT,REFERENCES, SIGNATURE_SHORT_DENSE,SPAM_PHRASE_00_01 version=2.43
|
X-Spam-Level |
|
Markus Neteler wrote:
> The command crashing is:
>
> r.mapcalc "test=@census1994dryUTM34S"
> 6%
> Segmentation fault
>
> Thanks for looking into this. The map was imported with r.in.shape.
OK. The segfault occurs when G_get_cat() is called for an invalid
category (in this case 0; the cats table starts at 1). However, the
problem actually occurs when G__quant_get_rule_for_d_raster_val()
returns NULL (no quantisation rule matches the argument), and
G_quant_get_cell_value() tries to dereference the NULL pointer.
The problem with r.mapcalc is due to the algorithm which is used for
caching the results of G_get_cat() (in translate_from_cats()). Results
are stored in a btree; however, rather than storing individual values,
blocks of 64 values are stored. When the code is called for a value in
the range 1-63, it attempts to retrieve the values for the range 0-63,
resulting in an error.
The algorithm was copied verbatim from the original r.mapcalc, and I
would guess that it would have the same problem.
There are a variety of possible fixes.
1. Change G_quant_get_cell_value() to handle the case where
G__quant_get_rule_for_d_raster_val() returns NULL. I'm not sure what
it should return in this case (although it shouldn't segfault). In
this particular case (r.mapcalc) it shouldn't actually matter what was
returned, as the value should never actually be used.
2. Change translate_from_cats() to cache individual values in the
btree, rather than blocks. However, this may incur a substantial
penalty in terms of speed or memory usage or both. Discarding the
btree altogether would eliminate any memory issues, but the speed
penalty would probably be horrendous (the quantisation table lookup is
a linear search).
3. Change translate_from_cats() to trap invalid categories which would
cause a segfault. However, it isn't entirely clear what the test
should be; there's a fair amount of code between the call to
G_get_cat() and the quantisation table lookup.
--
Glynn Clements <glynn.clements@virgin.net>
|
|
Tue, Jan 14 2003
10:04:29
|
|
Mail sent by neteler@itc.it
|
|
Return-Path |
<neteler@itc.it>
|
Delivered-To |
grass-bugs@lists.intevation.de
|
Date |
Tue, 14 Jan 2003 10:04:24 +0100
|
From |
Markus Neteler <neteler@itc.it>
|
To |
Glynn Clements <glynn.clements@virgin.net>
|
Cc |
Request Tracker <grass-bugs@intevation.de>, grass5@grass.itc.it
|
Subject |
Re: [GRASS5] [bug #1502] (grass) r.mapcalc error
|
Message-ID |
<20030114100424.A23470@itc.it>
|
Mail-Followup-To |
Glynn Clements <glynn.clements@virgin.net>, Request Tracker <grass-bugs@intevation.de>, grass5@grass.itc.it
|
References |
<20030109082117.D943213B01@lists.intevation.de> <20030110183153.A18566@itc.it> <15903.21591.591195.116054@cerise.nosuchdomain.co.uk> <20030113152253.G7614@itc.it> <15907.28437.650858.933396@cerise.nosuchdomain.co.uk>
|
Mime-Version |
1.0
|
Content-Type |
text/plain; charset=us-ascii
|
Content-Disposition |
inline
|
User-Agent |
Mutt/1.2.5.1i
|
In-Reply-To |
<15907.28437.650858.933396@cerise.nosuchdomain.co.uk>; from glynn.clements@virgin.net on Tue, Jan 14, 2003 at 01:59:49AM +0000
|
X-Spam-Status |
No, hits=-3.3 required=5.0 tests=IN_REP_TO,QUOTED_EMAIL_TEXT,REFERENCES,SPAM_PHRASE_01_02, USER_AGENT,USER_AGENT_MUTT version=2.43
|
X-Spam-Level |
|
On Tue, Jan 14, 2003 at 01:59:49AM +0000, Glynn Clements wrote:
>
> Markus Neteler wrote:
>
> > The command crashing is:
> >
> > r.mapcalc "test=@census1994dryUTM34S"
> > 6%
> > Segmentation fault
> >
> > Thanks for looking into this. The map was imported with r.in.shape.
>
> OK. The segfault occurs when G_get_cat() is called for an invalid
> category (in this case 0; the cats table starts at 1).
Excuse me, but this is not clear to me. The cats file starts with
1:0.0000
Do you mean that 0.0000 is invalid?
> However, the
> problem actually occurs when G__quant_get_rule_for_d_raster_val()
> returns NULL (no quantisation rule matches the argument), and
> G_quant_get_cell_value() tries to dereference the NULL pointer.
>
> The problem with r.mapcalc is due to the algorithm which is used for
> caching the results of G_get_cat() (in translate_from_cats()). Results
> are stored in a btree; however, rather than storing individual values,
> blocks of 64 values are stored. When the code is called for a value in
> the range 1-63, it attempts to retrieve the values for the range 0-63,
> resulting in an error.
Ah, now I see the problem.
> The algorithm was copied verbatim from the original r.mapcalc, and I
> would guess that it would have the same problem.
Correct, I have just tested the old r.mapcalc which also fails.
> There are a variety of possible fixes.
>
> 1. Change G_quant_get_cell_value() to handle the case where
> G__quant_get_rule_for_d_raster_val() returns NULL. I'm not sure what
> it should return in this case (although it shouldn't segfault). In
> this particular case (r.mapcalc) it shouldn't actually matter what was
> returned, as the value should never actually be used.
>
> 2. Change translate_from_cats() to cache individual values in the
> btree, rather than blocks. However, this may incur a substantial
> penalty in terms of speed or memory usage or both. Discarding the
> btree altogether would eliminate any memory issues, but the speed
> penalty would probably be horrendous (the quantisation table lookup is
> a linear search).
My maps are rather big :-)
> 3. Change translate_from_cats() to trap invalid categories which would
> cause a segfault. However, it isn't entirely clear what the test
> should be; there's a fair amount of code between the call to
> G_get_cat() and the quantisation table lookup.
It is surprising that this problem was not yet already fixed some
years ago... Hope we can find a solution!
Markus
|
|
Tue, Jan 14 2003
23:06:10
|
|
Mail sent by glynn.clements@virgin.net
|
|
Return-Path |
<glynn.clements@virgin.net>
|
Delivered-To |
grass-bugs@lists.intevation.de
|
From |
Glynn Clements <glynn.clements@virgin.net>
|
MIME-Version |
1.0
|
Content-Type |
text/plain; charset=us-ascii
|
Content-Transfer-Encoding |
7bit
|
Message-ID |
<15908.5824.182520.55169@cerise.nosuchdomain.co.uk>
|
Date |
Tue, 14 Jan 2003 13:55:12 +0000
|
To |
Markus Neteler <neteler@itc.it>
|
Cc |
Request Tracker <grass-bugs@intevation.de>, grass5@grass.itc.it
|
Subject |
Re: [GRASS5] [bug #1502] (grass) r.mapcalc error
|
In-Reply-To |
<20030114100424.A23470@itc.it>
|
References |
<20030109082117.D943213B01@lists.intevation.de> <20030110183153.A18566@itc.it> <15903.21591.591195.116054@cerise.nosuchdomain.co.uk> <20030113152253.G7614@itc.it> <15907.28437.650858.933396@cerise.nosuchdomain.co.uk> <20030114100424.A23470@itc.it>
|
X-Mailer |
VM 6.94 under 21.4 (patch 10) "Military Intelligence" XEmacs Lucid
|
X-Spam-Status |
No, hits=-3.1 required=5.0 tests=DATE_IN_PAST_06_12,EMAIL_ATTRIBUTION,IN_REP_TO, QUOTED_EMAIL_TEXT,REFERENCES,SIGNATURE_SHORT_DENSE, SPAM_PHRASE_01_02 version=2.43
|
X-Spam-Level |
|
Markus Neteler wrote:
> > > The command crashing is:
> > >
> > > r.mapcalc "test=@census1994dryUTM34S"
> > > 6%
> > > Segmentation fault
> > >
> > > Thanks for looking into this. The map was imported with r.in.shape.
> >
> > OK. The segfault occurs when G_get_cat() is called for an invalid
> > category (in this case 0; the cats table starts at 1).
>
> Excuse me, but this is not clear to me. The cats file starts with
> 1:0.0000
> Do you mean that 0.0000 is invalid?
No; requesting the label for category zero is invalid.
> > There are a variety of possible fixes.
> >
> > 1. Change G_quant_get_cell_value() to handle the case where
> > G__quant_get_rule_for_d_raster_val() returns NULL. I'm not sure what
> > it should return in this case (although it shouldn't segfault). In
> > this particular case (r.mapcalc) it shouldn't actually matter what was
> > returned, as the value should never actually be used.
Actually, returning null should be OK. This would seem to be the
simplest (and most efficient) solution.
--
Glynn Clements <glynn.clements@virgin.net>
|
|
Tue, Mar 18 2003
21:02:25
|
|
Comments added by guest
|
|
I ran into this bug today in a project, and since I was just talking to Markus
today about the need for work on all these little bugs, I delved into it
instead of going around it (as I did last time it happened to me !)... I think
I have fixed it in my own source but would like confirmation:
Markus Neteler wrote:
> > > The command crashing is:
> > >
> > > r.mapcalc "test=@census1994dryUTM34S"
> > > 6%
> > > Segmentation fault
...
Glynn Clements wrote:
> > OK. The segfault occurs when G_get_cat() is called for an invalid
> > category (in this case 0; the cats table starts at 1).
> > There are a variety of possible fixes.
> >
> > 1. Change G_quant_get_cell_value() to handle the case where
> > G__quant_get_rule_for_d_raster_val() returns NULL. I'm not sure what
> > it should return in this case (although it shouldn't segfault). In
> > this particular case (r.mapcalc) it shouldn't actually matter what was
> > returned, as the value should never actually be used.
>Actually, returning null should be OK. This would seem to be the
>simplest (and most efficient) solution.
So - if I understood correctly, this can be fixed by adding a test for a null
return from G_quant_get_rule_for_d_raster_val() (into p), in
G_quant_get_cell_value() in quant.c. So I've made such a change:
bouteloua:/usr/local/src/grass/src/libes/gis> diff quant.c
~/grassmods/libes/gis/quant.c
815c815,818
< return quant_interpolate (p->dLow, p->dHigh, p->cLow, p->cHigh,
---
> if (! p)
> return NO_DATA;
> else
> return quant_interpolate (p->dLow, p->dHigh, p->cLow, p->cHigh,
And now my example problem works (r.mapcalc serial = "int(@serial.cat)" in
this case).
Is this the right correction ? Any negative side effects ?
Scott Mitchell (smitch at geog.utoronto.ca)
|
|
Fri, Apr 11 2003
19:44:03
|
|
Taken by smitchell
|
|
Fri, Apr 11 2003
19:45:23
|
|
Status changed to resolved by smitchell
|
|
Fri, Apr 11 2003
19:45:23
|
|
Mail sent by smitchell
|
|
Should be fixed now.
On March 19, Glynn Clements reported on GRASS5 list:
Scott W. Mitchell wrote:
> I've been looking at bug #1502, which talks about cases where
> r.mapcalc crashes in certain situations with a formula that looks at
> categories, and a lookup of a category for a no data condition occurs
> (very summarized, see the bug tracker for details). Looks like I've just
> run into this bug in my own work, so I was glad to find a reference to it.
[snip]
> I made a small change in quant.c, home of G_quant_get_cell_value, to check
> for a return from G_quant_get_rule_for_d_raster_val before calling
> quant_interpolate:
[snip]
> This seems to work for me, hopefully if this is the right change we can
> apply it ? I'm new to this, let me know if I'm interpreting / suggesting
> incorrectly.
That's what I was suggesting; I'll make that change.
--
Glynn Clements <glynn.clements@virgin.net>
|
|