Fri, Oct 14 2005
17:45:45
|
|
Request created by guest
|
|
Subject: r.out.bin output incorrect compared with r.out.netcdf
Platform: GNU/Linux/i386
grass binary for platform: Compiled from Sources
GRASS Version: 6.0.1
r.out.bin fails with the -h option on DCELL maps.
see http://thread.gmane.org/gmane.comp.gis.grass.user/7618
Looking at the debian source code 6.0.1 it appears that someone
commented out lines of code in r.out.bin module that included an {, a simple
checkout shows that there is an unbalanced } at the end of the file.
See, line 121 of SRC/raster/r.out.bin/main.c |
|
Mon, Oct 17 2005
09:31:41
|
|
Mail sent by hamish_nospam@yahoo.com
|
|
Return-Path |
<hamish_nospam@yahoo.com>
|
Delivered-To |
grass-bugs@lists.intevation.de
|
Date |
Mon, 17 Oct 2005 20:31:20 +1300
|
From |
Hamish <hamish_nospam@yahoo.com>
|
To |
Request Tracker <grass-bugs@intevation.de>
|
Cc |
grass5@grass.itc.it
|
Subject |
Re: [GRASS5] [bug #3742] (grass) r.out.bin output incorrect compared with r.out.netcdf
|
Message-Id |
<20051017203120.6b4ccdea.hamish_nospam@yahoo.com>
|
In-Reply-To |
<20051014154545.86EC6102BFD@lists.intevation.de>
|
References |
<20051014154545.86EC6102BFD@lists.intevation.de>
|
X-Mailer |
Sylpheed version 1.0.4 (GTK+ 1.2.10; i386-pc-linux-gnu)
|
X-Face |
M<EoB)"*Z~u!,vFhXmw}R_KbdBta*P_=T|rbBL'e1/CQ9;/1g\BU3&!=y8ria$2Uk!HT&BB 8i?|X_+7~1jsy}F~g$2va%3fV`*=L(*cem[@3\yg,G,@rg6/QMJ
|
Mime-Version |
1.0
|
Content-Type |
text/plain; charset=US-ASCII
|
Content-Transfer-Encoding |
7bit
|
X-Spam-Status |
No, hits=-4.0 tagged_above=-999.0 required=3.0 tests=BAYES_00, FORGED_YAHOO_RCVD
|
X-Spam-Level |
|
> this bug's URL: http://intevation.de/rt/webrt?serial_num=3742
> ---------------------------------------------------------------------
>
> Subject: r.out.bin output incorrect compared with r.out.netcdf
>
> Platform: GNU/Linux/i386
> grass binary for platform: Compiled from Sources
> GRASS Version: 6.0.1
>
> r.out.bin fails with the -h option on DCELL maps.
"fails" how? Segmentation Fault? Runs ok, but output not as expected?
Try byte swapping the output of r.out.bin and try again (-s)?
> see http://thread.gmane.org/gmane.comp.gis.grass.user/7618
> > []% r.out.bin -h null=NaN in=srtm out=srtm.grd
note null=NaN should not work [yet it does! G_parser bug!]. As written
r.out.bin will only take an integer for the null value at the command
line. I don't know if this should be allowed to be something of another
type. A char string seems wrong for a binary file, I don't know if floats
and double are ok or if the cast to int for CELL maps would be "wrong"
in that case. Allowing float or double for that would be the only way to
get "nan" accepted for null= as it doesn't exist for integers (yet
G_parser() allows any string for parm->type = TYPE_INTEGER; ??).
Test added to r.out.bin disallow this in future.
my best guess at the bug is this:
[note null_str is defined as an int, not a string]
/* Set up Parameters for GMT header */
if (flag.gmt_hd->answer) {
...
if (flag.swap->answer)
TIFFSwabLong((uint32 *) & null_str);
sprintf(buf, "%d", null_str);
...
}
then below
if (!G_is_null_value(ptr, out_type)) {
... }
else {
...
if (out_type == DCELL_TYPE) {
null_val_d = (double)null_str;
if (flag.swap->answer)
TIFFSwabDouble((double *)&null_val_d);
fwrite(&null_val_d, sizeof(double), 1, fp);
}
... }
the GMT -h flag swaps the endianness of null_str and then later the
binary writer swaps the swaped version? The fix would be to swap a temp
variable not the main one when writing the .hdr file, the same way the
main binary writing code does. (is swapping it in the header.remark
string even appropriate??)
> Looking at the debian source code 6.0.1 it appears that someone
> commented out lines of code in r.out.bin module that included an {, a
> simple checkout shows that there is an unbalanced } at the end of the
> file.
>
> See, line 121 of SRC/raster/r.out.bin/main.c
I don't think that's it. For one thing it won't compile with an
unbalanced {}. I think that code is trying to allow you to take the
output filename from the map name if output= is not given. *char vs
char[123] is why that fails.
I have cleaned up the code a bit in CVS so it is easier to read and
fixed a BIL hdr file nodata bug.
I don't use GMT or ArcView day-to-day so can't really test.
Hamish
|
|
Tue, Oct 18 2005
17:05:45
|
|
Mail sent by glynn@gclements.plus.com
|
|
Return-Path |
<glynn@gclements.plus.com>
|
Delivered-To |
grass-bugs@lists.intevation.de
|
From |
Glynn Clements <glynn@gclements.plus.com>
|
MIME-Version |
1.0
|
Content-Type |
text/plain; charset=us-ascii
|
Content-Transfer-Encoding |
7bit
|
Message-ID |
<17237.3907.170514.142821@cerise.gclements.plus.com>
|
Date |
Tue, 18 Oct 2005 16:05:39 +0100
|
To |
Hamish <hamish_nospam@yahoo.com>
|
Cc |
Request Tracker <grass-bugs@intevation.de>, grass5@grass.itc.it
|
Subject |
Re: [GRASS5] [bug #3742] (grass) r.out.bin output incorrect compared with r.out.netcdf
|
In-Reply-To |
<20051017203120.6b4ccdea.hamish_nospam@yahoo.com>
|
References |
<20051014154545.86EC6102BFD@lists.intevation.de> <20051017203120.6b4ccdea.hamish_nospam@yahoo.com>
|
X-Mailer |
VM 7.07 under 21.4 (patch 15) "Security Through Obscurity" XEmacs Lucid
|
X-Spam-Status |
No, hits=-4.9 tagged_above=-999.0 required=3.0 tests=BAYES_00
|
X-Spam-Level |
|
Hamish wrote:
> > Subject: r.out.bin output incorrect compared with r.out.netcdf
> >
> > Platform: GNU/Linux/i386
> > grass binary for platform: Compiled from Sources
> > GRASS Version: 6.0.1
> >
> > r.out.bin fails with the -h option on DCELL maps.
>
> "fails" how? Segmentation Fault? Runs ok, but output not as expected?
> Try byte swapping the output of r.out.bin and try again (-s)?
>
>
> > see http://thread.gmane.org/gmane.comp.gis.grass.user/7618
> > > []% r.out.bin -h null=NaN in=srtm out=srtm.grd
>
> note null=NaN should not work [yet it does! G_parser bug!].
G_parser() doesn't actually check that argument values are legitimate
unless the ->options field is non-null.
> As written
> r.out.bin will only take an integer for the null value at the command
> line. I don't know if this should be allowed to be something of another
> type. A char string seems wrong for a binary file, I don't know if floats
> and double are ok or if the cast to int for CELL maps would be "wrong"
> in that case. Allowing float or double for that would be the only way to
> get "nan" accepted for null= as it doesn't exist for integers (yet
> G_parser() allows any string for parm->type = TYPE_INTEGER; ??).
> Test added to r.out.bin disallow this in future.
r.out.bin doesn't check the return value from sscanf().
--
Glynn Clements <glynn@gclements.plus.com>
|
|
Wed, Oct 19 2005
01:02:58
|
|
Mail sent by hamish_nospam@yahoo.com
|
|
Return-Path |
<hamish_nospam@yahoo.com>
|
Delivered-To |
grass-bugs@lists.intevation.de
|
Date |
Wed, 19 Oct 2005 12:02:38 +1300
|
From |
Hamish <hamish_nospam@yahoo.com>
|
To |
Glynn Clements <glynn@gclements.plus.com>
|
Cc |
grass-bugs@intevation.de, grass5@grass.itc.it
|
Subject |
Re: [GRASS5] [bug #3742] (grass) r.out.bin output incorrect compared with r.out.netcdf
|
Message-Id |
<20051019120238.0640ca91.hamish_nospam@yahoo.com>
|
In-Reply-To |
<17237.3907.170514.142821@cerise.gclements.plus.com>
|
References |
<20051014154545.86EC6102BFD@lists.intevation.de> <20051017203120.6b4ccdea.hamish_nospam@yahoo.com> <17237.3907.170514.142821@cerise.gclements.plus.com>
|
X-Mailer |
Sylpheed version 1.0.4 (GTK+ 1.2.10; i386-pc-linux-gnu)
|
X-Face |
M<EoB)"*Z~u!,vFhXmw}R_KbdBta*P_=T|rbBL'e1/CQ9;/1g\BU3&!=y8ria$2Uk!HT&BB 8i?|X_+7~1jsy}F~g$2va%3fV`*=L(*cem[@3\yg,G,@rg6/QMJ
|
Mime-Version |
1.0
|
Content-Type |
text/plain; charset=US-ASCII
|
Content-Transfer-Encoding |
7bit
|
X-Spam-Status |
No, hits=-4.0 tagged_above=-999.0 required=3.0 tests=BAYES_00, FORGED_YAHOO_RCVD
|
X-Spam-Level |
|
> > > Subject: r.out.bin output incorrect compared with r.out.netcdf
..
> > > > []% r.out.bin -h null=NaN in=srtm out=srtm.grd
> >
> > note null=NaN should not work [yet it does! G_parser bug!].
>
> G_parser() doesn't actually check that argument values are legitimate
> unless the ->options field is non-null.
I didn't know that. Ok, it's just a string.
Should it be testing the string against opt->type? (,opt->multiple)
(ie is there a reason not to do so, or has it just not been done yet?)
> > As written r.out.bin will only take an integer for the null value at
> > the command line.
...
> r.out.bin doesn't check the return value from sscanf().
I've already added that test in CVS/HEAD.
Glynn, I was hoping you would know the answer to the outstanding GMT
header code byteswap question?
/* Set up Parameters for GMT header */
if (flag.gmt_hd->answer) {
...
if (flag.swap->answer)
TIFFSwabLong((uint32 *) & null_str);
sprintf(buf, "%d", null_str);
...
}
does this pollute null_str byteswapping later on in the code?
(ie swapping an already swapped variable)
also should that (and the rest of the code) test be
if (swapFlag)
not
if (flag.swap->answer)
or should "swapFlag" be renamed something better?
The mix & match of using both has gotten me a bit confused, hopefully
it is all ok.
Hamish
|
|
Wed, Oct 19 2005
18:51:39
|
|
Mail sent by glynn@gclements.plus.com
|
|
Return-Path |
<glynn@gclements.plus.com>
|
Delivered-To |
grass-bugs@lists.intevation.de
|
From |
Glynn Clements <glynn@gclements.plus.com>
|
MIME-Version |
1.0
|
Content-Type |
text/plain; charset=us-ascii
|
Content-Transfer-Encoding |
7bit
|
Message-ID |
<17238.31120.983275.109698@cerise.gclements.plus.com>
|
Date |
Wed, 19 Oct 2005 17:51:28 +0100
|
To |
Hamish <hamish_nospam@yahoo.com>
|
Cc |
grass-bugs@intevation.de, grass5@grass.itc.it
|
Subject |
Re: [GRASS5] [bug #3742] (grass) r.out.bin output incorrect compared with r.out.netcdf
|
In-Reply-To |
<20051019120238.0640ca91.hamish_nospam@yahoo.com>
|
References |
<20051014154545.86EC6102BFD@lists.intevation.de> <20051017203120.6b4ccdea.hamish_nospam@yahoo.com> <17237.3907.170514.142821@cerise.gclements.plus.com> <20051019120238.0640ca91.hamish_nospam@yahoo.com>
|
X-Mailer |
VM 7.07 under 21.4 (patch 15) "Security Through Obscurity" XEmacs Lucid
|
X-Spam-Status |
No, hits=-4.9 tagged_above=-999.0 required=3.0 tests=BAYES_00
|
X-Spam-Level |
|
Hamish wrote:
> > > > Subject: r.out.bin output incorrect compared with r.out.netcdf
> ..
> > > > > []% r.out.bin -h null=NaN in=srtm out=srtm.grd
> > >
> > > note null=NaN should not work [yet it does! G_parser bug!].
> >
> > G_parser() doesn't actually check that argument values are legitimate
> > unless the ->options field is non-null.
>
> I didn't know that. Ok, it's just a string.
>
> Should it be testing the string against opt->type? (,opt->multiple)
Arguably.
> (ie is there a reason not to do so, or has it just not been done yet?)
Given that it hasn't been done so far, it's always possible that some
modules are setting opt->type to TYPE_INTEGER but allowing arguments
which aren't actually integers. Adding the type-checking would prevent
such "enhancements" from working.
I'd say: go for it. If it breaks a module, fix the module (i.e. change
it to use TYPE_STRING instead).
> Glynn, I was hoping you would know the answer to the outstanding GMT
> header code byteswap question?
>
> /* Set up Parameters for GMT header */
> if (flag.gmt_hd->answer) {
> ...
> if (flag.swap->answer)
> TIFFSwabLong((uint32 *) & null_str);
> sprintf(buf, "%d", null_str);
> ...
> }
>
> does this pollute null_str byteswapping later on in the code?
> (ie swapping an already swapped variable)
It looks that way.
Also, this:
fprintf(fp_1, "nodata %f\n", null_str);
is bogus, as null_str is an int, but fprintf will be expecting a
double.
I suggest that null_str should actually be a double, and the
conversion of the argument to null= should have separate integer/FP
cases depending upon the input map type.
> also should that (and the rest of the code) test be
> if (swapFlag)
> not
> if (flag.swap->answer)
No.
> or should "swapFlag" be renamed something better?
Yes. It should be called something like outputLittleEndian or
outputIntel, as that's what it represents.
On a related note, the meaning of the -s flag is ill-conceived; the
output shouldn't depend upon the host's byte-order, only upon the
options.
It would be better to have e.g. endian={big,little,host}, so you can
use endian=big or endian=little to get consistent results regardless
of the byte order of the host on which you happen to be executing the
command, or endian=host if you just want whatever will be most
efficient on the system you're using.
--
Glynn Clements <glynn@gclements.plus.com>
|
|
Thu, Oct 20 2005
01:08:43
|
|
Mail sent by hamish_nospam@yahoo.com
|
|
Return-Path |
<hamish_nospam@yahoo.com>
|
Delivered-To |
grass-bugs@lists.intevation.de
|
Date |
Thu, 20 Oct 2005 12:08:25 +1300
|
From |
Hamish <hamish_nospam@yahoo.com>
|
To |
Glynn Clements <glynn@gclements.plus.com>
|
Cc |
grass-bugs@intevation.de, grass5@grass.itc.it
|
Subject |
Re: [GRASS5] [bug #3742] (grass) r.out.bin output incorrect compared with r.out.netcdf
|
Message-Id |
<20051020120825.1a5731b6.hamish_nospam@yahoo.com>
|
In-Reply-To |
<17238.31120.983275.109698@cerise.gclements.plus.com>
|
References |
<20051014154545.86EC6102BFD@lists.intevation.de> <20051017203120.6b4ccdea.hamish_nospam@yahoo.com> <17237.3907.170514.142821@cerise.gclements.plus.com> <20051019120238.0640ca91.hamish_nospam@yahoo.com> <17238.31120.983275.109698@cerise.gclements.plus.com>
|
X-Mailer |
Sylpheed version 1.0.4 (GTK+ 1.2.10; i386-pc-linux-gnu)
|
X-Face |
M<EoB)"*Z~u!,vFhXmw}R_KbdBta*P_=T|rbBL'e1/CQ9;/1g\BU3&!=y8ria$2Uk!HT&BB 8i?|X_+7~1jsy}F~g$2va%3fV`*=L(*cem[@3\yg,G,@rg6/QMJ
|
Mime-Version |
1.0
|
Content-Type |
text/plain; charset=US-ASCII
|
Content-Transfer-Encoding |
7bit
|
X-Spam-Status |
No, hits=-4.0 tagged_above=-999.0 required=3.0 tests=BAYES_00, FORGED_YAHOO_RCVD
|
X-Spam-Level |
|
> > > > > > []% r.out.bin -h null=NaN in=srtm out=srtm.grd
> > > >
> > > > note null=NaN should not work [yet it does! G_parser bug!].
> > >
> > > G_parser() doesn't actually check that argument values are
> > > legitimate unless the ->options field is non-null.
> >
> > I didn't know that. Ok, it's just a string.
> >
> > Should it be testing the string against opt->type? (,opt->multiple)
>
> Arguably.
>
> > (ie is there a reason not to do so, or has it just not been done
> > yet?)
>
> Given that it hasn't been done so far, it's always possible that some
> modules are setting opt->type to TYPE_INTEGER but allowing arguments
> which aren't actually integers. Adding the type-checking would prevent
> such "enhancements" from working.
>
> I'd say: go for it. If it breaks a module, fix the module (i.e. change
> it to use TYPE_STRING instead).
I came across this the other day with d.grid. In order to take ddd:mm:ss
input for lat/lon grid spacing it sets size= as TYPE_STRING. If this
confuses on the help page or GUI hinting, opt->key_desc can be set.
> > Glynn, I was hoping you would know the answer to the outstanding GMT
> >
> > header code byteswap question?
> >
> > /* Set up Parameters for GMT header */
> > if (flag.gmt_hd->answer) {
> > ...
> > if (flag.swap->answer)
> > TIFFSwabLong((uint32 *) & null_str);
> > sprintf(buf, "%d", null_str);
> > ...
> > }
> >
> > does this pollute null_str byteswapping later on in the code?
> > (ie swapping an already swapped variable)
>
> It looks that way.
ok; I don't know if this swap is actually needed, but will copy to a temp
variable first to isolate any damage.
> Also, this:
>
> fprintf(fp_1, "nodata %f\n", null_str);
>
> is bogus, as null_str is an int, but fprintf will be expecting a
> double.
I fixed that in CVS the other day.
> I suggest that null_str should actually be a double, and the
> conversion of the argument to null= should have separate integer/FP
> cases depending upon the input map type.
ok, sounds reasonable.
> > or should "swapFlag" be renamed something better?
>
> Yes. It should be called something like outputLittleEndian or
> outputIntel, as that's what it represents.
ok
> On a related note, the meaning of the -s flag is ill-conceived; the
> output shouldn't depend upon the host's byte-order, only upon the
> options.
>
> It would be better to have e.g. endian={big,little,host}, so you can
> use endian=big or endian=little to get consistent results regardless
> of the byte order of the host on which you happen to be executing the
> command, or endian=host if you just want whatever will be most
> efficient on the system you're using.
I am going to leave this to someone with more experience with big
endian systems and the ability to test the outcome in a real sense.
Hamish
|
|