Details Ticket 3742


Comment | Reply | Take | Resolve


Serial Number 3742
Subject r.out.bin output incorrect compared with r.out.netcdf
Area wish6
Queue grass
Requestors patoledo@ing.uchile.cl
Owner none
Status open
Last User Contact Thu Oct 20 01:08:43 2005 (3 yr ago)
Current Priority 30
Final Priority 70
Due No date assigned
Last Action Thu Oct 20 01:08:43 2005 (3 yr ago)
Created Fri Oct 14 17:45:45 2005 (3 yr ago)

Transaction History Ticket 3742


Fri, Oct 14 2005 17:45:45    Request created by guest  
Subject: r.out.bin output incorrect compared with r.out.netcdf

Platform: GNU/Linux/i386
grass binary for platform: Compiled from Sources
GRASS Version: 6.0.1

r.out.bin fails with the -h option on DCELL maps. 
see http://thread.gmane.org/gmane.comp.gis.grass.user/7618

Looking at the debian source code 6.0.1 it appears that someone
commented out lines of code in r.out.bin module that included an {, a simple
checkout shows that there is an unbalanced } at the end of the file.

See, line 121 of SRC/raster/r.out.bin/main.c
Mon, Oct 17 2005 09:31:41    Mail sent by hamish_nospam@yahoo.com  
Return-Path <hamish_nospam@yahoo.com>
Delivered-To grass-bugs@lists.intevation.de
Date Mon, 17 Oct 2005 20:31:20 +1300
From Hamish <hamish_nospam@yahoo.com>
To Request Tracker <grass-bugs@intevation.de>
Cc grass5@grass.itc.it
Subject Re: [GRASS5] [bug #3742] (grass) r.out.bin output incorrect compared with r.out.netcdf
Message-Id <20051017203120.6b4ccdea.hamish_nospam@yahoo.com>
In-Reply-To <20051014154545.86EC6102BFD@lists.intevation.de>
References <20051014154545.86EC6102BFD@lists.intevation.de>
X-Mailer Sylpheed version 1.0.4 (GTK+ 1.2.10; i386-pc-linux-gnu)
X-Face M<EoB)"*Z~u!,vFhXmw}R_KbdBta*P_=T|rbBL'e1/CQ9;/1g\BU3&!=y8ria$2Uk!HT&BB 8i?|X_+7~1jsy}F~g$2va%3fV`*=L(*cem[@3\yg,G,@rg6/QMJ
Mime-Version 1.0
Content-Type text/plain; charset=US-ASCII
Content-Transfer-Encoding 7bit
X-Spam-Status No, hits=-4.0 tagged_above=-999.0 required=3.0 tests=BAYES_00, FORGED_YAHOO_RCVD
X-Spam-Level
> this bug's URL: http://intevation.de/rt/webrt?serial_num=3742
> ---------------------------------------------------------------------
> 
> Subject: r.out.bin output incorrect compared with r.out.netcdf
> 
> Platform: GNU/Linux/i386
> grass binary for platform: Compiled from Sources
> GRASS Version: 6.0.1
> 
> r.out.bin fails with the -h option on DCELL maps. 

"fails" how? Segmentation Fault? Runs ok, but output not as expected?
Try byte swapping the output of r.out.bin and try again (-s)?


> see http://thread.gmane.org/gmane.comp.gis.grass.user/7618
> > []% r.out.bin -h null=NaN in=srtm out=srtm.grd

note null=NaN should not work [yet it does! G_parser bug!]. As written
r.out.bin will only take an integer for the null value at the command
line. I don't know if this should be allowed to be something of another
type. A char string seems wrong for a binary file, I don't know if floats 
and double are ok or if the cast to int for CELL maps would be "wrong" 
in that case. Allowing float or double for that would be the only way to
get "nan" accepted for null= as it doesn't exist for integers (yet 
G_parser() allows any string for parm->type = TYPE_INTEGER; ??).
Test added to r.out.bin disallow this in future.


my best guess at the bug is this:
[note null_str is defined as an int, not a string]

    /* Set up Parameters for GMT header */
    if (flag.gmt_hd->answer) {
...
	if (flag.swap->answer)
	    TIFFSwabLong((uint32 *) & null_str);
	sprintf(buf, "%d", null_str);
...
    }

then below

	    if (!G_is_null_value(ptr, out_type)) {
... }
	    else {
...
		if (out_type == DCELL_TYPE) {
		    null_val_d = (double)null_str;
		    if (flag.swap->answer)
			TIFFSwabDouble((double *)&null_val_d);
		    fwrite(&null_val_d, sizeof(double), 1, fp);
		}
... }


the GMT -h flag swaps the endianness of null_str and then later the 
binary writer swaps the swaped version? The fix would be to swap a temp 
variable not the main one when writing the .hdr file, the same way the
main binary writing code does. (is swapping it in the header.remark 
string even appropriate??)


> Looking at the debian source code 6.0.1 it appears that someone
> commented out lines of code in r.out.bin module that included an {, a
> simple checkout shows that there is an unbalanced } at the end of the
> file.
> 
> See, line 121 of SRC/raster/r.out.bin/main.c

I don't think that's it. For one thing it won't compile with an 
unbalanced {}. I think that code is trying to allow you to take the
output filename from the map name if output= is not given. *char vs
char[123] is why that fails.

I have cleaned up the code a bit in CVS so it is easier to read and 
fixed a BIL hdr file nodata bug.

I don't use GMT or ArcView day-to-day so can't really test.



Hamish


Tue, Oct 18 2005 17:05:45    Mail sent by glynn@gclements.plus.com  
Return-Path <glynn@gclements.plus.com>
Delivered-To grass-bugs@lists.intevation.de
From Glynn Clements <glynn@gclements.plus.com>
MIME-Version 1.0
Content-Type text/plain; charset=us-ascii
Content-Transfer-Encoding 7bit
Message-ID <17237.3907.170514.142821@cerise.gclements.plus.com>
Date Tue, 18 Oct 2005 16:05:39 +0100
To Hamish <hamish_nospam@yahoo.com>
Cc Request Tracker <grass-bugs@intevation.de>, grass5@grass.itc.it
Subject Re: [GRASS5] [bug #3742] (grass) r.out.bin output incorrect compared with r.out.netcdf
In-Reply-To <20051017203120.6b4ccdea.hamish_nospam@yahoo.com>
References <20051014154545.86EC6102BFD@lists.intevation.de> <20051017203120.6b4ccdea.hamish_nospam@yahoo.com>
X-Mailer VM 7.07 under 21.4 (patch 15) "Security Through Obscurity" XEmacs Lucid
X-Spam-Status No, hits=-4.9 tagged_above=-999.0 required=3.0 tests=BAYES_00
X-Spam-Level
Hamish wrote:

> > Subject: r.out.bin output incorrect compared with r.out.netcdf
> > 
> > Platform: GNU/Linux/i386
> > grass binary for platform: Compiled from Sources
> > GRASS Version: 6.0.1
> > 
> > r.out.bin fails with the -h option on DCELL maps. 
> 
> "fails" how? Segmentation Fault? Runs ok, but output not as expected?
> Try byte swapping the output of r.out.bin and try again (-s)?
> 
> 
> > see http://thread.gmane.org/gmane.comp.gis.grass.user/7618
> > > []% r.out.bin -h null=NaN in=srtm out=srtm.grd
> 
> note null=NaN should not work [yet it does! G_parser bug!].

G_parser() doesn't actually check that argument values are legitimate
unless the ->options field is non-null.

> As written
> r.out.bin will only take an integer for the null value at the command
> line. I don't know if this should be allowed to be something of another
> type. A char string seems wrong for a binary file, I don't know if floats 
> and double are ok or if the cast to int for CELL maps would be "wrong" 
> in that case. Allowing float or double for that would be the only way to
> get "nan" accepted for null= as it doesn't exist for integers (yet 
> G_parser() allows any string for parm->type = TYPE_INTEGER; ??).
> Test added to r.out.bin disallow this in future.

r.out.bin doesn't check the return value from sscanf().

-- 
Glynn Clements <glynn@gclements.plus.com>


Wed, Oct 19 2005 01:02:58    Mail sent by hamish_nospam@yahoo.com  
Return-Path <hamish_nospam@yahoo.com>
Delivered-To grass-bugs@lists.intevation.de
Date Wed, 19 Oct 2005 12:02:38 +1300
From Hamish <hamish_nospam@yahoo.com>
To Glynn Clements <glynn@gclements.plus.com>
Cc grass-bugs@intevation.de, grass5@grass.itc.it
Subject Re: [GRASS5] [bug #3742] (grass) r.out.bin output incorrect compared with r.out.netcdf
Message-Id <20051019120238.0640ca91.hamish_nospam@yahoo.com>
In-Reply-To <17237.3907.170514.142821@cerise.gclements.plus.com>
References <20051014154545.86EC6102BFD@lists.intevation.de> <20051017203120.6b4ccdea.hamish_nospam@yahoo.com> <17237.3907.170514.142821@cerise.gclements.plus.com>
X-Mailer Sylpheed version 1.0.4 (GTK+ 1.2.10; i386-pc-linux-gnu)
X-Face M<EoB)"*Z~u!,vFhXmw}R_KbdBta*P_=T|rbBL'e1/CQ9;/1g\BU3&!=y8ria$2Uk!HT&BB 8i?|X_+7~1jsy}F~g$2va%3fV`*=L(*cem[@3\yg,G,@rg6/QMJ
Mime-Version 1.0
Content-Type text/plain; charset=US-ASCII
Content-Transfer-Encoding 7bit
X-Spam-Status No, hits=-4.0 tagged_above=-999.0 required=3.0 tests=BAYES_00, FORGED_YAHOO_RCVD
X-Spam-Level
> > > Subject: r.out.bin output incorrect compared with r.out.netcdf
..
> > > > []% r.out.bin -h null=NaN in=srtm out=srtm.grd
> > 
> > note null=NaN should not work [yet it does! G_parser bug!].
> 
> G_parser() doesn't actually check that argument values are legitimate
> unless the ->options field is non-null.

I didn't know that. Ok, it's just a string.

Should it be testing the string against opt->type? (,opt->multiple)
(ie is there a reason not to do so, or has it just not been done yet?)


> > As written r.out.bin will only take an integer for the null value at
> > the command line.
...
> r.out.bin doesn't check the return value from sscanf().

I've already added that test in CVS/HEAD.


Glynn, I was hoping you would know the answer to the outstanding GMT 
header code byteswap question?

    /* Set up Parameters for GMT header */
    if (flag.gmt_hd->answer) {
...
	if (flag.swap->answer)
	    TIFFSwabLong((uint32 *) & null_str);
	sprintf(buf, "%d", null_str);
...
    }


does this pollute null_str byteswapping later on in the code?
 (ie swapping an already swapped variable)


also should that (and the rest of the code) test be
if (swapFlag)
 not
if (flag.swap->answer)

or should "swapFlag" be renamed something better?
The mix & match of using both has gotten me a bit confused, hopefully 
it is all ok.



Hamish


Wed, Oct 19 2005 18:51:39    Mail sent by glynn@gclements.plus.com  
Return-Path <glynn@gclements.plus.com>
Delivered-To grass-bugs@lists.intevation.de
From Glynn Clements <glynn@gclements.plus.com>
MIME-Version 1.0
Content-Type text/plain; charset=us-ascii
Content-Transfer-Encoding 7bit
Message-ID <17238.31120.983275.109698@cerise.gclements.plus.com>
Date Wed, 19 Oct 2005 17:51:28 +0100
To Hamish <hamish_nospam@yahoo.com>
Cc grass-bugs@intevation.de, grass5@grass.itc.it
Subject Re: [GRASS5] [bug #3742] (grass) r.out.bin output incorrect compared with r.out.netcdf
In-Reply-To <20051019120238.0640ca91.hamish_nospam@yahoo.com>
References <20051014154545.86EC6102BFD@lists.intevation.de> <20051017203120.6b4ccdea.hamish_nospam@yahoo.com> <17237.3907.170514.142821@cerise.gclements.plus.com> <20051019120238.0640ca91.hamish_nospam@yahoo.com>
X-Mailer VM 7.07 under 21.4 (patch 15) "Security Through Obscurity" XEmacs Lucid
X-Spam-Status No, hits=-4.9 tagged_above=-999.0 required=3.0 tests=BAYES_00
X-Spam-Level
Hamish wrote:

> > > > Subject: r.out.bin output incorrect compared with r.out.netcdf
> ..
> > > > > []% r.out.bin -h null=NaN in=srtm out=srtm.grd
> > > 
> > > note null=NaN should not work [yet it does! G_parser bug!].
> > 
> > G_parser() doesn't actually check that argument values are legitimate
> > unless the ->options field is non-null.
> 
> I didn't know that. Ok, it's just a string.
> 
> Should it be testing the string against opt->type? (,opt->multiple)

Arguably.

> (ie is there a reason not to do so, or has it just not been done yet?)

Given that it hasn't been done so far, it's always possible that some
modules are setting opt->type to TYPE_INTEGER but allowing arguments
which aren't actually integers. Adding the type-checking would prevent
such "enhancements" from working.

I'd say: go for it. If it breaks a module, fix the module (i.e. change
it to use TYPE_STRING instead).

> Glynn, I was hoping you would know the answer to the outstanding GMT 
> header code byteswap question?
> 
>     /* Set up Parameters for GMT header */
>     if (flag.gmt_hd->answer) {
> ...
> 	if (flag.swap->answer)
> 	    TIFFSwabLong((uint32 *) & null_str);
> 	sprintf(buf, "%d", null_str);
> ...
>     }
> 
> does this pollute null_str byteswapping later on in the code?
>  (ie swapping an already swapped variable)

It looks that way.

Also, this:

	fprintf(fp_1, "nodata %f\n", null_str);

is bogus, as null_str is an int, but fprintf will be expecting a
double.

I suggest that null_str should actually be a double, and the
conversion of the argument to null= should have separate integer/FP
cases depending upon the input map type.

> also should that (and the rest of the code) test be
> if (swapFlag)
>  not
> if (flag.swap->answer)

No.

> or should "swapFlag" be renamed something better?

Yes. It should be called something like outputLittleEndian or
outputIntel, as that's what it represents.

On a related note, the meaning of the -s flag is ill-conceived; the
output shouldn't depend upon the host's byte-order, only upon the
options.

It would be better to have e.g. endian={big,little,host}, so you can
use endian=big or endian=little to get consistent results regardless
of the byte order of the host on which you happen to be executing the
command, or endian=host if you just want whatever will be most
efficient on the system you're using.

-- 
Glynn Clements <glynn@gclements.plus.com>


Thu, Oct 20 2005 01:08:43    Mail sent by hamish_nospam@yahoo.com  
Return-Path <hamish_nospam@yahoo.com>
Delivered-To grass-bugs@lists.intevation.de
Date Thu, 20 Oct 2005 12:08:25 +1300
From Hamish <hamish_nospam@yahoo.com>
To Glynn Clements <glynn@gclements.plus.com>
Cc grass-bugs@intevation.de, grass5@grass.itc.it
Subject Re: [GRASS5] [bug #3742] (grass) r.out.bin output incorrect compared with r.out.netcdf
Message-Id <20051020120825.1a5731b6.hamish_nospam@yahoo.com>
In-Reply-To <17238.31120.983275.109698@cerise.gclements.plus.com>
References <20051014154545.86EC6102BFD@lists.intevation.de> <20051017203120.6b4ccdea.hamish_nospam@yahoo.com> <17237.3907.170514.142821@cerise.gclements.plus.com> <20051019120238.0640ca91.hamish_nospam@yahoo.com> <17238.31120.983275.109698@cerise.gclements.plus.com>
X-Mailer Sylpheed version 1.0.4 (GTK+ 1.2.10; i386-pc-linux-gnu)
X-Face M<EoB)"*Z~u!,vFhXmw}R_KbdBta*P_=T|rbBL'e1/CQ9;/1g\BU3&!=y8ria$2Uk!HT&BB 8i?|X_+7~1jsy}F~g$2va%3fV`*=L(*cem[@3\yg,G,@rg6/QMJ
Mime-Version 1.0
Content-Type text/plain; charset=US-ASCII
Content-Transfer-Encoding 7bit
X-Spam-Status No, hits=-4.0 tagged_above=-999.0 required=3.0 tests=BAYES_00, FORGED_YAHOO_RCVD
X-Spam-Level
> > > > > > []% r.out.bin -h null=NaN in=srtm out=srtm.grd
> > > > 
> > > > note null=NaN should not work [yet it does! G_parser bug!].
> > > 
> > > G_parser() doesn't actually check that argument values are
> > > legitimate unless the ->options field is non-null.
> > 
> > I didn't know that. Ok, it's just a string.
> > 
> > Should it be testing the string against opt->type? (,opt->multiple)
> 
> Arguably.
> 
> > (ie is there a reason not to do so, or has it just not been done
> > yet?)
> 
> Given that it hasn't been done so far, it's always possible that some
> modules are setting opt->type to TYPE_INTEGER but allowing arguments
> which aren't actually integers. Adding the type-checking would prevent
> such "enhancements" from working.
> 
> I'd say: go for it. If it breaks a module, fix the module (i.e. change
> it to use TYPE_STRING instead).

I came across this the other day with d.grid. In order to take ddd:mm:ss
input for lat/lon grid spacing it sets size= as TYPE_STRING. If this
confuses on the help page or GUI hinting, opt->key_desc can be set.


> > Glynn, I was hoping you would know the answer to the outstanding GMT
> > 
> > header code byteswap question?
> > 
> >     /* Set up Parameters for GMT header */
> >     if (flag.gmt_hd->answer) {
> > ...
> > 	if (flag.swap->answer)
> > 	    TIFFSwabLong((uint32 *) & null_str);
> > 	sprintf(buf, "%d", null_str);
> > ...
> >     }
> > 
> > does this pollute null_str byteswapping later on in the code?
> >  (ie swapping an already swapped variable)
> 
> It looks that way.

ok; I don't know if this swap is actually needed, but will copy to a temp
variable first to isolate any damage.

 
> Also, this:
> 
> 	fprintf(fp_1, "nodata %f\n", null_str);
> 
> is bogus, as null_str is an int, but fprintf will be expecting a
> double.

I fixed that in CVS the other day.

 
> I suggest that null_str should actually be a double, and the
> conversion of the argument to null= should have separate integer/FP
> cases depending upon the input map type.

ok, sounds reasonable.
 

> > or should "swapFlag" be renamed something better?
> 
> Yes. It should be called something like outputLittleEndian or
> outputIntel, as that's what it represents.

ok

> On a related note, the meaning of the -s flag is ill-conceived; the
> output shouldn't depend upon the host's byte-order, only upon the
> options.
> 
> It would be better to have e.g. endian={big,little,host}, so you can
> use endian=big or endian=little to get consistent results regardless
> of the byte order of the host on which you happen to be executing the
> command, or endian=host if you just want whatever will be most
> efficient on the system you're using.

I am going to leave this to someone with more experience with big
endian systems and the ability to test the outcome in a real sense.


Hamish


Comment | Reply | Take | Resolve

You are currently authenticated as guest.
[Show Configuration] [Login as another user]

Users Guide - Mail Commands - Homepage of RequestTracker 1.0.7 - list any request