Details Ticket 4070


Comment | Reply | Take | Open


Serial Number 4070
Subject r.in.srtm fails to assign null() to nodata properly - propably a GDAL bug
Area grass6
Queue grass
Requestors werchowyna@epf.pl
Owner none
Status resolved
Last User Contact Thu Feb 9 11:25:43 2006 (3 yr ago)
Current Priority 30
Final Priority 70
Due No date assigned
Last Action Thu Feb 9 11:25:43 2006 (3 yr ago)
Created Wed Feb 8 11:51:55 2006 (3 yr ago)

Transaction History Ticket 4070


Wed, Feb 8 2006 11:51:55    Request created by guest  
Subject: r.in.srtm fails to assign null() to nodata properly - propably a GDAL
bug

Platform: GNU/Linux/x86
grass obtained from: Trento Italy site
grass binary for platform: Compiled from Sources
GRASS Version: CVS 2006.02.07

NOTE: this doesn't happen with all SRTM 3' tiles. But e.g. this one is affected.
I'm pretty sure it's not a problem of the tile itself - only r.in.srtm fails
(due to r.in.gdal failure), however r.in.bin handles it fine. Please read below.
# Download ftp://e0srp01u.ecs.nasa.gov/srtm/version1/Eurasia/N51E016.hgt.zip
# Import it in a WGS84 ll location (first you might need to add "-o" switch
# to r.in.gdal command in r.in.srtm script, as there is some error preventing
# GDAL from recognizing Grass location settings anymore).

$ r.in.srtm input=N51E016 output=N51E016_v1
Extracting N51E016...
Archive:  N51E016.hgt.zip
  inflating: N51E016.hgt
Converting input file to BIL...
Over-riding projection check.
Proceeding with import...
 100%
CREATING SUPPORT FILES FOR N51E016_v1
SETTING GREY COLOR TABLE FOR N51E016_v1 (16bit, image range)
r.in.gdal complete.
WARNING: Your color rules do not cover the whole range of data!

							^
							|
						  # NOTE THIS!!!


Color table for [N51E016_v1] set to srtm
Done: generated map N51E016_v1
(Note: Holes in the data can be closed with 'r.fillnulls' using splines)




# Indeed, the max value is wrong:

$ r.info -r N51E016_v1
min=55
max=32768




# The problem is somwhere in the GDAL - the intermediate .bil file that
# r.in.srtm creates, has wrong value range reported by gdalinfo:

$ gdalinfo -mm N51E016.bil
Driver: EHdr/ESRI .hdr Labelled
Size is 1201, 1201
Coordinate System is:
GEOGCS["wgs84",
    DATUM["WGS_1984",
        SPHEROID["wgs84",6378137,298.257223563],
        TOWGS84[0.000000,0.000000,0.000000]],
    PRIMEM["Greenwich",0],
    UNIT["degree",0.0174532925199433]]
Origin = (15.999583,52.000417)
Pixel Size = (0.00083333,-0.00083333)
Corner Coordinates:
Upper Left  (  15.9995833,  52.0004167) ( 15d59'58.50"E, 52d 0'1.50"N)
Lower Left  (  15.9995833,  50.9995833) ( 15d59'58.50"E, 50d59'58.50"N)
Upper Right (  17.0004167,  52.0004167) ( 17d 0'1.50"E, 52d 0'1.50"N)
Lower Right (  17.0004167,  50.9995833) ( 17d 0'1.50"E, 50d59'58.50"N)
Center      (  16.5000000,  51.5000000) ( 16d30'0.00"E, 51d30'0.00"N)
Band 1 Block=1201x1 Type=UInt16, ColorInterp=Undefined
    Computed Min/Max=55.000,32768.000
  NoData Value=-32768
				^
				|
			  # WRONG !!!



# But - importing the same N51E016.bil with r.in.bin in XY location works like
# a charm, thus proves the problem is somewhere in GDAL:

$ r.in.bin input=/home/john/tmp/srtm_v1/N51E016.bil output=N51E016_v1_rinbin
bytes=2 rows=1201 cols=1201 -b
Byte Swapping Turned On!
Using N=1201.000000 S=0.000000 E=1201.000000 W=0.000000
Percent Complete:  100%
CREATING SUPPORT FILES FOR N51E016_v1_rinbin



# See how CORRECT min/max differs now from the r.in.gdal output:

$ r.info -r N51E016_v1_rinbin
min=-32768
max=483



# And here's what it should be after setting -32768 to null:

$ r.null map=N51E016_v1_rinbin setnull=-32768
Writing new data for [N51E016_v1_rinbin]...  100%

$ r.info -r N51E016_v1_rinbin
min=55
max=483



Maciek
Wed, Feb 8 2006 11:55:09    Mail sent by guest  
Frank,

Please take a look at it. This might be a bug in GDAL handling of .bil rasters.
NOTE: this doesn't happen with all SRTM 3' tiles. But e.g. this one is
affected. I'm pretty sure it's not a problem of the tile itself - only
r.in.srtm fails (due to r.in.gdal failure), however r.in.bin handles it fine.
Please read below.

# Download ftp://e0srp01u.ecs.nasa.gov/srtm/version1/Eurasia/N51E016.hgt.zip
# Import it in a WGS84 ll location (first you might need to add "-o" switch
# to r.in.gdal command in r.in.srtm script, as there is some error preventing
# GDAL from recognizing Grass location settings anymore).

$ r.in.srtm input=N51E016 output=N51E016_v1
Extracting N51E016...
Archive:  N51E016.hgt.zip
  inflating: N51E016.hgt
Converting input file to BIL...
Over-riding projection check.
Proceeding with import...
 100%
CREATING SUPPORT FILES FOR N51E016_v1
SETTING GREY COLOR TABLE FOR N51E016_v1 (16bit, image range)
r.in.gdal complete.
WARNING: Your color rules do not cover the whole range of data!

							^
							|
						  # NOTE THIS!!!


Color table for [N51E016_v1] set to srtm
Done: generated map N51E016_v1
(Note: Holes in the data can be closed with 'r.fillnulls' using splines)




# Indeed, the max value is wrong:

$ r.info -r N51E016_v1
min=55
max=32768




# The problem is somwhere in the GDAL - the intermediate .bil file that
# r.in.srtm creates, has wrong value range reported by gdalinfo:

$ gdalinfo -mm N51E016.bil
Driver: EHdr/ESRI .hdr Labelled
Size is 1201, 1201
Coordinate System is:
GEOGCS["wgs84",
    DATUM["WGS_1984",
        SPHEROID["wgs84",6378137,298.257223563],
        TOWGS84[0.000000,0.000000,0.000000]],
    PRIMEM["Greenwich",0],
    UNIT["degree",0.0174532925199433]]
Origin = (15.999583,52.000417)
Pixel Size = (0.00083333,-0.00083333)
Corner Coordinates:
Upper Left  (  15.9995833,  52.0004167) ( 15d59'58.50"E, 52d 0'1.50"N)
Lower Left  (  15.9995833,  50.9995833) ( 15d59'58.50"E, 50d59'58.50"N)
Upper Right (  17.0004167,  52.0004167) ( 17d 0'1.50"E, 52d 0'1.50"N)
Lower Right (  17.0004167,  50.9995833) ( 17d 0'1.50"E, 50d59'58.50"N)
Center      (  16.5000000,  51.5000000) ( 16d30'0.00"E, 51d30'0.00"N)
Band 1 Block=1201x1 Type=UInt16, ColorInterp=Undefined
    Computed Min/Max=55.000,32768.000
  NoData Value=-32768
				^
				|
			  # WRONG !!!



# But - importing the same N51E016.bil with r.in.bin in XY location works like
# a charm, thus proves the problem is somewhere in GDAL:

$ r.in.bin input=/home/john/tmp/srtm_v1/N51E016.bil output=N51E016_v1_rinbin
bytes=2 rows=1201 cols=1201 -b
Byte Swapping Turned On!
Using N=1201.000000 S=0.000000 E=1201.000000 W=0.000000
Percent Complete:  100%
CREATING SUPPORT FILES FOR N51E016_v1_rinbin



# See how CORRECT min/max differs now from the r.in.gdal output:

$ r.info -r N51E016_v1_rinbin
min=-32768
max=483



# And here's what it should be after setting -32768 to null:

$ r.null map=N51E016_v1_rinbin setnull=-32768
Writing new data for [N51E016_v1_rinbin]...  100%

$ r.info -r N51E016_v1_rinbin
min=55
max=483



Maciek
Thu, Feb 9 2006 05:26:14    Mail sent by warmerdam@pobox.com  
Return-Path <fwarmerdam@gmail.com>
Delivered-To grass-bugs@lists.intevation.de
DomainKey-Signature a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:sender:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=PSH1EulyU9r5ZEldJtjyjafhvuNT9slbuggDMmfVgpcKV4Z396LJrXg6JeuPOQ7cOuAS8kIwtaqdNu3jZYx00zCbtdd7FC75A5uhNnKwf03D7fITOTDGfLZlYxcf+WUH54gMo5+AhMXyY50n4KP0nAYk4iriGArA9/NTr9TVdsw=
Message-ID <931f8ea90602082025p2f36ea2emcfd937b9dfbaff33@mail.gmail.com>
Date Wed, 8 Feb 2006 23:25:40 -0500
From Frank Warmerdam <warmerdam@pobox.com>
Sender fwarmerdam@gmail.com
To guest user via RT <grass-bugs@intevation.de>
Subject Re: [bug #4070] (grass) r.in.srtm fails to assign null() to nodata properly - propably a GDAL bug
In-Reply-To <20060208105509.2537B1006C1@lists.intevation.de>
MIME-Version 1.0
Content-Type text/plain; charset=ISO-8859-1
Content-Transfer-Encoding quoted-printable
Content-Disposition inline
References <20060208105509.2537B1006C1@lists.intevation.de>
X-Spam-Status No, hits=-4.9 tagged_above=-999.0 required=3.0 tests=BAYES_00
X-Spam-Level
On 2/8/06, guest user via RT <grass-bugs@intevation.de> wrote:
> Frank,
>
> Please take a look at it. This might be a bug in GDAL handling of .bil ra=
sters.
...
> CREATING SUPPORT FILES FOR N51E016_v1
> SETTING GREY COLOR TABLE FOR N51E016_v1 (16bit, image range)
> r.in.gdal complete.
> WARNING: Your color rules do not cover the whole range of data!

Maciek,

I didn't get the message when I tried r.in.srtm.  I did make
some changes to signed/unsigned handling in the EHDR
driver which I think is being used in this case.  At least
one of the changes I had to change back and I am wondering
if you got an interim version.  Could you retry with a GDAL from
CVS (nightly)?

If that still causes the problem, please send me the .hdr file
being generated for you.

Best regards,
--
---------------------------------------+-----------------------------------=
---
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam@pobox.c=
om
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Programmer for Rent


Thu, Feb 9 2006 11:25:43    Status changed to resolved by msieczka  
Thu, Feb 9 2006 11:25:43    Mail sent by msieczka  
Maciek wrote:

>> CREATING SUPPORT FILES FOR N51E016_v1
>> SETTING GREY COLOR TABLE FOR N51E016_v1 (16bit, image range)
>> r.in.gdal complete.
>> WARNING: Your color rules do not cover the whole range of data!

Frank wrote:

> I didn't get the message when I tried r.in.srtm.  I did make
> some changes to signed/unsigned handling in the EHDR
> driver which I think is being used in this case.  At least
> one of the changes I had to change back and I am wondering
> if you got an interim version.  Could you retry with a GDAL from
> CVS (nightly)?

All fine now. Using today's gdal CVS I get correct min/max for the .bil file
created by r.in.srtm:

$ gdalinfo -mm ~/tmp/srtm_v1/N51E016.bil
Driver: EHdr/ESRI .hdr Labelled

<snip>

computed Min/Max=55.000,483.000
  NoData Value=-32768



So r.in.srtm works fine again.

The GDAL version which was making r.in.srtm fail like described in the report,
was CVS 2006-01-09.

Thanks Frank.

Closing it.

Maciek
Comment | Reply | Take | Open

You are currently authenticated as guest.
[Show Configuration] [Login as another user]

Users Guide - Mail Commands - Homepage of RequestTracker 1.0.7 - list any request