Details Ticket 4250


Comment | Reply | Resolve


Serial Number 4250
Subject v.out.ogr: CHAR columns are trimmed to 80 chars when writing shp
Area grass6
Queue grass
Requestors tutey@o2.pl
Owner fwarmerdam
Status open
Last User Contact Tue Jul 4 14:24:17 2006 (2 yr ago)
Current Priority 70
Final Priority 70
Due No date assigned
Last Action Wed Jul 26 18:26:30 2006 (2 yr ago)
Created Thu Apr 6 11:47:00 2006 (2 yr ago)

Transaction History Ticket 4250


Thu, Apr 6 2006 11:47:00    Request created by guest  
Subject: v.out.ogr: CHAR columns are trimmed to 80 chars when writing shp

Platform: GNU/Linux/x86
grass obtained from: CVS
grass binary for platform: Compiled from Sources
GRASS Version: 2006-04-05

I have a vector with a table containing CHAR column long for 249 chars, using
DBF driver. Within Grass it works fine. After exporting to a shapefile with v.out.ogr,
the 249 char long column is trimmed to 80 characters only.

Maciek
Fri, Apr 7 2006 03:44:24    Mail sent by hamish_nospam@yahoo.com  
Return-Path <hamish_nospam@yahoo.com>
Delivered-To grass-bugs@lists.intevation.de
Date Fri, 7 Apr 2006 13:44:09 +1200
From Hamish <hamish_nospam@yahoo.com>
To Request Tracker <grass-bugs@intevation.de>
Cc grass5@grass.itc.it
Subject Re: [GRASS5] [bug #4250] (grass) v.out.ogr: CHAR columns are trimmed to 80 chars when writing shp
Message-Id <20060407134409.5c5ed9d9.hamish_nospam@yahoo.com>
In-Reply-To <20060406094700.527C91006B5@lists.intevation.de>
References <20060406094700.527C91006B5@lists.intevation.de>
X-Mailer Sylpheed version 1.0.4 (GTK+ 1.2.10; i386-pc-linux-gnu)
X-Face M<EoB)"*Z~u!,vFhXmw}R_KbdBta*P_=T|rbBL'e1/CQ9;/1g\BU3&!=y8ria$2Uk!HT&BB 8i?|X_+7~1jsy}F~g$2va%3fV`*=L(*cem[@3\yg,G,@rg6/QMJ
Mime-Version 1.0
Content-Type text/plain; charset=US-ASCII
Content-Transfer-Encoding 7bit
X-Spam-Status No, hits=-4.0 tagged_above=-999.0 required=3.0 tests=BAYES_00, FORGED_YAHOO_RCVD
X-Spam-Level
> this bug's URL: http://intevation.de/rt/webrt?serial_num=4250
> ---------------------------------------------------------------------
> 
> Subject: v.out.ogr: CHAR columns are trimmed to 80 chars when writing
> shp
.. 
> I have a vector with a table containing CHAR column long for 249
> chars, using DBF driver. Within Grass it works fine. After exporting
> to a shapefile with v.out.ogr, the 249 char long column is trimmed to
> 80 characters only.


what does 'dbview -e' say about the dbf coulumns before and after export?

what if you just copy the $MAPSET/dbf/ file to be with your .shp file?


Hamish


Fri, Apr 7 2006 10:24:52    Mail sent by msieczka  
Hamish wrote:

> what does 'dbview -e' say about the dbf coulumns before and after export?
### the dbf BEFORE export

$ dbview -e /home/grassdata/caves_utm33/wlasnosc2/dbf/test.dbf

Field Name      Type    Length  Decimal Pos
Cat               N        11       0
Parcel no         C        80       0
Maps use          C        80       0
Maps class        C        80       0
Maps cover        C        20       0
Owner cod         N        11       0
Owner nam         C       249       0
Owner nmb         N        11       0
Owner loc         C        80       0
Tenant cod        N        11       0
Tenant nam        C        80       0
Tenant loc        C        80       0
Host cod          N        11       0
Area              N        20       6
Lndrcl lev        N        11       0
Lndrcl clt        N        11       0



### the dbf AFTER export

$ dbview -e /home/john/test/test.dbf

Field Name      Type    Length  Decimal Pos
Cat               N        11       0
Parcel no         C        80       0
Maps use          C        80       0
Maps class        C        80       0
Maps cover        C        80       0
Owner cod         N        11       0
Owner nam         C        80       0
Owner nmb         N        11       0
Owner loc         C        80       0
Tenant cod        N        11       0
Tenant nam        C        80       0
Tenant loc        C        80       0
Host cod          N        11       0
Area              N        24      15
Lndrcl lev        N        11       0
Lndrcl clt        N        11       0

### Interesting is that the "Area" attribute properties changed too, isn't
### that wrong?

> what if you just copy the $MAPSET/dbf/ file to be with your .shp file?

If I do that, the relationship between objects in the shapefile and dbf file
is lost. Querries in QGIS prove that. Apparently the dbf is somewhat processed
by v.out.ogr when writing a shapefile.

Maciek
Fri, Apr 7 2006 10:27:23    Mail sent by msieczka  
Oh, and "Maps cover" lenght changed too - was 20 originaly, after export it's
80 (see above).

Maciek
Fri, Apr 7 2006 16:33:29    Mail sent by kyngchaos@kyngchaos.com  
Return-Path <woklist@kyngchaos.com>
Delivered-To grass-bugs@lists.intevation.de
In-Reply-To <20060407082452.CE99A101FCC@lists.intevation.de>
References <20060407082452.CE99A101FCC@lists.intevation.de>
Mime-Version 1.0 (Apple Message framework v749.3)
Content-Type text/plain; charset=US-ASCII; delsp=yes; format=flowed
Message-Id <B0F522E6-7E1C-4DD8-A651-F41641BDDD53@kyngchaos.com>
Cc grass5@grass.itc.it
Reply-To William Kyngesburye <kyngchaos@kyngchaos.com>
Content-Transfer-Encoding 7bit
From William Kyngesburye <woklist@kyngchaos.com>
Subject Re: [GRASS5] [bug #4250] (grass) v.out.ogr: CHAR columns are trimmed to 80 chars when writing shp
Date Fri, 7 Apr 2006 09:33:16 -0500
To Maciek Sieczka via RT <grass-bugs@intevation.de>
X-Mailer Apple Mail (2.749.3)
X-Spam-Status No, hits=-4.9 tagged_above=-999.0 required=3.0 tests=BAYES_00
X-Spam-Level
This is almost exactly the problem I had with ogr a while back (isn't  
fixed in 1.3.1 so I've patched my copy, maybe fixed in CVS).  This  
problem is for mysql and postgres tables - OGR doesn't translate  
varchar field types.  In the case when OGR can't figure out the field  
type, it defaults to a char type of length _80_.

I looked in the grass ogr driver source, and there's the same  
problem.  The grass driver doesn't set *any* field lengths at all, so  
it defaults to 80.

ogrgrasslayer.cpp, lines 148-161.

-----
William Kyngesburye <kyngchaos@kyngchaos.com>
http://www.kyngchaos.com/

All generalizations are dangerous, even this one.


Fri, Apr 7 2006 20:01:49    Mail sent by msieczka  
Thanks for your observations William!

But please all note that the problem is not limited to VARCHAR columns only.
DOUBLE columns attributes are also changed by v.out.ogr, which we don't want
to happen (see how "Area" column lenght and number of decimal places is affected).
I hope somebody will be able to fix these soon.

Maciek
Fri, Apr 7 2006 22:05:55    Mail sent by kyngchaos@kyngchaos.com  
Return-Path <woklist@kyngchaos.com>
Delivered-To grass-bugs@lists.intevation.de
In-Reply-To <20060407180149.776791006B4@lists.intevation.de>
References <20060407180149.776791006B4@lists.intevation.de>
Mime-Version 1.0 (Apple Message framework v749.3)
Content-Type text/plain; charset=US-ASCII; delsp=yes; format=flowed
Message-Id <30C8A291-A0AE-482E-BB9A-28A22DD3C3E0@kyngchaos.com>
Cc grass5@grass.itc.it
Reply-To William Kyngesburye <kyngchaos@kyngchaos.com>
Content-Transfer-Encoding 7bit
From William Kyngesburye <woklist@kyngchaos.com>
Subject Re: [bug #4250] (grass) v.out.ogr: CHAR columns are trimmed to 80 chars when writing shp
Date Fri, 7 Apr 2006 15:05:49 -0500
To Maciek Sieczka via RT <grass-bugs@intevation.de>
X-Mailer Apple Mail (2.749.3)
X-Spam-Status No, hits=-4.9 tagged_above=-999.0 required=3.0 tests=BAYES_00
X-Spam-Level
The varchar problem was the similar issue for mysql/pgsql.  DBF  
tables don't have such a thing, just fixed-length char.  But the way  
the grass gdal driver is written, it doesn't set the length for char  
columns, or any other columns, so number columns will default to  
whatever they default to in OGR (24.15 by the looks of your example).

Fixing it is a little beyond me.  I can do basic stuff by example  
(that's how I was able to fix the varchar problem in mysql/pgsql),  
but I'd have to dig around in the drivers to find an example to fix  
the grass dbf problem.

Maybe Radim can help.

-----
William Kyngesburye <kyngchaos@kyngchaos.com>
http://www.kyngchaos.com/

"History is an illusion caused by the passage of time, and time is an  
illusion caused by the passage of history."

- Hitchhiker's Guide to the Galaxy


Sat, Apr 8 2006 06:10:39    Mail sent by hamish_nospam@yahoo.com  
Return-Path <hamish_nospam@yahoo.com>
Delivered-To grass-bugs@lists.intevation.de
Date Sat, 8 Apr 2006 16:10:23 +1200
From Hamish <hamish_nospam@yahoo.com>
To Maciek Sieczka via RT <grass-bugs@intevation.de>
Cc grass5@grass.itc.it
Subject Re: [bug #4250] (grass) v.out.ogr: CHAR columns are trimmed to 80 chars when writing shp
Message-Id <20060408161023.16160083.hamish_nospam@yahoo.com>
In-Reply-To <20060407082452.CE99A101FCC@lists.intevation.de>
References <20060407082452.CE99A101FCC@lists.intevation.de>
X-Mailer Sylpheed version 1.0.4 (GTK+ 1.2.10; i386-pc-linux-gnu)
X-Face M<EoB)"*Z~u!,vFhXmw}R_KbdBta*P_=T|rbBL'e1/CQ9;/1g\BU3&!=y8ria$2Uk!HT&BB 8i?|X_+7~1jsy}F~g$2va%3fV`*=L(*cem[@3\yg,G,@rg6/QMJ
Mime-Version 1.0
Content-Type text/plain; charset=US-ASCII
Content-Transfer-Encoding 7bit
X-Spam-Status No, hits=-4.0 tagged_above=-999.0 required=3.0 tests=BAYES_00, FORGED_YAHOO_RCVD
X-Spam-Level
> $ dbview -e /home/grassdata/caves_utm33/wlasnosc2/dbf/test.dbf
> 
> Field Name      Type    Length  Decimal Pos
> Cat               N        11       0
...
> Owner cod         N        11       0
> Owner nam         C       249       0
> Owner nmb         N        11       0
> Owner loc         C        80       0


are those field names "Owner nam" or "Owner_nam"? I didn't think dbf
fields could have spaces; if two words maybe it is just reading the
first and getting confused as there are multiple "Owner"s? Or is dbview
just translating the names?

did you try sqlite instead of dbf? (doubt any improvement, but maybe
worth a try)



Hamish


Sat, Apr 8 2006 09:08:49    Mail sent by werchowyna@epf.pl  
Return-Path <werchowyna@epf.pl>
Delivered-To grass-bugs@lists.intevation.de
Date Sat, 8 Apr 2006 09:08:35 +0200
From Maciek Sieczka <werchowyna@epf.pl>
To Hamish <hamish_nospam@yahoo.com>
Cc grass-bugs@intevation.de, grass5@grass.itc.it
Subject Re: [GRASS5] Re: [bug #4250] (grass) v.out.ogr: CHAR columns are trimmed to 80 chars when writing shp
Message-Id <20060408090835.337fe6e5.werchowyna@epf.pl>
In-Reply-To <20060408161023.16160083.hamish_nospam@yahoo.com>
References <20060407082452.CE99A101FCC@lists.intevation.de> <20060408161023.16160083.hamish_nospam@yahoo.com>
X-Mailer Sylpheed version 2.1.1 (GTK+ 2.8.6; i486-pc-linux-gnu)
Mime-Version 1.0
Content-Type text/plain; charset=US-ASCII
Content-Transfer-Encoding 7bit
X-Spam-Status No, hits=-4.9 tagged_above=-999.0 required=3.0 tests=BAYES_00
X-Spam-Level
On Sat, 8 Apr 2006 16:10:23 +1200
Hamish <hamish_nospam@yahoo.com> wrote:

> > $ dbview -e /home/grassdata/caves_utm33/wlasnosc2/dbf/test.dbf
> > 
> > Field Name      Type    Length  Decimal Pos
> > Cat               N        11       0
> ...
> > Owner cod         N        11       0
> > Owner nam         C       249       0
> > Owner nmb         N        11       0
> > Owner loc         C        80       0
> 
> 
> are those field names "Owner nam" or "Owner_nam"?

That's how dbview displays; in OOCalc2 they are "OWNER_COD", OWNER_NAM"
etc. respectively.

>  I didn't think dbf fields could have spaces; if two words maybe it
> is just reading the first and getting confused as there are multiple
> "Owner"s? Or is dbview just translating the names?

Columns name are OK, no doubt bout it.

> did you try sqlite instead of dbf?

No.

> (doubt any improvement, but maybe worth a try)

Then I'll take my time ;).

Maciek

--------------------
W polskim Internecie s? setki milion?w stron. My przekazujemy Tobie tylko najlepsze
z nich!
http://katalog.panoramainternetu.pl/


Tue, Jul 4 2006 14:24:17    Owner changed to fwarmerdam by mneteler  
Tue, Jul 4 2006 14:24:17    Mail sent by mneteler  
Frank,

[issue that the column length is not communicated to OGR
in the GRASS-OGR plugin]

I had a look at
 frmts/grass/pkg/ogrgrasslayer.cpp
line 146-165

In fact, there is no definition of the column
length.

Should it be set with OGRFieldDefn()? What's the
C equivalent to it?

GRASS offers db_set_column_length(column, fsize);
which should deliver the column length. Not sure how
to add that in the plugin.

Best regards

 Markus
Wed, Jul 26 2006 18:26:30    User changed to tutey@o2.pl by msieczka  
Comment | Reply | Resolve

You are currently authenticated as guest.
[Show Configuration] [Login as another user]

Users Guide - Mail Commands - Homepage of RequestTracker 1.0.7 - list any request