Details Ticket 2904


Comment | Reply | Take | Resolve


Serial Number 2904
Subject v.in.sites for larger site files does not finish
Area grass6
Queue grass
Requestors hmitaso@unity.ncsu.edu
Owner none
Status open
Last User Contact Mon Jan 17 09:05:57 2005 (4 yr ago)
Current Priority 30
Final Priority 70
Due No date assigned
Last Action Wed Sep 27 14:54:49 2006 (2 yr ago)
Created Wed Jan 12 06:07:15 2005 (4 yr ago)

Transaction History Ticket 2904


Wed, Jan 12 2005 06:07:15    Request created by guest  
Subject: v.in.sites for larger site files does not finish

Platform: GNU/Linux/i386
grass obtained from: CVS
grass binary for platform: Compiled from Sources
GRASS Version: 5.7 CVS 20040111

v.in.sites has a memory problem for data sets that include 500000 points or more
and have more than x,y. The problem is similar to the one reported for v.in.ascii.
For 280000 points it does the job, although it takes over 30 minutes,
but it is possible to see where it gets stuck:

v.in.sites atmlid99.iee out=atmlid99iee
Input format: dimension: 2   strings: 0   FP:1
Transfering sites to vect file
########################### it gets stuck here
282294 sites read
########################### and here
Building topology ...
282294 primitives registered
Building areas:  100%
0 areas built
0 isles built
Attaching islands: Attaching centroids:  100%
Topology was built.
Number of nodes     :   281449
Number of primitives:   282294
Number of points    :   282294
Number of lines     :   0
Number of boundaries:   0
Number of centroids :   0
Number of areas     :   0
Number of isles     :   0
Vector file complete
Sat, Jan 15 2005 18:40:11    Comments added by guest  
Cc: grass5@grass.itc.it

I tried to track down the problem - here is more detailed info, but no
solution yet:

The transfering of sites to vector file seems to slow down as the number
of points increases (maybe db_append_string has to go through everything
that has been written each time it adds a new point?)
but the real problem seems to be in db_close_database(driver) -
the program spends by far the most time on that function,
for illustration, 280000 points takes 3 minutes doing the transfering sites
to vect file but it spends 8 minutes on db_close_database and it extremely
slows down the machine (windows, mouse practically don't work on my computer
during those 8 minutes).  Everything else runs for a negligible time.
Slower importing of 280000 points is not really a problem, but
for larger files (500000+) I was unable to get it past
db_close_database(driver) after running it for hours.
So the bug apperas to be somewhere in lib/db/dbmi_driver/d_closedb.c or
d_close_cur.c, but I am only guessing here.
Mon, Jan 17 2005 09:05:57    Mail sent by blazek@itc.it  
Return-Path <blazek@itc.it>
Delivered-To grass-bugs@lists.intevation.de
Message-ID <41EB71D9.8080101@itc.it>
Date Mon, 17 Jan 2005 09:05:45 +0100
From Radim Blazek <blazek@itc.it>
User-Agent Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6) Gecko/20040113
X-Accept-Language en-us, en
MIME-Version 1.0
To guest user via RT <grass-bugs@intevation.de>
Cc grass5@grass.itc.it, Helena <hmitaso@unity.ncsu.edu>
Subject Re: [GRASS5] [bug #2904] (grass) v.in.sites for larger site files does not finish
References <20050115174011.34A30102BE2@lists.intevation.de>
In-Reply-To <20050115174011.34A30102BE2@lists.intevation.de>
Content-Type text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding 7bit
X-Spam-Status No, hits=-4.9 tagged_above=-999.0 required=3.0 tests=BAYES_00
X-Spam-Level
It will be in the driver implementation of db__driver_close_database(),
either in save_table() or free_table() in grass51/db/drivers/dbf/db.c.

Can you please try to comment the free_table (i); ( row 110 )
in the grass51/db/drivers/dbf/db.c.

In theory, one driver can be reused for another database, but AKAIK
it is not in current GRASS. So it is faster just close the driver.
OTOH, it can become a problem if DBMI swith to dlopen because of windows 
port.

Radim


guest user via RT wrote:
> this bug's URL: http://intevation.de/rt/webrt?serial_num=2904
> 
> Request number 2904 was commented on by 'guest' (guest user). 
> Responding to this message will send mail to the requestor.
> 			
> 			Request Tracker
> 			rt@intevation.de
> 
> --------------------------------------------------------------
> Cc: grass5@grass.itc.it
> 
> I tried to track down the problem - here is more detailed info, but no
> solution yet:
> 
> The transfering of sites to vector file seems to slow down as the number
> of points increases (maybe db_append_string has to go through everything
> that has been written each time it adds a new point?)
> but the real problem seems to be in db_close_database(driver) -
> the program spends by far the most time on that function,
> for illustration, 280000 points takes 3 minutes doing the transfering sites
> to vect file but it spends 8 minutes on db_close_database and it extremely
> slows down the machine (windows, mouse practically don't work on my computer
> during those 8 minutes).  Everything else runs for a negligible time.
> Slower importing of 280000 points is not really a problem, but
> for larger files (500000+) I was unable to get it past
> db_close_database(driver) after running it for hours.
> So the bug apperas to be somewhere in lib/db/dbmi_driver/d_closedb.c or
> d_close_cur.c, but I am only guessing here.
> 
> 
> 
> -------------------------------------------- Managed by Request Tracker
> 
> _______________________________________________
> grass5 mailing list
> grass5@grass.itc.it
> http://grass.itc.it/mailman/listinfo/grass5


Fri, Sep 2 2005 18:37:44    Area changed to grass6 by mneteler  
Mon, Sep 18 2006 16:33:20    Comments added by guest  
http://www.forumage.com/?mforum=designerreplica <a href=http://www.forumage.com/?mforum=designerrepl
ica>designer
replica handbags</a> designer replica handbags
Mon, Sep 18 2006 16:37:54    Comments added by guest  
http://www.forumage.com/?mforum=designerreplica <a href=http://www.forumage.com/?mforum=designerrepl
ica>designer
replica handbags</a> designer replica handbags
Wed, Sep 20 2006 08:23:26    Comments added by guest  
http://www.forumage.com/?mforum=hobohandbags <a href="http://www.forumage.com/?mforum=hobohandbags">
hobo
handbags</a> hobo handbags
Thu, Sep 21 2006 02:49:30    Comments added by guest  
http://www.forumage.com/?mforum=salvationbracel <a href="http://www.forumage.com/?mforum=salvationbr
acel">salvation
bracelets</a> salvation bracelets
Thu, Sep 21 2006 03:05:06    Comments added by guest  
http://www.forumage.com/?mforum=salvationbracel <a href="http://www.forumage.com/?mforum=salvationbr
acel">salvation
bracelets</a> salvation bracelets
Thu, Sep 21 2006 15:37:21    Comments added by guest  
http://www.bloggen.be/rosarybracelets <a href="http://www.bloggen.be/rosarybracelets">rosary
bracelets</a> rosary bracelets
Fri, Sep 22 2006 15:40:01    Comments added by guest  
http://xoomer.alice.it/gpokil <a href="http://xoomer.alice.it/gpokil">Free nokia
ringtones</a> Free nokia ringtones
Fri, Sep 22 2006 21:14:21    Comments added by guest  
http://www.bloggen.be/zend/ <a href="http://www.bloggen.be/zend/">replica watches</a>
replica watches
Fri, Sep 22 2006 23:20:44    Comments added by guest  
http://rolexreplica.awardspace.com/ <a href="http://rolexreplica.awardspace.com/">Rolex
replica</a> rolex replica
Sat, Sep 23 2006 01:53:38    Comments added by guest  
Hello, nice site look this:
<a href="http://lokimno.info/free-ringtones-for-cingular.html">free ringtones
for cingular</a>
[URL="http://lokimno.info/verizon-ringtone.html"]verizon ringtone[/URL]
[URL="http://lokimno.info/family-guy-ringtones.html"]family guy ringtones[/URL]
http://lokimno.info/chicken-ringtone.html
http://lokimno.info/free-nokia-3390-ringtones.html

End ^) See you
Sat, Sep 23 2006 13:36:47    Comments added by guest  
http://www.forumage.com/index.php?mforum=chimichangareci <a href="http://www.forumage.com/index.php?
mforum=chimichangareci">chimichanga
recipe</a> chimichanga recipe
Sat, Sep 23 2006 14:39:22    Comments added by guest  
http://www.forumage.com/?mforum=buyphentermine <a href="http://www.forumage.com/?mforum=buyphentermi
ne">buy
phentermine</a> buy phentermine
Sun, Sep 24 2006 15:29:36    Comments added by guest  
Great site, look this:
http://porinol.info/ringtones-for-us-cellular-phones.php
http://porinol.info/no-internet-required-ringtones.php
<a href="http://porinol.info/simpsons-ringtone-simpsons-ringtone.php">simpsons
ringtone simpsons ringtone</a>
<a href="http://porinol.info/free-ringtone.php">free ringtone</a>
<a href="http://porinol.info/christian-ringtones.php">christian ringtones</a>
http://porinol.info/adult-ringtones.php
http://porinol.info/motorola-ringtones-free.php
<a href="http://porinol.info/south-park-ringtones.php">south park ringtones</a>
<a href="http://porinol.info/download-free-ringtones-to-cellphone.php">download
free ringtones to cellphone</a>
http://porinol.info/free-ringtone.php
Mon, Sep 25 2006 14:01:29    Comments added by guest  
Hello, nice site look this:
<a href="http://sato.awardspace.com/replica-versace-sun-glasses.htm">replica
versace sun glasses</a>
http://sato.awardspace.com/wholesale-replica-sun-glasses.htm
http://sato.awardspace.com/replica-christian-dior-sun-glasses.htm
http://sato.awardspace.com/wholesale-replica-sun-glasses.htm
http://sato.awardspace.com/replica-versace-sun-glasses.htm
Mon, Sep 25 2006 22:02:10    Comments added by guest  
Great site, look this:
<a href=«http://sendinok.info/magnetic-bracelet.html">magnetic bracelet</a>
[URL="http://sendinok.info/diva-bracelet.html"]diva bracelet[/URL]
[URL="http://sendinok.info/breast-cancer-awareness-bracelet.html"]breast cancer
awareness bracelet[/URL]
[URL="http://sendinok.info/dog-charm-bracelet.html"]dog charm bracelet[/URL]
http://sendinok.info/rosary-bracelets.html
[URL="http://sendinok.info/custom-rubber-bracelets.html"]custom rubber bracelets[/URL]
http://sendinok.info/charm-bracelet-watch.html
http://sendinok.info/sterling-silver-charm-bracelet.html
<a href=«http://sendinok.info/charm-bracelet-links.html">charm bracelet links</a>
[URL="http://sendinok.info/stainless-steel-bracelets.html"]stainless steel bracelets[/URL]
Wed, Sep 27 2006 14:54:49    Comments added by guest  
Great site, look this:
<a href="http://domkifo.info/cheap-replica-rolex.php">cheap replica rolex</a>
[URL="http://domkifo.info/rolex-used-watches.php"]rolex used watches[/URL]
[URL="http://domkifo.info/a-e-rolex-watches.php"]a e rolex watches[/URL]
[URL="http://domkifo.info/rolex-yachtmaster.php"]rolex yachtmaster[/URL]
http://domkifo.info/rolex-watch-reviews.php
http://domkifo.info/rolex-kentucky.php
<a href="http://domkifo.info/rolex-1803-wg.php">rolex 1803 wg</a>
http://domkifo.info/rolex-vintage.php
[URL="http://domkifo.info/japanese-replica-rolex.php"]japanese replica rolex[/URL]
<a href="http://domkifo.info/would-jesus-wear-a-rolex.php">would jesus wear a
rolex</a>
Comment | Reply | Take | Resolve

You are currently authenticated as guest.
[Show Configuration] [Login as another user]

Users Guide - Mail Commands - Homepage of RequestTracker 1.0.7 - list any request