Fri, Mar 10 2006
16:13:57
|
|
Request created by guest
|
|
Subject: v.clean: uses 1GB RAM, 1 GB swap for a vector with only 628 boundaries
Platform: GNU/Linux/x86
grass obtained from: CVS
grass binary for platform: Compiled from Sources
GRASS Version: 2006-02-20
v.buffer used all my memory (1 GB RAM and 1GB swap) when building a clean buffer
for my vector file, and I had to kill it to continue my work.
For a test, I forced v.buffer to skip cleaning, using option debug=buffer, and
tried to clean this "dirty" output myself:
v.clean input=rogow_parcels_06_water_buff100 output=rogow_parcels_06_water_buff100_cl
type=boundary tool=break
Again, it ate all my memory. The rogow_parcels_06_water_buff100 is very small:
Number of boundaries: 628
v.clean reaches the memory limit at about:
Intersections: 78194 (line 150883)
There must be some problem with memory handling in the vector code.
I'm putting the location with (only) my problematic vector file, if somebody
is interested in fixing the problem.
http://www.biol.uni.wroc.pl/sieczka/udostepnione/grass/huha.tar.bz2 (364 KB)
I'm not looking for a workaround - I accomplished my task using r.buffer, r.to.vect.
Maciek
|
|
Fri, Mar 10 2006
17:14:49
|
|
Mail sent by hmitaso@unity.ncsu.edu
|
|
Return-Path |
<hmitaso@unity.ncsu.edu>
|
Delivered-To |
grass-bugs@lists.intevation.de
|
Message-ID |
<4411A5EF.5080407@unity.ncsu.edu>
|
Date |
Fri, 10 Mar 2006 11:14:39 -0500
|
From |
Helena Mitasova <hmitaso@unity.ncsu.edu>
|
User-Agent |
Mozilla Thunderbird 1.0.2 (X11/20050317)
|
X-Accept-Language |
en-us, en
|
MIME-Version |
1.0
|
To |
Request Tracker <grass-bugs@intevation.de>
|
Cc |
grass5@grass.itc.it
|
Subject |
Re: [GRASS5] [bug #4164] (grass) v.clean: uses 1GB RAM, 1 GB swap for a vector with only 628 boundaries
|
References |
<20060310151357.6AF801006C9@lists.intevation.de>
|
In-Reply-To |
<20060310151357.6AF801006C9@lists.intevation.de>
|
Content-Type |
text/plain; charset=ISO-8859-1; format=flowed
|
Content-Transfer-Encoding |
7bit
|
X-PMX-Version |
4.7.1.128075, Antispam-Engine: 2.1.0.0, Antispam-Data: 2006.03.10.065104
|
X-Spam-Status |
No, hits=-4.9 tagged_above=-999.0 required=3.0 tests=BAYES_00
|
X-Spam-Level |
|
Request Tracker wrote:
> this bug's URL: http://intevation.de/rt/webrt?serial_num=4164
> -------------------------------------------------------------------------
>
> Subject: v.clean: uses 1GB RAM, 1 GB swap for a vector with only 628 boundaries
>
> Platform: GNU/Linux/x86
> grass obtained from: CVS
> grass binary for platform: Compiled from Sources
> GRASS Version: 2006-02-20
>
> v.buffer used all my memory (1 GB RAM and 1GB swap) when building a clean buffer
for my vector file, and I had to kill it to continue my work.
>
> For a test, I forced v.buffer to skip cleaning, using option debug=buffer,
and tried to clean this "dirty" output myself:
>
> v.clean input=rogow_parcels_06_water_buff100 output=rogow_parcels_06_water_buff100_cl
type=boundary tool=break
>
> Again, it ate all my memory. The rogow_parcels_06_water_buff100 is very small:
>
> Number of boundaries: 628
>
> v.clean reaches the memory limit at about:
>
> Intersections: 78194 (line 150883)
>
> There must be some problem with memory handling in the vector code.
>
> I'm putting the location with (only) my problematic vector file, if somebody
is interested in fixing the problem.
>
> http://www.biol.uni.wroc.pl/sieczka/udostepnione/grass/huha.tar.bz2 (364 KB)
>
> I'm not looking for a workaround - I accomplished my task using r.buffer, r.to.vect.
>
> Maciek
>
>
> -------------------------------------------- Managed by Request Tracker
>
> _______________________________________________
> grass5 mailing list
> grass5@grass.itc.it
> http://grass.itc.it/mailman/listinfo/grass5
Although this may not be related to your problem, v.buffer may be
calling V_build which has a memory management problem/feature that it
eats up all your memory. But it should not happen with 628 boundaries
- we have that problem with about 600,000+ points.
Maybe Radim can explain whether this may be the same. If you digg
through the emails related to v.in.ascii and handling of large files
there was some discussion between Hamish and Radim on how to solve it.
Helena
|
|
Thu, Mar 16 2006
17:48:31
|
|
Status changed to resolved by msieczka
|
|
Thu, Mar 16 2006
17:48:30
|
|
Mail sent by msieczka
|
|
Seems fixed in today's CVS (2006.03.16). Maybe the fix was:
http://grass.itc.it/pipermail/grass-commit/2006-March/021025.html
Nothing else related I can find - but on the other hand fix' author didn't
reffered to this bug. So I'm not sure what the cure was actually.
?
Anyway, closing it.
Maciek
|
|
Fri, Mar 17 2006
10:23:10
|
|
Mail sent by radim.blazek@gmail.com
|
|
Return-Path |
<radim.blazek@gmail.com>
|
Delivered-To |
grass-bugs@lists.intevation.de
|
DomainKey-Signature |
a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=e2y87Im3il0GEc/gzJ13lHP0Oi/bJO+kQjB5xOANZHHoCSqD3gQCPUr5q5+InpiP7Pg3MhdyzjU3bxNMbw5WyYhnrfJflne1J0nyRSeWBbx0rGhAwe1+hH2J2b5zxDLWSz5NDNOayXKz31wZlFK3ZSgX0Y8vYy+fwJ98Hal5cwc=
|
Message-ID |
<340505ef0603170119y4239e603t65ca8dbe099dcf9a@mail.gmail.com>
|
Date |
Fri, 17 Mar 2006 10:19:56 +0100
|
From |
"Radim Blazek" <radim.blazek@gmail.com>
|
To |
"Maciek Sieczka via RT" <grass-bugs@intevation.de>
|
Subject |
Re: [GRASS5] [bug #4164] (grass) v.clean: uses 1GB RAM, 1 GB swap for a vector with only 628 boundaries
|
Cc |
grass5@grass.itc.it, hmitaso@unity.ncsu.edu
|
In-Reply-To |
<20060316164831.30B3C1005A3@lists.intevation.de>
|
MIME-Version |
1.0
|
Content-Type |
text/plain; charset=ISO-8859-1
|
Content-Transfer-Encoding |
quoted-printable
|
Content-Disposition |
inline
|
References |
<20060316164831.30B3C1005A3@lists.intevation.de>
|
X-Spam-Status |
No, hits=-4.9 tagged_above=-999.0 required=3.0 tests=BAYES_00
|
X-Spam-Level |
|
On 3/16/06, Maciek Sieczka via RT <grass-bugs@intevation.de> wrote:
> Seems fixed in today's CVS (2006.03.16). Maybe the fix was:
> http://grass.itc.it/pipermail/grass-commit/2006-March/021025.html
>
> Nothing else related I can find - but on the other hand fix' author didn'=
t
> reffered to this bug. So I'm not sure what the cure was actually.
It seems to be strange. Did you get correct result from v.buffer???!!!
It maybe that I just introduced another bug.
Radim
> ?
>
> Anyway, closing it.
>
> Maciek
>
>
> -------------------------------------------- Managed by Request Tracker
>
> _______________________________________________
> grass5 mailing list
> grass5@grass.itc.it
> http://grass.itc.it/mailman/listinfo/grass5
>
|
|
Fri, Mar 17 2006
10:23:10
|
|
Status changed to open by _rt_system
|
|
Tue, Mar 21 2006
21:03:00
|
|
Mail sent by msieczka
|
|
Radim wrote:
> It seems to be strange. Did you get correct result from v.buffer???!!!
> It maybe that I just introduced another bug.
Radim,
Sorry for late reply. I've been away for some time.
Regarding the issue: indeed, v.buffer's output is not a clean buffer. I
reproduced my orignal v.buffer command:
v.buffer input=x_extr output=x_extr_buff100 type=boundary layer=1 buffer=100
scale=1.0 tolerance=0.01 debug=clean
and in spite specifying debug=clean the output is full of tiny polygons made
of crossing circles and arcs. What's good, their topology looks allright after
inspection in Qgis Grass digit. Even better, that v.buffer used very little
memory - several times less than before; which is great! It didn't even touch
swap!
Summarising: the memory consumption is sane now, yippie!, but buffer is not
created.
Do you need my dataset for your work on this?
Maciek |
|
Wed, Mar 22 2006
09:25:06
|
|
Mail sent by radim.blazek@gmail.com
|
|
Return-Path |
<radim.blazek@gmail.com>
|
Delivered-To |
grass-bugs@lists.intevation.de
|
DomainKey-Signature |
a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=AFS06mP8fzfmbofNlSeYcGnX8oVugvRdpb7qrdDhJed5kvLk8RJK1euUGQMdDra2IOsXn8UK6/9ObGTBlq/DoAWNNVX7d3j/yHQxECckT6VUupw1Ch0srtmU9HJnhvreJ77/4UUz7gqcLHqdjq6CftQLb5VIGMOcKaUDkyWdmqY=
|
Message-ID |
<340505ef0603220025i4baf8390t985fe1e1b3f8e2d@mail.gmail.com>
|
Date |
Wed, 22 Mar 2006 09:25:02 +0100
|
From |
"Radim Blazek" <radim.blazek@gmail.com>
|
To |
"Maciek Sieczka via RT" <grass-bugs@intevation.de>
|
Subject |
Re: [bug #4164] (grass) v.clean: uses 1GB RAM, 1 GB swap for a vector with only 628 boundaries
|
Cc |
grass5@grass.itc.it
|
In-Reply-To |
<20060321200300.15E66100160@lists.intevation.de>
|
MIME-Version |
1.0
|
Content-Type |
text/plain; charset=ISO-8859-1
|
Content-Transfer-Encoding |
quoted-printable
|
Content-Disposition |
inline
|
References |
<20060321200300.15E66100160@lists.intevation.de>
|
X-Spam-Status |
No, hits=-4.9 tagged_above=-999.0 required=3.0 tests=BAYES_00
|
X-Spam-Level |
|
On 3/21/06, Maciek Sieczka via RT <grass-bugs@intevation.de> wrote:
> Radim wrote:
> Sorry for late reply. I've been away for some time.
>
> Regarding the issue: indeed, v.buffer's output is not a clean buffer. I
> reproduced my orignal v.buffer command:
>
> v.buffer input=3Dx_extr output=3Dx_extr_buff100 type=3Dboundary layer=3D1=
buffer=3D100
> scale=3D1.0 tolerance=3D0.01 debug=3Dclean
>
> and in spite specifying debug=3Dclean the output is full of tiny polygons=
made
> of crossing circles and arcs. What's good, their topology looks allright =
after
> inspection in Qgis Grass digit.
Thats correct, debug means debug, if you want final output don't use debug.
Radim
> Even better, that v.buffer used very little
> memory - several times less than before; which is great! It didn't even t=
ouch
>
> swap!
>
>
>
> Summarising: the memory consumption is sane now, yippie!, but buffer is n=
ot
>
> created.
>
>
>
> Do you need my dataset for your work on this?
>
>
>
> Maciek
>
> -------------------------------------------- Managed by Request Tracker
>
|
|
Thu, Mar 23 2006
08:43:39
|
|
Mail sent by radim.blazek@gmail.com
|
|
Return-Path |
<radim.blazek@gmail.com>
|
Delivered-To |
grass-bugs@lists.intevation.de
|
DomainKey-Signature |
a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=YqvdgevCdKgHXTEv1oSqv6NvFefNAg6WU9rHE+qdlv6aWzhYc/X+ANvNNHdxhAibsZDqNnjdKotdxFiK0ccr4cV7CDMERWpwqvolieBW/dEvj8u24Zf2yFe8DaMRhBPg4Okw09PWkFPi0C78nOf61rRBbUpTG8CXPLa8WZybVaI=
|
Message-ID |
<340505ef0603222343t7426c080rca155e5422d3c19c@mail.gmail.com>
|
Date |
Thu, 23 Mar 2006 08:43:36 +0100
|
From |
"Radim Blazek" <radim.blazek@gmail.com>
|
To |
"Maciek Sieczka" <werchowyna@epf.pl>
|
Subject |
Re: [GRASS5] Re: [bug #4164] (grass) v.clean: uses 1GB RAM, 1 GB swap for a vector with only 628 boundaries
|
Cc |
grass-bugs@intevation.de, grass5@grass.itc.it
|
In-Reply-To |
<20060322224654.4b1d4f2b.werchowyna@epf.pl>
|
MIME-Version |
1.0
|
Content-Type |
text/plain; charset=ISO-8859-1
|
Content-Transfer-Encoding |
quoted-printable
|
Content-Disposition |
inline
|
References |
<20060321200300.15E66100160@lists.intevation.de> <340505ef0603220025i4baf8390t985fe1e1b3f8e2d@mail.gmail.com> <20060322224654.4b1d4f2b.werchowyna@epf.pl>
|
X-Spam-Status |
No, hits=-4.9 tagged_above=-999.0 required=3.0 tests=BAYES_00
|
X-Spam-Level |
|
On 3/22/06, Maciek Sieczka <werchowyna@epf.pl> wrote:
> Now, not using "debug=3Dclean": the v.buffer creates _mostly_ good result
> when compared to r.buffer at resolution =3D1. But there are at least 2
> places with errors visible in v.buffer behaviour. Please see the
> attached screendump, you will spot them easily.
>
> The YELLOW lines are the v.buffer and r.buffer input.
>
> The RED backgroung is r.buffer output (GOOD).
>
> The BLACK lines are v.buffer output, with at least 2 ERRORS visible,
> indicated by GREEN circles.
>
> Are those possible to be fixed Radim?
Not now by me. If possible publish your input data so that it can
be used for debugging in future.
How do look those 2 places if run with debug=3Dclean?
Radim
> Maciek
>
>
> --------------------
> W polskim Internecie s? setki milion?w stron. My przekazujemy Tobie tylko=
najlepsze z nich!
> http://katalog.panoramainternetu.pl/
>
>
>
|
|
Fri, Mar 24 2006
20:07:32
|
|
Mail sent by msieczka
|
|
On Thu, 23 Mar 2006 08:43:36 +0100
"Radim Blazek" <radim.blazek@gmail.com> wrote:
> On 3/22/06, Maciek Sieczka <werchowyna@epf.pl> wrote:
> > Now, not using "debug=clean": the v.buffer creates _mostly_ good
> > result when compared to r.buffer at resolution =1. But there are at
> > least 2 places with errors visible in v.buffer behaviour. Please
> > see the attached screendump, you will spot them easily.
> >
> > The YELLOW lines are the v.buffer and r.buffer input.
> >
> > The RED backgroung is r.buffer output (GOOD).
> >
> > The BLACK lines are v.buffer output, with at least 2 ERRORS visible,
> > indicated by GREEN circles.
> >
> > Are those possible to be fixed Radim?
> >
> Not now by me. If possible publish your input data so that it can
> be used for debugging in future.
Here (3,3 MB):
http://www.biol.uni.wroc.pl/sieczka/udostepnione/grass/huha2.tar.bz2
There is an input vector boundaries
x_extr
and the 100m buffer with a visible fault
x_extr_buff100
I deleted my original input and created it once more to prepare this
sample dataset - and somehow now there is only 1 error in the
v.buffer output instead of 2 as previously... Hmm. Maybe my fault.
> How do look those 2 places if run with debug=clean?
Topologicaly clean, but the fault in buffer is still clearly visible.
Maciek
|
|
Wed, Jul 26 2006
18:21:29
|
|
User changed to tutey@o2.pl by msieczka
|
|