Wed, May 2 2001
13:27:02
|
|
Request created by mneteler
|
|
Subject: r.proj: reads the entire raster into memory
r.proj
It needs a small rewrite to not read the entire raster into
memory. It is useless with large rasters and will seriously bog down your
machine. I haven't had a chance to really grok it, but I think "rowio"
would be appropriate -- although, even that wouldn't be required. I think
at most it needs three rows in memory for cubic interpolation.
-> comment from Morten Hulden: This can only happen with *large* maps.
|
|
Wed, May 2 2001
14:52:52
|
|
Comments added by guest
|
|
A three row cache from the input image will not normally be adequate. In
a reprojection case with a non-trivial rotational effect generating one
row of output data will touch many (dozens or hundreds) rows of the input
image.
A traditional approach to this sort of problem is to maintain a cache of
source file scanlines. The size of this cache should default to something
like 24MB but be controllable by a commandline parameter.
This approach will do well for small images and even for very large images
as long as the rotational effect of the rotation is minimal.
A more sophisticated approach I have seen employed in a previous life was
to watch for cache thrashing, and if it occurs to try and generate the output
in strips, successively reducing the size of output vertical strips generated
until thrashing stops or some minimum is reached. Generating in strips
results in multiple passes over the source and destination image, but event
in the worst case of a 90 degree rotation the number of passes over the
source image can be contrained to roughly the number of strips. Without
this approach a 90 degree image will generally require one pass over the
source image for every output scanline!
Note that the problems is r.proj are presumably shared by the rectify code
though I haven't looked there. I would strongly encourage the generalization
of image warping into a subroutine taking an arbitrary transformation
function. The caching and resampling can be implemented once in this
subroutine and then it can be used for r.proj, the rectify code and
potentially other places.
I might possibly be willing to take this task on, but it might be a long
time till I get around to doing it.
|
|
Wed, May 2 2001
14:54:00
|
|
Comments added by fwarmerdam
|
|
Sorry ... that last comment was from me, Frank Warmerdam. I guess
I wasn't logged in properly.
|
|
Fri, Nov 9 2001
13:13:27
|
|
Owner changed to fwarmerdam by alange
|
|
Fri, Nov 9 2001
13:14:18
|
|
Comments added by alange
|
|
Hi,
i assigned this wish to Frank Warmerdam, as he suggested a solution.
Andreas |
|
Fri, Nov 9 2001
15:08:08
|
|
Comments added by fwarmerdam
|
|
I will accept this as a GRASS 5.1 action item. I may address this as part
of a generic image warping capability for GRASS and other packages.
Don't expect any action for 4-8 months.
|
|
Fri, Jan 4 2002
18:02:57
|
|
User changed to neteler@itc.it by mneteler
|
|
Tue, May 14 2002
12:45:52
|
|
Mail sent by guest
|
|
To take up this old story again: I rewrote r.proj about 18 months ago, among
other things to make it 'region sensitive'. So the statement 'reads the
entire raster into memory' is not exactly true. Only the part of the input
map that is overlapping the current region will be read into memory (save
three additional rows/columns on each side to allow for cubic interpolation).
Nevertheless, if the current region is large and the machine is low on
memory, there may be problems allocating the needed amount of memory. This
can be solved 'manually' by defining smaller regions in the output location
and reprojecting into them separately, and then r.patching the resulting
rasters.
I think the best way to procede would be to integrate this 'manual'
workaround into r.proj itself, making defining smaller regions and r.patching
transparent to the user. The smaller regions would only exist temporarily
while the program runs and be deleted after r.patching. All this should be a
conditional function that only triggers if memory allocation for the whole
overlapping region fails.
Morten Hulden
PS. Please note that trying to get around memory restictions the reverse way,
i.e. by splitting the _input_ map into smaller parts, may not give
satisfactory results: Cells representing areas near the edges of the split
input maps may have NULL holes or wrong values because interpolation methods
cannot work reliably on the edge-cells of the input maps (missing neighbors).
|
|
Sat, Jan 21 2006
15:20:49
|
|
Area changed to wish6 by msieczka
|
|
Tue, Sep 26 2006
18:32:12
|
|
Comments added by guest
|
|
Good Luck! http://xoomer.alice.it/pik0/poker-rooms/
|
|
Tue, Sep 26 2006
22:37:31
|
|
Comments added by guest
|
|
Cool design http://xoomer.alice.it/pik0/razz-poker/
|
|
Wed, Sep 27 2006
01:22:11
|
|
Comments added by guest
|
|
Great work on website. <a href="http://xoomer.alice.it/pik0/rules-of-poker/">rules
of poker</a> [url=http://xoomer.alice.it/pik0/rules-of-poker/]rules of poker[/url]
http://xoomer.alice.it/pik0/rules-of-poker/ |
|
Fri, Dec 8 2006
04:08:07
|
|
Comments added by hbowman
|
|
search note: this bug contains SPAM !
fwd from -dev list:
From: Glynn Clements
Subject: Re: [GRASS-user] problems using r.proj with large data set
Date: Fri, 8 Dec 2006 02:09:28 +0000
To: Maciej Sieczka
[CC'd to grass-dev]
Maciej Sieczka wrote:
> > Is this a problem with large files that I will just have to work around or
> > is it something to do with my setup?
>
> Propably the same, very old issue:
> http://intevation.de/rt/webrt?serial_num=241
I looked into this a while ago. Unfortunately, you can't use rowio (or
a home-grown equivalent), as libgis doesn't allow the projection to be
changed while maps are open. So, you have to read the entire input
map, close it, change the projection, then write the output map.
To get around the memory issues, you would first need to copy the
relevant portion of the input map to a temporary file, then use a
cache backed by that file.
The segment library would do the job, although it could add a
significant performance overhead.
--
Glynn Clements <glynn gclements.plus com>
_______________________________________________
grass-dev mailing list
grass-dev grass.itc it
http://grass.itc.it/mailman/listinfo/grass-dev
|
|
Mon, Dec 18 2006
12:05:27
|
|
Mail sent by mneteler
|
|
Glynn has submitted r.proj.seg to CVS-HEAD. |
|
Mon, Feb 5 2007
21:43:23
|
|
Mail sent by mneteler
|
|
AFAIK r.proj.seg still needs to replace r.proj in CVS.
This would close the historic bug report
https://intevation.de/rt/webrt?serial_num=241
Markus |
|
Mon, Feb 5 2007
21:44:50
|
|
Comments added by mneteler
|
|
Cc: glynn@gclements.plus.com
Glynn,
may I ask you to resolve the r.proj.seg issue?
https://intevation.de/rt/webrt?serial_num=241
thanks,
Markus
PS: Don't manage to change the bug owner... |
|
Tue, Feb 6 2007
08:19:41
|
|
Mail sent by glynn@gclements.plus.com
|
|
Return-Path |
<glynn@gclements.plus.com>
|
Delivered-To |
grass-bugs@lists.intevation.de
|
From |
Glynn Clements <glynn@gclements.plus.com>
|
MIME-Version |
1.0
|
Content-Type |
text/plain; charset=us-ascii
|
Content-Transfer-Encoding |
7bit
|
Message-ID |
<17864.11274.112050.496065@cerise.gclements.plus.com>
|
Date |
Tue, 6 Feb 2007 07:19:38 +0000
|
To |
Markus Neteler via RT <grass-bugs@intevation.de>
|
Cc |
|
Subject |
Re: [bug #241] (grass) r.proj: reads the entire raster into memory
|
In-Reply-To |
<20070205204450.3E7721006D0@lists.intevation.de>
|
References |
<20070205204450.3E7721006D0@lists.intevation.de>
|
X-Mailer |
VM 7.07 under 21.4 (patch 20) "Double Solitaire" XEmacs Lucid
|
X-Virus-Scanned |
by amavisd-new at intevation.de
|
X-Spam-Status |
No, hits=-5 tagged_above=-999 required=3 tests=[BAYES_00=-5]
|
X-Spam-Level |
|
Markus Neteler via RT wrote:
> this bug's URL: http://intevation.de/rt/webrt?serial_num=241
>
> Request number 241 was commented on by 'mneteler' (Markus Neteler).
> Responding to this message will send mail to the requestor.
>
> Request Tracker
> rt@intevation.de
>
> --------------------------------------------------------------
> Cc: glynn@gclements.plus.com
>
> Glynn,
>
>
>
> may I ask you to resolve the r.proj.seg issue?
>
> https://intevation.de/rt/webrt?serial_num=241
What issue?
Do you mean that you want it to install as r.proj, in place of the
existing version?
--
Glynn Clements <glynn@gclements.plus.com>
|
|
Tue, Feb 6 2007
09:54:12
|
|
Mail sent by mneteler
|
|
AFAIK there is the intention to replace the
old r.proj code with r.proj.seg while keeping the name.
Is that wrong? If yes, what's the scope of having both
GRASS 6.3.cvs (nc_spm):~/grass63/raster > grep r.proj Makefile
r.proj \
r.proj.seg \
? For me that's confusing.
Markus |
|