Details Ticket 4426


Comment | Reply | Take | Open


Serial Number 4426
Subject Almost entire GRASS database deleted. Suspect grass startup.
Area grass6
Queue grass
Requestors cedricgrass@shockfamily.net
Owner none
Status resolved
Last User Contact Sat Nov 4 18:39:49 2006 (2 yr ago)
Current Priority 69
Final Priority 99
Due No date assigned
Last Action Sat Nov 4 18:39:49 2006 (2 yr ago)
Created Mon May 8 09:47:16 2006 (2 yr ago)

Transaction History Ticket 4426


Mon, May 8 2006 09:47:16    Request created by guest  
Subject: Almost entire GRASS database deleted. Suspect grass startup.

Platform: GNU/Linux/x86
grass obtained from: CVS
grass binary for platform: Compiled from Sources
GRASS Version: CVS development May 7th 2006

Virtually an entire mapset in a location was deleted somehow. These are all the
things I can remember doing since the last time I saw it alive:

Short version: PLEASE AUDIT STARTUP SCRIPTS, g.mapsets.tcl !!!

Most of this is probably not interesting.

Files were still being listed when I worked on development on gis.m testing running
v.digit started from a layer.

I made the commit: Digitizing a vector layer might actually work now. Todo: grab
other layers that are selected and use them for the draw commands.

I made a bunch of changes to mouse scrolling in gis.m.

I tested gis.m and opening guis for a bunch of modules and moved the cursor around
and used the mouse wheel and clicked on frames and things.

I made these commits:

* Mouse scroll the layer tree and options windows.

* Improvements to scrolling. Scrolling is much more specific (must have 
heyboard or mouse focus). Binding scrolling is no longer a code leak.

I saved a scrolling change I hadn't saved and tested yet, made some more tests
of gis.m just like above and made the commit: No more code or memmory leak. (Forgot
to save part of edit before 
testing and committing)

I started making edits to gronsole.tcl to fix scrolling.

I closed all of grass (I left)

I reopened grass by clicking on a launcher button that, in an xterm, runs:
bash -c "cd ~/grassdata; grass61"
I don't usually run grass this way. I think I normally open a terminal and run
grass61.

The new startup screen came up; I was mildly surprised (I havn't seen it very
many times yet).  I clicked on the button in the lower left corner (Location
and PERMANENT mapset were already selected).

I finished changes to gronsole.tcl (bindings on scrollings) and tested it. As
I had suspected It didn't really work.

I got an email from Michael, "V.digit doesn't work with  new maps??". I revisited
the changes of my earlier commit "Digitizing a vector...". I observed that if
I tried to make a new map like Michael did that my test for existing map failed
with an error. I decided to just use the lack of an error as evidence of the
existing map with g.findfile and changed the chunk of code to:

###############################################################################
# Determine if an element already exists

proc Gm::element_exists {elem name} {
	set failure [catch {exec [list "|g.findfile" "element=$elem" "file=$name"] >&
/dev/null}]

	return [expr {! $failure}]
}

###############################################################################
I tested this, making a new map (newmaptest below (edited - not actual name;
actual name reflected my frustration with committing a bug)). It worked. I then
added a vector layer to gis.m and clicked on the button to list existing maps.
The only one present was my new map. At this point I started looking around and
found the following files:

.  ..  .bash_history  .bashrc  DEFAULT_WIND  .gislock  MYNAME  .tmp  vector 
WIND  windows

./.tmp:
.  ..  Foxwaltz

./.tmp/Foxwaltz:
.  ..  14033.0  14033.0.pgm  14033.0.ppm  14033.1

./vector:
.  ..  newmaptest

./vector/newmaptest:
.  ..  cidx  coor  dbln  head  hist  topo

./windows:
.  ..  gism_temp_region

The existence of gism_temp_region and no other regions (I wasn't displaying anything)
means that gis.m was run more recently than the failure, though that doesn't
narrow it down much.

.bash_history reveals nothing. It doesn't really have the commands I've been
running in the console.

The files DEFAULT_WIND and WIND both contain:
proj:       99
zone:       0
north:      1
south:      0
east:       1
west:       0
cols:       1
rows:       1
e-w resol:  1
n-s resol:  1
top:        1
bottom:     0
cols3:      1
rows3:      1
depths:     1
e-w resol3: 1
n-s resol3: 1
t-b resol:  1

Note: the proj corruption of my windows occured a while ago. At the time I associated
it with the new ______ (I can't remember what). I have no idea how it happened.
I'm not certain if that's the same "proj" and "zone" that used to pop up. I think
it isn't because when I'd run g.region -p I the projection and 
zone would show descriptive text too. The survival of this data (or reintroduction
of it) could be a big clue.


Places that are likely to have caused this error:
The startup scripts

Things that were for sure run:
gis.m
v.digit
g.findfile

I also remember I accidentally ran g.mapsets.tcl from the menu while trying to
open guis to scroll arround. I closed it by clicking the window close button.
This is probably also worth looking into. (Little used, suspicious name)

Mon, May 8 2006 09:48:37    Priority changed to 99 by cshock  
Mon, May 8 2006 09:48:48    Final Priority changed to 99 by cshock  
Mon, May 8 2006 18:22:28    Mail sent by cshock  
Please help by suggesting what program could have done this short list of
symptoms:

All grass database files (vectors, rasters, regions, etc.) and their
directories completely removed.

Projection and other support files except for MYNAME removed. MYNAME still has
the correct name for the location, so the whole folder wasn't deleted.

WIND and DEFAULT_WIND either removed and rewritten with proj and zone lines or
all the defaults of 0 and 1 were written to the remaining files.
Mon, May 8 2006 20:26:17    Mail sent by michael.barton@asu.edu  
Return-Path <Michael.Barton@asu.edu>
Delivered-To grass-bugs@lists.intevation.de
Date Mon, 08 May 2006 11:25:06 -0700
From Michael Barton <michael.barton@asu.edu>
Subject Re: [GRASS-dev] [bug #4426] (grass) Almost entire GRASS database deleted. Suspect grass startup.
In-reply-to <20060508074716.D2753101EF1@lists.intevation.de>
To Paolo Cavallini via RT <grass-bugs@intevation.de>, grass-dev@grass.itc.it, Cedric Shock <cedricgrass@shockfamily.net>
Message-id <C084DB12.21340%michael.barton@asu.edu>
MIME-version 1.0
Content-type text/plain; charset=US-ASCII
Content-transfer-encoding 7bit
User-Agent Microsoft-Entourage/11.2.3.060209
Thread-Topic [GRASS-dev] [bug #4426] (grass) Almost entire GRASS database deleted. Suspect grass startup.
Thread-Index AcZyzLrU+XCudd6/EdqJDwAUUSYxwg==
X-Spam-Status No, hits=-4.9 tagged_above=-999.0 required=3.0 tests=BAYES_00
X-Spam-Level
Cedric,

I grabbed all of your recent changes from the cvs last night so I could make
some modifications.

So I think I'm working with the same thing you are in terms of TclTk (though
my binary C files are from Lorenzo's 22 April build).

I just closed GRASS, then reopened it. It worked fine. No loss of data. It's
hard to see how the startup could do anything that would delete data. There
is nothing in the code that modifies anything. It just looks for stuff and
reads it.

The culprit must lie elsewhere.

Michael
__________________________________________
Michael Barton, Professor of Anthropology
School of Human Evolution & Social Change
Center for Social Dynamics and Complexity
Arizona State University

phone: 480-965-6213
fax: 480-965-7671
www: http://www.public.asu.edu/~cmbarton


> From: Request Tracker <grass-bugs@intevation.de>
> Reply-To: Request Tracker <grass-bugs@intevation.de>
> Date: Mon,  8 May 2006 09:47:16 +0200 (CEST)
> To: <grass-dev@grass.itc.it>
> Subject: [GRASS-dev] [bug #4426] (grass) Almost entire GRASS database deleted.
> Suspect grass startup.
> 
> this bug's URL: http://intevation.de/rt/webrt?serial_num=4426
> -------------------------------------------------------------------------
> 
> Subject: Almost entire GRASS database deleted. Suspect grass startup.
> 
> Platform: GNU/Linux/x86
> grass obtained from: CVS
> grass binary for platform: Compiled from Sources
> GRASS Version: CVS development May 7th 2006
> 
> Virtually an entire mapset in a location was deleted somehow. These are all
> the things I can remember doing since the last time I saw it alive:
> 
> Short version: PLEASE AUDIT STARTUP SCRIPTS, g.mapsets.tcl !!!
> 
> Most of this is probably not interesting.
> 
> Files were still being listed when I worked on development on gis.m testing
> running v.digit started from a layer.
> 
> I made the commit: Digitizing a vector layer might actually work now. Todo:
> grab other layers that are selected and use them for the draw commands.
> 
> I made a bunch of changes to mouse scrolling in gis.m.
> 
> I tested gis.m and opening guis for a bunch of modules and moved the cursor
> around and used the mouse wheel and clicked on frames and things.
> 
> I made these commits:
> 
> * Mouse scroll the layer tree and options windows.
> 
> * Improvements to scrolling. Scrolling is much more specific (must have
> heyboard or mouse focus). Binding scrolling is no longer a code leak.
> 
> I saved a scrolling change I hadn't saved and tested yet, made some more tests
> of gis.m just like above and made the commit: No more code or memmory leak.
> (Forgot to save part of edit before
> testing and committing)
> 
> I started making edits to gronsole.tcl to fix scrolling.
> 
> I closed all of grass (I left)
> 
> I reopened grass by clicking on a launcher button that, in an xterm, runs:
> bash -c "cd ~/grassdata; grass61"
> I don't usually run grass this way. I think I normally open a terminal and
run
> grass61.
> 
> The new startup screen came up; I was mildly surprised (I havn't seen it very
> many times yet).  I clicked on the button in the lower left corner (Location
> and PERMANENT mapset were already selected).
> 
> I finished changes to gronsole.tcl (bindings on scrollings) and tested it.
As
> I had suspected It didn't really work.
> 
> I got an email from Michael, "V.digit doesn't work with  new maps??". I
> revisited the changes of my earlier commit "Digitizing a vector...". I
> observed that if I tried to make a new map like Michael did that my test for
> existing map failed with an error. I decided to just use the lack of an error
> as evidence of the existing map with g.findfile and changed the chunk of code
> to:
> 
> 
##############################################################################>
#
> # Determine if an element already exists
> 
> proc Gm::element_exists {elem name} {
> set failure [catch {exec [list "|g.findfile" "element=$elem" "file=$name"]
>&
> /dev/null}]
> 
> return [expr {! $failure}]
> }
> 
> 
##############################################################################>
#
> 
> I tested this, making a new map (newmaptest below (edited - not actual name;
> actual name reflected my frustration with committing a bug)). It worked. I
> then added a vector layer to gis.m and clicked on the button to list existing
> maps. The only one present was my new map. At this point I started looking
> around and found the following files:
> 
> .  ..  .bash_history  .bashrc  DEFAULT_WIND  .gislock  MYNAME  .tmp  vector
> WIND  windows
> 
> ./.tmp:
> .  ..  Foxwaltz
> 
> ./.tmp/Foxwaltz:
> .  ..  14033.0  14033.0.pgm  14033.0.ppm  14033.1
> 
> ./vector:
> .  ..  newmaptest
> 
> ./vector/newmaptest:
> .  ..  cidx  coor  dbln  head  hist  topo
> 
> ./windows:
> .  ..  gism_temp_region
> 
> The existence of gism_temp_region and no other regions (I wasn't displaying
> anything) means that gis.m was run more recently than the failure, though that
> doesn't narrow it down much.
> 
> .bash_history reveals nothing. It doesn't really have the commands I've been
> running in the console.
> 
> The files DEFAULT_WIND and WIND both contain:
> proj:       99
> zone:       0
> north:      1
> south:      0
> east:       1
> west:       0
> cols:       1
> rows:       1
> e-w resol:  1
> n-s resol:  1
> top:        1
> bottom:     0
> cols3:      1
> rows3:      1
> depths:     1
> e-w resol3: 1
> n-s resol3: 1
> t-b resol:  1
> 
> Note: the proj corruption of my windows occured a while ago. At the time I
> associated it with the new ______ (I can't remember what). I have no idea how
> it happened. I'm not certain if that's the same "proj" and "zone" that used
to
> pop up. I think it isn't because when I'd run g.region -p I the projection
and
> zone would show descriptive text too. The survival of this data (or
> reintroduction of it) could be a big clue.
> 
> 
> Places that are likely to have caused this error:
> The startup scripts
> 
> Things that were for sure run:
> gis.m
> v.digit
> g.findfile
> 
> I also remember I accidentally ran g.mapsets.tcl from the menu while trying
to
> open guis to scroll arround. I closed it by clicking the window close button.
> This is probably also worth looking into. (Little used, suspicious name)
> 
> 
> 
> -------------------------------------------- Managed by Request Tracker
> 
> 


Mon, May 8 2006 20:32:11    Mail sent by hmitaso@unity.ncsu.edu  
Return-Path <hmitaso@unity.ncsu.edu>
Delivered-To grass-bugs@lists.intevation.de
Message-ID <445F8EFB.7070709@unity.ncsu.edu>
Date Mon, 08 May 2006 14:33:31 -0400
From Helena Mitasova <hmitaso@unity.ncsu.edu>
User-Agent Thunderbird 1.5 (X11/20060313)
MIME-Version 1.0
To Cedric Shock via RT <grass-bugs@intevation.de>
Cc grass-dev@grass.itc.it
Subject Re: [GRASS-dev] [bug #4426] (grass) Almost entire GRASS database deleted. Suspect grass startup.
References <20060508162228.D7AEB101EF8@lists.intevation.de>
In-Reply-To <20060508162228.D7AEB101EF8@lists.intevation.de>
Content-Type text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding 7bit
X-PMX-Version 5.1.2.240295, Antispam-Engine: 2.3.0.1, Antispam-Data: 2006.5.8.111606
X-Spam-Status No, hits=-4.9 tagged_above=-999.0 required=3.0 tests=BAYES_00
X-Spam-Level
Cedric Shock via RT wrote:
> Please help by suggesting what program could have done this short list of
> symptoms:
>
> All grass database files (vectors, rasters, regions, etc.) and their
> directories completely removed.
>
> Projection and other support files except for MYNAME removed. MYNAME still
has
> the correct name for the location, so the whole folder wasn't deleted.
>
> WIND and DEFAULT_WIND either removed and rewritten with proj and zone lines
or
> all the defaults of 0 and 1 were written to the remaining files.
>
> -------------------------------------------- Managed by Request Tracker
>   
this happened to me about 10 years ago when working with GRASS4.1.
I was testing a completely rewritten version of r.flow and we have never 
figured out what caused it.
We could not track it to r.flow itself so it must have been something 
related to installing a new module
while working in an existing GRASS location and running that new module 
then erased the files.
But I have recompiled and installed new modules while running GRASS many 
times since then and
I haven't seen this happen anymore.

Helena
> _______________________________________________
> grass-dev mailing list
> grass-dev@grass.itc.it
> http://grass.itc.it/mailman/listinfo/grass-dev
>   


-- 
Helena Mitasova
Department of Marine, Earth and Atmospheric Sciences
North Carolina State University
1125 Jordan Hall
NCSU Box 8208
Raleigh, NC 27695-8208
http://skagit.meas.ncsu.edu/~helena/

email: hmitaso@unity.ncsu.edu
ph: 919-513-1327 (no voicemail)
fax 919 515-7802


Tue, May 9 2006 09:07:17    Mail sent by hamish_nospam@yahoo.com  
Return-Path <hamish_nospam@yahoo.com>
Delivered-To grass-bugs@lists.intevation.de
Date Tue, 9 May 2006 19:07:06 +1200
From Hamish <hamish_nospam@yahoo.com>
To Cedric Shock via RT <grass-bugs@intevation.de>
Cc grass-dev@grass.itc.it
Subject Re: [GRASS-dev] [bug #4426] (grass) Almost entire GRASS database deleted. Suspect grass startup.
Message-Id <20060509190706.367c2072.hamish_nospam@yahoo.com>
In-Reply-To <20060508162228.D7AEB101EF8@lists.intevation.de>
References <20060508162228.D7AEB101EF8@lists.intevation.de>
X-Mailer Sylpheed version 1.0.4 (GTK+ 1.2.10; i386-pc-linux-gnu)
X-Face M<EoB)"*Z~u!,vFhXmw}R_KbdBta*P_=T|rbBL'e1/CQ9;/1g\BU3&!=y8ria$2Uk!HT&BB 8i?|X_+7~1jsy}F~g$2va%3fV`*=L(*cem[@3\yg,G,@rg6/QMJ
Mime-Version 1.0
Content-Type text/plain; charset=US-ASCII
Content-Transfer-Encoding 7bit
X-Spam-Status No, hits=-4.0 tagged_above=-999.0 required=3.0 tests=BAYES_00, FORGED_YAHOO_RCVD
X-Spam-Level
> Please help by suggesting what program could have done this short list
> of symptoms:
> 
> All grass database files (vectors, rasters, regions, etc.) and their
> directories completely removed.
> 
> Projection and other support files except for MYNAME removed. MYNAME
> still has the correct name for the location, so the whole folder
> wasn't deleted.

> WIND and DEFAULT_WIND either removed and rewritten with proj and zone
> lines or all the defaults of 0 and 1 were written to the remaining
> files.

this suggests it happened during a session, g.region creates a new WIND
if it finds that it is missing.

were you in the PERMANENT mapset at the time?

stuff to check:

$GISBASE/etc/clean_temp

I always get lots of /tmp/grass6-$USER-$$  dirs left over after exiting
GRASS, suggesting they might not be getting removed correctly.
(maybe from repeated `grass61 -gui` <cancel> ?)

e2fsck?

It is interesting that directories were removed too(?), suggesting
"rm -rf" not "rm -f", or some sort of "find | rm -f":

A while back I made a mistake coding a script which didn't check that a
g.tempfile was successfully created. At the cleanup() stage it did
"rm -f ${TMP}*", but g.tempfile failed and TMP was unset, so it ran as
"rm -f *" and wiped off all the files (but not dirs) in a home dir.
luckily my victim had a recent backup :/

I don't think common unlink() will remove dirs.

Is there a map name with loc/mapset name the same as a map name?

corrupted stream of data sent to the shell? `find` files moved to
other parts of the file system?

luck,
Hamish


Tue, May 9 2006 09:08:46    Mail sent by hamish_nospam@yahoo.com  
Return-Path <hamish_nospam@yahoo.com>
Delivered-To grass-bugs@lists.intevation.de
Date Tue, 9 May 2006 19:08:39 +1200
From Hamish <hamish_nospam@yahoo.com>
To Cedric Shock via RT <grass-bugs@intevation.de>
Cc grass-dev@grass.itc.it
Subject Re: [GRASS-dev] [bug #4426] (grass) Almost entire GRASS database deleted. Suspect grass startup.
Message-Id <20060509190839.254fd977.hamish_nospam@yahoo.com>
In-Reply-To <20060508162228.D7AEB101EF8@lists.intevation.de>
References <20060508162228.D7AEB101EF8@lists.intevation.de>
X-Mailer Sylpheed version 1.0.4 (GTK+ 1.2.10; i386-pc-linux-gnu)
X-Face M<EoB)"*Z~u!,vFhXmw}R_KbdBta*P_=T|rbBL'e1/CQ9;/1g\BU3&!=y8ria$2Uk!HT&BB 8i?|X_+7~1jsy}F~g$2va%3fV`*=L(*cem[@3\yg,G,@rg6/QMJ
Mime-Version 1.0
Content-Type text/plain; charset=US-ASCII
Content-Transfer-Encoding 7bit
X-Spam-Status No, hits=-4.0 tagged_above=-999.0 required=3.0 tests=BAYES_00, FORGED_YAHOO_RCVD
X-Spam-Level
> Please help by suggesting what program could have done this short list
> of symptoms:
> 
> All grass database files (vectors, rasters, regions, etc.) and their
> directories completely removed.
> 
> Projection and other support files except for MYNAME removed. MYNAME
> still has the correct name for the location, so the whole folder
> wasn't deleted.
> 
> WIND and DEFAULT_WIND either removed and rewritten with proj and zone
> lines or all the defaults of 0 and 1 were written to the remaining
> files.


what happens if you try and create a new location reusing an existing
name?


??
Hamish


Tue, May 9 2006 11:02:50    Mail sent by glynn@gclements.plus.com  
Return-Path <glynn@gclements.plus.com>
Delivered-To grass-bugs@lists.intevation.de
From Glynn Clements <glynn@gclements.plus.com>
MIME-Version 1.0
Content-Type text/plain; charset=us-ascii
Content-Transfer-Encoding 7bit
Message-ID <17504.23219.946478.810936@cerise.gclements.plus.com>
Date Tue, 9 May 2006 10:02:43 +0100
To Hamish <hamish_nospam@yahoo.com>
Cc Cedric Shock via RT <grass-bugs@intevation.de>, grass-dev@grass.itc.it
Subject Re: [GRASS-dev] [bug #4426] (grass) Almost entire GRASS database deleted. Suspect grass startup.
In-Reply-To <20060509190706.367c2072.hamish_nospam@yahoo.com>
References <20060508162228.D7AEB101EF8@lists.intevation.de> <20060509190706.367c2072.hamish_nospam@yahoo.com>
X-Mailer VM 7.07 under 21.4 (patch 15) "Security Through Obscurity" XEmacs Lucid
X-Spam-Status No, hits=-3.0 tagged_above=-999.0 required=3.0 tests=BAYES_00, REMOVE_REMOVAL_2WORD
X-Spam-Level
Hamish wrote:

> I don't think common unlink() will remove dirs.

It won't; unlink() removes files, rmdir() removes (empty) directories,
remove() removes either. There is no system call or library function
to delete a directory tree.

> Is there a map name with loc/mapset name the same as a map name?
> 
> corrupted stream of data sent to the shell? `find` files moved to
> other parts of the file system?

That's a useful point; "mv" and rename() will rename an entire
directory tree with no complaints.

The only "rm -rf" commands in etc/Init.sh shouldn't be able to remove
the wrong directory even if $tmp is unset or there were problems in
constructing its value.

The current clean_temp program only deletes files, not directories
(not even empty ones).

-- 
Glynn Clements <glynn@gclements.plus.com>


Mon, May 15 2006 15:25:08    Mail sent by neteler@itc.it  
Return-Path <neteler@itc.it>
Delivered-To grass-bugs@lists.intevation.de
Date Mon, 15 May 2006 15:24:52 +0200
From Markus Neteler <neteler@itc.it>
To Glynn Clements <glynn@gclements.plus.com>
Cc Roberto Flor <flor@itc.it>, grass-bugs@intevation.de
Subject Re: grass61 new clean_tmp.c (also comment to [bug #4426])
Message-ID <20060515132452.GB32151@bartok.itc.it>
References <443288E8.6010704@itc.it> <17458.48410.893244.755509@cerise.gclements.plus.com> <4433BB97.3030801@itc.it> <17460.9894.124453.347187@cerise.gclements.plus.com> <20060503154532.GT15014@bartok.itc.it> <17497.8429.727770.422160@cerise.gclements.plus.com>
Mime-Version 1.0
Content-Type text/plain; charset=us-ascii
Content-Disposition inline
In-Reply-To <17497.8429.727770.422160@cerise.gclements.plus.com>
X-PGP-Key http://www.gdf-hannover.de/neteler/markus_gpgkey.asc
X-PGP-Fingerprint D4D5 2F80 120E AD60 E2F6 2297 21B3 D02B E1E7 E789
User-Agent Mutt/1.5.11
X-Spam-Status No, hits=-4.9 tagged_above=-999.0 required=3.0 tests=BAYES_00
X-Spam-Level
On Wed, May 03, 2006 at 10:30:21PM +0100, Glynn Clements wrote:
> Markus Neteler wrote:
> 
> > attached the latest version of clean_tmp.c and getpar.c
> > from Roberto. If you prefer, I can also send it to the
> > list for review.
> 
> > 	if ( (pathlen=snprintf(buf,BUF_MAX,"%s/%s",pathname,cur_entry->d_name))
>= BUF_MAX) 
> > 		G_fatal_error("clean_temp: exceeded maximum pathname length %d, got %d,
should'nt happen",BUF_MAX,pathlen);
> 
> Unfortunately:
> 
> 1. snprintf() is C99; not all systems have it.

So far it is implemented a few times in GRASS:

find . -type f -name "*.c" -exec grep -l snprintf {} \;
./raster3d/r3.in.ascii/main.c
./db/drivers/dbf/dbfexe.c
./raster/r.support/front/front.c
./raster/r.support/front/check.c
./raster/r.support/front/run.c
./raster/r.support/modhead/check_un.c
./raster/r.support/modhead/modhead.c
./raster/r.support/modhead/ask_format.c
./lib/db/sqlp/lex.yy.c
./lib/db/dbmi_client/select.c
./lib/gis/user_config.c
./lib/vector/dglib/examples/components.c


Should this become at least a G_snprintf() wrapper
in libgis?

> 2. Some snprintf() implementations return a negative result if the output is
truncated.
> 
> It's simpler to use a manual check, e.g.:
> 
>     int pathlen = strlen(pathname);
> 
>     ...
> 
>     while ((cur_entry = readdir(curdir)))
>     {
>         ...
> 
>         if (pathlen + strlen(cur_entry->d_name) + 2 > BUF_MAX)
>         {
>                 G_warning(...);
>                 continue;
>         }
>

This should then go into G_snprintf()?

Markus


Tue, May 16 2006 05:21:13    Mail sent by glynn@gclements.plus.com  
Return-Path <glynn@gclements.plus.com>
Delivered-To grass-bugs@lists.intevation.de
From Glynn Clements <glynn@gclements.plus.com>
MIME-Version 1.0
Content-Type text/plain; charset=us-ascii
Content-Transfer-Encoding 7bit
Message-ID <17513.17675.982612.184751@cerise.gclements.plus.com>
Date Tue, 16 May 2006 04:20:43 +0100
To Markus Neteler <neteler@itc.it>
Cc Roberto Flor <flor@itc.it>, grass-bugs@intevation.de
Subject Re: grass61 new clean_tmp.c (also comment to [bug #4426])
In-Reply-To <20060515132452.GB32151@bartok.itc.it>
References <443288E8.6010704@itc.it> <17458.48410.893244.755509@cerise.gclements.plus.com> <4433BB97.3030801@itc.it> <17460.9894.124453.347187@cerise.gclements.plus.com> <20060503154532.GT15014@bartok.itc.it> <17497.8429.727770.422160@cerise.gclements.plus.com> <20060515132452.GB32151@bartok.itc.it>
X-Mailer VM 7.07 under 21.4 (patch 15) "Security Through Obscurity" XEmacs Lucid
X-Spam-Status No, hits=-4.9 tagged_above=-999.0 required=3.0 tests=BAYES_00
X-Spam-Level
Markus Neteler wrote:

> > > attached the latest version of clean_tmp.c and getpar.c
> > > from Roberto. If you prefer, I can also send it to the
> > > list for review.
> > 
> > > 	if ( (pathlen=snprintf(buf,BUF_MAX,"%s/%s",pathname,cur_entry->d_name))
>= BUF_MAX) 
> > > 		G_fatal_error("clean_temp: exceeded maximum pathname length %d, got %d,
should'nt happen",BUF_MAX,pathlen);
> > 
> > Unfortunately:
> > 
> > 1. snprintf() is C99; not all systems have it.
> 
> So far it is implemented a few times in GRASS:
> 
> find . -type f -name "*.c" -exec grep -l snprintf {} \;
> ./raster3d/r3.in.ascii/main.c
> ./db/drivers/dbf/dbfexe.c
> ./raster/r.support/front/front.c
> ./raster/r.support/front/check.c
> ./raster/r.support/front/run.c
> ./raster/r.support/modhead/check_un.c
> ./raster/r.support/modhead/modhead.c
> ./raster/r.support/modhead/ask_format.c
> ./lib/db/sqlp/lex.yy.c
> ./lib/db/dbmi_client/select.c
> ./lib/gis/user_config.c
> ./lib/vector/dglib/examples/components.c
> 
> 
> Should this become at least a G_snprintf() wrapper
> in libgis?

I'm not sure.

If libc doesn't provide snprintf, the only options are to:

1. Implement it from scratch.
2. Write a wrapper which uses fprintf() to write the string to a
temporary file then reads the temporary file into the buffer.
3. Write a wrapper which discards the size argument and calls
vsprintf().

Option 1 is a fair amount of work (even extracting the code from libc
isn't simple). Option 2 is messy and option 3 creates the risk of
buffer overruns.

This issue has been discussed on the list in the past; IIRC, the
discussion was initiated by someone trying to compile GRASS on a
system which lacked snprintf(). I don't recall the conclusion.

Changing the code to use G_asprintf() instead is one alternative
(although G_asprintf() uses a temporary file on systems which lack
asprintf()).

-- 
Glynn Clements <glynn@gclements.plus.com>


Wed, May 17 2006 14:16:38    Mail sent by neteler@itc.it  
Return-Path <neteler@itc.it>
Delivered-To grass-bugs@lists.intevation.de
Date Wed, 17 May 2006 14:16:29 +0200
From Markus Neteler <neteler@itc.it>
To Glynn Clements <glynn@gclements.plus.com>
Cc Roberto Flor <flor@itc.it>, grass-bugs@intevation.de
Subject Re: grass61 new clean_tmp.c (also comment to [bug #4426])
Message-ID <20060517121629.GK24070@bartok.itc.it>
References <443288E8.6010704@itc.it> <17458.48410.893244.755509@cerise.gclements.plus.com> <4433BB97.3030801@itc.it> <17460.9894.124453.347187@cerise.gclements.plus.com> <20060503154532.GT15014@bartok.itc.it> <17497.8429.727770.422160@cerise.gclements.plus.com> <20060515132452.GB32151@bartok.itc.it> <17513.17675.982612.184751@cerise.gclements.plus.com>
Mime-Version 1.0
Content-Type text/plain; charset=us-ascii
Content-Disposition inline
In-Reply-To <17513.17675.982612.184751@cerise.gclements.plus.com>
X-PGP-Key http://www.gdf-hannover.de/neteler/markus_gpgkey.asc
X-PGP-Fingerprint D4D5 2F80 120E AD60 E2F6 2297 21B3 D02B E1E7 E789
User-Agent Mutt/1.5.11
X-Spam-Status No, hits=-4.9 tagged_above=-999.0 required=3.0 tests=BAYES_00
X-Spam-Level
On Tue, May 16, 2006 at 04:20:43AM +0100, Glynn Clements wrote:
> Markus Neteler wrote:
> > > 
> > > 1. snprintf() is C99; not all systems have it.
> > 
> > So far it is implemented a few times in GRASS:
...
> > Should this become at least a G_snprintf() wrapper
> > in libgis?
> 
> I'm not sure.

This was also your suggestion in 2002 :-)
 http://grass.itc.it/pipermail/grass5/2002-April/005218.html
 "Should I add a stub implementation to libgis?"

Here is the full thread:
 http://grass.itc.it/pipermail/grass5/2002-April/thread.html#5117
 "snprintf and other compile errors on IRIX"

IMHO, a quick solution to the problem is:
- make G_snprintf() a wrapper around snprintf()
- test therein if the system support snprintf(), if not, issue
  an error (or write an implementation, see below).

I wonder if we should support every system on earth - in any case
we could identify systems with snprintf() being unsupported
using G_snprintf().

BTW, here a thread on mingW:
 http://lists-archives.org/mingw-users/03714-mingw-snprintf-not-following-c99-standard.html
See here for a GPL'ed snprintf.c (suggested in above mingW thread):
 http://www.ijs.si/software/snprintf/

Markus


Mon, Jul 17 2006 17:53:52    Mail sent by mneteler  
All remaining snprintf() were changed to G_snprintf().

Markus
Mon, Jul 17 2006 17:54:54    Comments added by mneteler  
Also clean_temp was rewritten to remove directories within
the .tmp/ subdirectory.

Markus
Mon, Aug 14 2006 18:10:35    Priority changed to 69 by bernhard  
Mon, Aug 14 2006 18:13:29    Comments added by bernhard  
No activity for three week.  
Not reproducable. 
 
I am lowering priority. 
 
It could have been a bug in the operating system, 
file system or hardware as well. 
 
Freezing and analysing the filesystem might give a chance in such 
cases to establish a time line, when was which file deleted. 
 
Sat, Nov 4 2006 18:39:43    Mail sent by mneteler  
We assume that the problem is solved. Feel free to re-open if not.

Markus


https://intevation.de/rt/webrt?serial_num=4426
Sat, Nov 4 2006 18:39:49    Status changed to resolved by mneteler  
Sat, Nov 4 2006 18:39:49    Mail sent by mneteler  
We assume that the problem is solved. Feel free to re-open if not.

Markus


https://intevation.de/rt/webrt?serial_num=4426
Comment | Reply | Take | Open

You are currently authenticated as guest.
[Show Configuration] [Login as another user]

Users Guide - Mail Commands - Homepage of RequestTracker 1.0.7 - list any request