| Message/Author |
|
|
 P Cole
posted on Thursday, July 28, 2005 - 09:42 pm
|
|
|
Hi. What's the largest DEM (in Mb) that people have got CAESAR to work with?
Cheers,
Peter Cole |
|
|
| Tom posted on Friday, July 29, 2005 - 02:32 pm
|
|
|
Hello Pete,
5mb is about as much as I've used.. it makes quite a difference whether you are running it in catchment or reach mode. When in catchment mode, there are plenty of 'dead' cells where there is no fluvial activity - so the size of the total grid can be quite large (up to 1000 by 1000)..
Though the larger the grid, the more the memory it uses and the more time it takes to run.
This is where the 'memory limit' option can come in useful, as it can restrict the size of the array used to hold the grainsize data files.
Other options if your catchment is very large include chopping up the catchment into sub-catchments and running them on seperate machines. In effect parellising the model.
Hope this helps, if you want any more info on either of the above let me know.
Tom |
|
|
| Marco posted on Friday, July 29, 2005 - 03:19 pm
|
|
|
Hi Pete,
My short answer to you question is 1.2 MB.
However, the size of a DEM file will vary tremendously if the DEM elevations are specified as integer or real values. So much depends on the original data you have. For catchments, data are often supplied as integer numbers. Reach DEMs are often more accurate and specified as floating point values, thus increasing the DEM file size.
As Tom mentioned, the number of active cells (ie cells where elevation is specified rather than 'nodata' is a better indicator of the DEM size.
I've split my simulations in 4 sections: an upstream catchment (170 km2, at 50 m resolution: 69000 active cells), a tributary catchment (150 km2, at 50 m res: 58000 active cells), and an upper and a lower reach (both about 6km long, at 10 m resolution: 51000 and 55000 active cells). Each of the DEMs is about 1 MB in size.
For me, the advantage of splitting is twofold.
1: I can run the two catchments simultaneously, using two different PCs, and save some time in running the simulations.
2: I can run the reaches at higher resolution. The reach simulations are what I'm actually interested in. The catchment runs 'merely' generate input for the reach simulatios.
Hope this was a bit useful,
Marco Van De Wiel |
|
|
| Joe
posted on Monday, August 15, 2005 - 09:42 pm
|
|
|
Hello Pete,
I've had success running catchment simulations upto 11.5 MB and a reach simulation that was 14.4 MB. I have to say, both simulations were very slow and were fine for short duration (i.e. 1 - 10 years), but I wouldn't try running them for 10,000 years. I think the bigger problem you run into is filling up hard drives with output files if you decide to save them say every year.
Cheers
Joe |
|
|
| Peter_Cole
posted on Monday, August 29, 2005 - 09:38 pm
|
|
|
Thanks for the feedback guys. I finally loaded the DEM for visual inspection into a GIS (Idrisi). Clearly I'll have to do some serious work with my DEM as it's rather heafty, weighing in at 50Mb. I'm planning on running simulations in the order of 1-100 years. The resolution currently is 25m, with the area including inactive cells being 6911.06 km2 giving over 11 million cells. Thankfully only 3.4 million are active. Still, I can guess this will have to come down!
Tom, I see you mention 1000 x 1000 as the max size. What's keeping us from working with dimentions around 3000 x 3000 (this would fit my cachment as it stands)?
Cheers, |
|
|
| Tom posted on Tuesday, August 30, 2005 - 08:45 pm
|
|
|
Hi Pete,
I'd seriously consider two options.
FIrst going to 50m grid cells, which would give you 1/4 the number -
second try running it in sub-catchments - certainly to start with, you'll get much more of a feel for how fast it runs and what problems you may have when you tackle the whole thing.
Its tempting to do the whole thing in one go, but this can just be really slow.
But it should work over those time scales.. the way to think is coarse and small, then scale up as it works..
Hope this helps,
Tom |
|
|
| Peter posted on Thursday, October 27, 2005 - 09:51 am
|
|
|
| I've finally got the Motueka DEM down to a smaller size (and rotated so that the river mouth is on the right hand edge). I have it at 50m, 100m & 200m resolution. They are 9.1Mb, 2.5Mb & 633Kb in size respectively. They load in CAESAR but unfortunately, when I try to load rainfall data, CAESAR crashes! As well, the DEM doesn't fit into the window space of CAESAR - a minor issue but it looks odd with the DEM appearing between buttons. |
|
|
| Tom posted on Thursday, October 27, 2005 - 10:00 am
|
|
|
Hello Peter,
THey're large!
How large is your rainfall file? Is it say 20 years at hourly resolution? I ask because it sounds like the computer is running out of memory. This is because when the program initialises, it sets up the arrays, and defines the array size according to 1. the DEM size (for most of the arrays - elevation, water depth etc..) and 2. the length of the run - as there are arrays that store the input data. If the latter are not set correctly - or are too long, then the model can run out of memory and crash - as it initialises. THings to check are how long have you set the runs up to be? If you think this may be the problem, try setting it to just run a days worth of data - and set the duration (max run duration) to 24 hours... and if that fixes it then slowly ramp up the time.. to a month - a year etc.. It may well be that you can only run the model with 5 or 10 year chunks of data - but thats not too much of a pain, as you can just fire it up again when its finished..
I'd go for that with the 100 or 200m res DEM to start with, and see how that gets going just for starters.
Tom |
|
|
| Back to top |
|
