I am running the Rec and Tourism model and encountered a Memory error. For one study area, I ran 4 different trials. When I chose grid size to be 100 m, I encountered this error. When I increased the grid size to be 1000 m, it worked on one trial where I used start and end year = 2016-2017. But when I tried again with start and end year =2005-2017, I encounter the MemoryError again.
At first we thought it was due to some missing data on a raster file, but when we adjusted the raster file, we still encounter the error. Attached are 2 log files (from 2 study areas) for your reference.
Thanks for reporting this. From the logs, it looks like two separate issues that both happen to produce a memory error.
In the first log, it looks like a memory error during zonal statistics. To reproduce that, could you share your output workspace H:\PuntaArenas_runs\test1_PA, and the rasters referenced in PuntaArenas_predictors.csv?
The second log shows a memory error before any of the predictor data are processed, just after grdding the AOI. To reproduce that one, could you share H:/Invest_recreation/Processed/Magallanes/RioVerde.shp?
thank you for your message. Could you clarify which file you would like to check for H:\PuntaArenas_runs\test1_PA ? Since the whole folder is too large to be shared here.
These are the rasters that are referenced in PuntaArenas_predictors.csv : landuse_pa.zip (51.7 KB) lat_pa.zip (3.7 MB)
There’s also another raster file (elevation) which is too big to be shared here, although I suspect that is the problematic file, due to some missing info in the elevation data (some areas have no elevation data).
Hi Liyana, it does look like the memory error is occurring on the elevation raster. You can always send me a link to dropbox, gdrive, etc if it’s too big to upload here. I would also need the gridded AOI from that run, which would be in the output workspace (pud_results.shp or predictor_data.shp). That’s the issue from your first log.
I’ll try to reproduce the issue from your second log using RioVerde.shp.
Your second log showed a MemoryError right after gridding the RioVerde.shp AOI. I was not able to reproduce that. I gridded that AOI with square cells, size 100 m, and the memory usage peaked at ~390 MB, when a zip archive of the gridded AOI is created.
Does that error from your second log reproduce for you? If so, how much memory do you have on your system?
It’s also worth mentioning that if your objective is to produce meaningful results from the regression, then it’s quite likely you will want a larger cell size. Very small cells will probably result in the vast majority of cells with 0 PUDs, and that can be problematic for a linear regression with PUD as the response variable.
Thanks for sharing these data. Testing this out, my machine required about 6GB or RAM to process that elevation raster with the AOI in test1_PA. That’s a lot of memory and more than we should expect to be available to users.
The good news is we have an easy fix to make that zonal statistics function more memory efficient and faster. In this case, with the fix, that same function used ~2GB of RAM. That fix will be available in the next release (or I can share a development build of InVEST with the change, if you would like).
Workarounds for you right now include using a larger cell size (it sounds like you may have decided to do this for other reasons as well), or using a machine with more available memory.
thanks a lot for checking my data. As I used 2 different cell size for my other AOIs, I would like to be consistent and have 2 results for the Rio Verde AOI as well. I’d be happy to try and run it with the development build of InVEST.
When I use the Sample data of InVEST Recreation model, I set workspace (D:/F/…), Results suffix (rr), Area of Interest (andros_aoi.shp), Start Year (2005), End Year (2017), Predictor Table (predictors.csv), Scenario Predictor Table (predictors_scenario.csv), grid type(square), cell size (90,or 1.1898), there is an error.
I am thankful to you for your kind help.