MemoryError - InVEST_3.7 Recreation and Tourism

Dear users and developers,

I am running the Rec and Tourism model and encountered a Memory error. For one study area, I ran 4 different trials. When I chose grid size to be 100 m, I encountered this error. When I increased the grid size to be 1000 m, it worked on one trial where I used start and end year = 2016-2017. But when I tried again with start and end year =2005-2017, I encounter the MemoryError again.

At first we thought it was due to some missing data on a raster file, but when we adjusted the raster file, we still encounter the error. Attached are 2 log files (from 2 study areas) for your reference.

Any help is appreciated!

InVEST-Recreation-Model-log-2019-11-27–15_57_22.txt (29.9 KB)

InVEST-Recreation-Model-log-2019-11-28–09_58_30.txt (3.3 KB)

Liyana

Thanks for reporting this. From the logs, it looks like two separate issues that both happen to produce a memory error.

In the first log, it looks like a memory error during zonal statistics. To reproduce that, could you share your output workspace H:\PuntaArenas_runs\test1_PA, and the rasters referenced in PuntaArenas_predictors.csv?

The second log shows a memory error before any of the predictor data are processed, just after grdding the AOI. To reproduce that one, could you share H:/Invest_recreation/Processed/Magallanes/RioVerde.shp?

Hi Dave,

thank you for your message. Could you clarify which file you would like to check for H:\PuntaArenas_runs\test1_PA ? Since the whole folder is too large to be shared here.

These are the rasters that are referenced in PuntaArenas_predictors.csv :
landuse_pa.zip (51.7 KB)
lat_pa.zip (3.7 MB)
There’s also another raster file (elevation) which is too big to be shared here, although I suspect that is the problematic file, due to some missing info in the elevation data (some areas have no elevation data).

H:/Invest_recreation/Processed/Magallanes/RioVerde.shp
RioVerde_shp.zip (52.5 KB)

Thank you for taking the time to help!

Liyana

Hi Liyana, it does look like the memory error is occurring on the elevation raster. You can always send me a link to dropbox, gdrive, etc if it’s too big to upload here. I would also need the gridded AOI from that run, which would be in the output workspace (pud_results.shp or predictor_data.shp). That’s the issue from your first log.

I’ll try to reproduce the issue from your second log using RioVerde.shp.

Your second log showed a MemoryError right after gridding the RioVerde.shp AOI. I was not able to reproduce that. I gridded that AOI with square cells, size 100 m, and the memory usage peaked at ~390 MB, when a zip archive of the gridded AOI is created.

Does that error from your second log reproduce for you? If so, how much memory do you have on your system?

It’s also worth mentioning that if your objective is to produce meaningful results from the regression, then it’s quite likely you will want a larger cell size. Very small cells will probably result in the vast majority of cells with 0 PUDs, and that can be problematic for a linear regression with PUD as the response variable.