I’ve previously run the coastal vulnerability model version 3.6 and am trying to re-run with the same data in 3.7. The only changes I’ve made to the input data are where they’re required by the new software version.
The model appears to be running fine until it gets to the “searching for geomorphology in point buffers” step. The current run has been showing that message for 12+ hours with no change, although according to task manager it’s still processing (CPU and memory usage are fluctuating). The same thing happened last time I ran it; I’ve since moved all of the data and workspace to the C drive (instead of network), but it doesn’t seem to have made a difference.
From what I can tell in the documentation (assuming the message is reflecting what the model is currently trying to do), it’s searching for geomorphology within 1/2 pixel of the shoreline. I’m using NOAA ESI for geomorphology and GSSHG for the shoreline; the two don’t exactly coincide but are pretty close for most of the area of interest.
The log for this run is attached. I’d really appreciate any ideas about what the issue is or what to try next.
Thanks for your help!
InVEST-Coastal-Vulnerability-log-2019-08-21–16_47_39.txt (25.2 KB)
Thanks for posting. The geomorphology processing is known to be one of the longer-running parts of this model due to a sub-optimal design. The good news is that we’ve tested a better design that shows about a 40x speedup! I’ll work on getting a development version of InVEST available for download with that improvement and I’ll post it here when it’s available.
In the meantime, I’m guessing you have a very highly detailed geomorphology layer so you could consider using a GIS tool to “simplify geometry”. That’s likely to help a bit, but probably won’t be a game-changer unless you’re willing to simplify so much that you sacrifice the high resolution in your data.
Thanks for the quick response! The 3.6 version was running in under 2 hours on the same machine with the same inputs, so the much longer run time convinced me that something was wrong. I’ll try simplifying the geomorphology layer and give it some more time. The updates you’re working on sound like a big improvement; I’ll keep an eye out for the new version.
Here’s a link to a development version of InVEST: (see new link below)
If you’re willing, it’s very helpful to us if you give it a try and let us know how it performs on your data. It has the more optimal geomorphology algorithm, and a few other speedups as well. A few tests I did show that this version will have runtimes close to the same or even faster than 3.6.
Also, you might be interested in trying a relatively new parallel processing feature of InVEST. If multiple CPUs are available, users can select the number to use by selecting from the dropdown menu in File > Settings > taskgraph_n_workers_parameter. The optimal number of CPUs is somewhat dependent on the input data, but usually there at least 2 operations that can run in parallel for this model.
Thanks! I tried the development version and got an error after a few minutes; looks like it’s something to do with the relief raster. I checked the input raster as well as the two intermediates created by the model (_clipped and clipped_projected tifs), and they all look ok to me. Log file is attached; I can send you the data if that would be helpful.
InVEST-Coastal-Vulnerability-log-2019-08-23–08_44_45.txt (14.2 KB)
Ahh thanks for testing this! This appears to be a problem just with this particular development build, not the model. I think I created this build with an incompatible version of one of the other python libraries. I’ll make sure I test the next one myself before posting it! More soon.
Finished successfully in 16 minutes - quite an improvement! I won’t have time to look at the outputs thoroughly until next week, but I’ll let you know if I notice anything unexpected. Thanks again!
Wow, that’s great to hear! Really appreciate you testing this for us. More feedback is always welcome!
One more issue with the new version that I’m hoping you can help with. When I open the main.coastal_exposure feature class in ArcGIS Pro, it seems to have trouble drawing the layer (sometimes it will appear initially but then disappear when I change the symbology), and when I open the attribute table I get an error box that reads: “Failed to open table. Error: Underlying DBMS error[Unable to execute transactional statement::database is locked].” I tried it both before and after closing the InVEST model program with the same results. Any suggestions?
Hi Katie, I haven’t experienced this myself, but my instinct is to double-check that no other software has that file open at the same time. And that could include moving the file to local drive if it’s on a shared network drive that is trying to read the file to sync it someplace else. Is anyone on the ESRI forums reporting similar issues with Geopackages?
Thanks…looks like ArcGIS Pro doesn’t handle geopackages that well. I was able to open it in ArcMap and export it as a shapefile to work with in Pro.