Thanks for your product and the useful documentation available. I wonder whether you could help me out with an issue I have stumbled upon whilst running the SDR model using my data - everything worked fine with the Gura catchment sample data.
I tried to run the model on a catchment which is about 1400kmq using input rasters with 10m pixel resolution. I first attempted to run the model at the whole catchment scale having the input files stored in different folders. Please find attached the log file (CATCHMENT_Invest xxx). I thought the model was working fine until it started the āTask._call(1234) INFO 0.0% completeā at 19:51:08. Regardless of the +10 minutes between one task call and the next one, nothing seemed to happen to the progression status of the task. This made me think something was potentially wrong. Furthermore, when I clicked on the program window, it froze and when I eventually tried to close it, it crashed. I thought the problem could be due to the size of my catchment.
Therefore, I then tried to run the model on a sub-catchment which is about 200kmq (always using input rasters with 10m pixel resolution). This time, I even stored all the input data in the same folder. The model ran quicker until it hit again the āTask._call(1234) INFO 0.0% completeā and did exactly what had done before. Please find attached the log file (SUBCATCHMENT_Invest xxx).
I canāt understand whether there is a problem at all or whether it simply takes a long to complete these āTask._call(1234) INFOā. However, given the fact this didnāt occur whilst using the Gura sample data, makes me think there is actually a problem with my input data? If so, which problem/s is likely to be?
With regards to my input data, the only two things I am not sure about are the DEM and Watersheds files. It says that the DEM should extend beyond the watersheds of interest, but mine is clipped to the watershed edge. Can this be the cause of the problem? About the Watersheds, instead of using a shapefile of polygons (sub-catchments), can I simply use the shapefile of the single watershed?
Thank you very much
Looking forward to hearing from you
All the best,
Massi
Iām glad to hear SDR is running well on the Gura sample data ā¦ thatās a good base case and so the model should be able to complete as expected.
In looking through your logfiles (thank you for including those!) and your notes about your study area, it looks like your inputs are quite large ā¦ on the order of about 20 billion pixels. This might take some time to compute, and it is possible that the progress logging might not indicate much progress (though Iād still expect there to be some progress).
That being said, the fact that the model is hanging makes me wonder if thereās an infinite loop in the modelās pitfilling operation. To make sure that isnāt the case, could you try filling your DEMās hydrological sinks? We usually recommend the Wang & Liu operation within SAGA/QGIS, but Iām sure ArcGIS has something comparable. Once the DEM has been pitfilled, could you try running the model again?
If filling sinks doesnāt fix the issue, could you provide a link to all of your input datasets so we can try running the model on your inputs and see if we can reproduce the issue?
I had filled my DEMās hydrological sinks before but I used the r.fill.dir tool of QGIS. I didnāt know that you guys would recommend the Wang & Liu tool within SAGA (you may want to consider specifying this in the inVEST documentation? If itās already there, my bad! I just hadnāt seen it.). Furthermore, I had no idea that a different depit/fill sinks tool would produce such a different output which is able to cause issues to the modelā¦
Anyway - thanks a lot again! Youāve been ace!
Massi
Oh awesome, thatās great to hear that filling hydrological sinks did the trick! Weāll make sure that some notes about filling hydrological sinks are in the userās guide.
Just so that we can make sure that this issue is fixed, would you be willing to share your DEM with us?
For what itās worth, there is a section of the User Guide Introduction called Working with the DEM, which lists filling sinks as a processing step, including recommending Wang & Liu. We can reword this to provide more detail if needed, and/or point to that section within the model chapters where DEMs are used? I donāt think we do that currently.
Thank you for pointing this out. Honestly, I hadnāt seen that before (my bad). However, I agree that you could point to that section within the model chapters where DEMs are used/mentioned or even in the tutorial? I think itād be easier for people to spot it and avoid making the same mistake I did.
Also, as I am reading the section now, would you mind clarifying to me what exactly you mean by āMultiple runs of Fill may be neededā? Iād have thought that once you run the fill sink tool once and obtain your filled dem thatād be it. Is this not the case? Do you mean that you might have to run again the fill tool this time using as input your dem that had already been filled once from the previous step (and so on for multiple times)?
Thank you! Iāve downloaded the DEM files and weāll take a look and see what we can find.
As for the differences between Wang&Liu and r.fill.dir, Iām afraid I donāt know the algorithms well enough to be able to compare and contrast them ā¦ but Iām also not surprised that they would perform differently. Some day it could be neat to have a comparison of the various pitfilling algorithms and in what ways they perform well, underperform, and when each should be used. If we have some time, we might be able to look into them!