Wow, @dowthut, that’s a very detailed article, which fills in a lot of the “data cleaning is hard” gaps that I noted in my response. Thanks to @Mariam for telling us about it in this post as well. It looks like a very valuable contribution to help us validate our modeling results, and I agree that the detail given should be applicable to other models and observed datasets.
Validation often is a rather complex process, as we can see with your reference to multiple R packages and other tools that can be brought in to help. So I also very much appreciate you providing your (well-commented!) R code with the paper.
As for watershed delineation, I don’t think we’ve done a formal comparison between DelineateIt and other watershed-delineation tools. My main experience has been more trial and error, with Arc Desktop’s watershed tool mostly not working, so trying ArcHydro, which was much better but buggy and complicated, trying a few others, then settling on DelineateIt a while back because it’s pretty simple and works, including on nested watersheds.
The main differences of note for me are the difference in stream network and resulting watershed between DEMs, more than the tool algorithm itself, since those results can vary dramatically. And there’s another place where it’s more art than science, deciding whether the stream network/watersheds from a particular DEM are “good enough” since they’re pretty much never exactly the same as a real-world stream map. Maybe there’s a tool out there to to quantify similarity between them that would make that process more objective, but I’ve never looked for one.
~ Stacie