DelineateIt Tool Slow

NatCap Software Support
1

Hi everyone! I am Maulana.

### What is the issue or question you have?

I am trying to do a global study on ecosystem services with the InVEST NDR Model. To do this, I of course need a watershed vector map.

I am using the DeliniateIt InVEST tool to generate watersheds from my DEM file, but after 1 hour, it is still not finished running.

Might anyone know why?

I spent one day of work trying to debug this and still no solution. The only alternative that I can see now is to try and generate the watersheds via ArcGIS if the error persists.

Thank you all!

What do you expect to happen?

  1. maybe its due to the size of my filled DEM file (around 634 MB)?

I am using the following DEM file from Download HydroSHEDS Core Products under “void filled DEM, global”

  1. maybe I chose a snap distance that is too small?

### What have you tried so far?

  1. Watershed Outlets

Tried using the following outlets (shape files) from HydroLAKES under “lake pour points”

Resulted in the following error message:
"Bounding box does not intersect at least one another: [-179, -56, -179, 83]

Tried to reproject the shape file so that it has the same projected coordinate system as the DEM file but still not working.

  1. Snap Distance

Tried 5, 10, 100

### Attach the logfile here:

2023-09-20 13:42:52,075 (natcap.invest.utils) utils.prepare_workspace(161) Level 100 Writing log messages to [C:\Users\ajma\Documents\ArcGIS\Projects\230915_InVEST_v1\Output\Output_DeliniateIt\Output_DelineateIt_v4\InVEST-natcap.invest.delineateit.delineateit-log-2023-09-20–13_42_52.txt]
2023-09-20 13:42:52,075 (main) cli.main(458) Level 100 Starting model with parameters:
Arguments for InVEST natcap.invest.delineateit.delineateit 3.13.0:
dem_path C:\Users\ajma\Documents\ArcGIS\Projects\230915_InVEST_v1\Output\hyd_glo_dem_15s_projected_filled.tif
detect_pour_points True
flow_threshold 100
n_workers -1
outlet_vector_path
results_suffix deliniated_hyd_glo_dem_15s_projected_filled_v4
skip_invalid_geometry False
snap_distance 100
snap_points True
workspace_dir C:\Users\ajma\Documents\ArcGIS\Projects\230915_InVEST_v1\Output\Output_DeliniateIt\Output_DelineateIt_v4

2023-09-20 13:43:36,432 (pygeoprocessing.routing.routing) Task._call(1093) INFO (fill pits): 0 of 1859113431 pixels complete
2023-09-20 13:43:47,019 (pygeoprocessing.routing.routing) Task._call(1093) INFO (fill pits): 44488320 of 1859113431 pixels complete
2023-09-20 13:43:58,103 (pygeoprocessing.routing.routing) Task._call(1093) INFO (fill pits): 71149312 of 1859113431 pixels complete
2023-09-20 13:44:09,009 (pygeoprocessing.routing.routing) Task._call(1093) INFO (fill pits): 88949760 of 1859113431 pixels complete
2023-09-20 13:44:20,001 (pygeoprocessing.routing.routing) Task._call(1093) INFO (fill pits): 106759424 of 1859113431 pixels complete
2023-09-20 13:44:31,016 (pygeoprocessing.routing.routing) Task._call(1093) INFO (fill pits): 133399168 of 1859113431 pixels complete
2023-09-20 13:44:42,005 (pygeoprocessing.routing.routing) Task._call(1093) INFO (fill pits): 160069888 of 1859113431 pixels complete
2023-09-20 13:44:53,009 (pygeoprocessing.routing.routing) Task._call(1093) INFO (fill pits): 186747264 of 1859113431 pixels complete
2023-09-20 13:45:04,000 (pygeoprocessing.routing.routing) Task._call(1093) INFO (fill pits): 204583296 of 1859113431 pixels complete
2023-09-20 13:45:15,006 (pygeoprocessing.routing.routing) Task._call(1093) INFO (fill pits): 231241344 of 1859113431 pixels complete
2023-09-20 13:45:26,000 (pygeoprocessing.routing.routing) Task._call(1093) INFO (fill pits): 249025408 of 1859113431 pixels complete
2023-09-20 13:45:37,002 (pygeoprocessing.routing.routing) Task._call(1093) INFO (fill pits): 275712512 of 1859113431 pixels complete
2023-09-20 13:45:48,000 (pygeoprocessing.routing.routing) Task._call(1093) INFO (fill pits): 311269120 of 1859113431 pixels complete
2023-09-20 13:45:59,013 (pygeoprocessing.routing.routing) Task._call(1093) INFO (fill pits): 337952512 of 1859113431 pixels complete
2023-09-20 13:46:10,004 (pygeoprocessing.routing.routing) Task._call(1093) INFO (fill pits): 364642816 of 1859113431 pixels complete
2023-09-20 13:46:21,000 (pygeoprocessing.routing.routing) Task._call(1093) INFO (fill pits): 409061248 of 1859113431 pixels complete
2023-09-20 13:46:32,000 (pygeoprocessing.routing.routing) Task._call(1093) INFO (fill pits): 444656256 of 1859113431 pixels complete
2023-09-20 13:46:43,003 (pygeoprocessing.routing.routing) Task._call(1093) INFO (fill pits): 480220544 of 1859113431 pixels complete
2023-09-20 13:46:54,007 (pygeoprocessing.routing.routing) Task._call(1093) INFO (fill pits): 515786240 of 1859113431 pixels complete
2023-09-20 13:47:05,016 (pygeoprocessing.routing.routing) Task._call(1093) INFO (fill pits): 551340928 of 1859113431 pixels complete
2023-09-20 13:47:16,006 (pygeoprocessing.routing.routing) Task._call(1093) INFO (fill pits): 578047616 of 1859113431 pixels complete
2023-09-20 13:47:27,002 (pygeoprocessing.routing.routing) Task._call(1093) INFO (fill pits): 622492800 of 1859113431 pixels complete
2023-09-20 13:47:38,004 (pygeoprocessing.routing.routing) Task._call(1093) INFO (fill pits): 666954368 of 1859113431 pixels complete
2023-09-20 13:47:49,007 (pygeoprocessing.routing.routing) Task._call(1093) INFO (fill pits): 702507648 of 1859113431 pixels complete
2023-09-20 13:48:00,016 (pygeoprocessing.routing.routing) Task._call(1093) INFO (fill pits): 729220608 of 1859113431 pixels complete
2023-09-20 13:48:11,004 (pygeoprocessing.routing.routing) Task._call(1093) INFO (fill pits): 773670016 of 1859113431 pixels complete
2023-09-20 13:48:22,001 (pygeoprocessing.routing.routing) Task._call(1093) INFO (fill pits): 818132736 of 1859113431 pixels complete
2023-09-20 13:48:33,001 (pygeoprocessing.routing.routing) Task._call(1093) INFO (fill pits): 862580480 of 1859113431 pixels complete
2023-09-20 13:48:44,003 (pygeoprocessing.routing.routing) Task._call(1093) INFO (fill pits): 889297792 of 1859113431 pixels complete
2023-09-20 13:48:55,008 (pygeoprocessing.routing.routing) Task._call(1093) INFO (fill pits): 933726080 of 1859113431 pixels complete
2023-09-20 13:49:06,004 (pygeoprocessing.routing.routing) Task._call(1093) INFO (fill pits): 960421504 of 1859113431 pixels complete
2023-09-20 13:49:17,006 (pygeoprocessing.routing.routing) Task._call(1093) INFO (fill pits): 1013777920 of 1859113431 pixels complete
2023-09-20 13:49:28,009 (pygeoprocessing.routing.routing) Task._call(1093) INFO (fill pits): 1075988480 of 1859113431 pixels complete
2023-09-20 13:49:39,002 (pygeoprocessing.routing.routing) Task._call(1093) INFO (fill pits): 1129379840 of 1859113431 pixels complete
2023-09-20 13:49:50,014 (pygeoprocessing.routing.routing) Task._call(1093) INFO (fill pits): 1191593088 of 1859113431 pixels complete
2023-09-20 13:50:01,006 (pygeoprocessing.routing.routing) Task._call(1093) INFO (fill pits): 1244959616 of 1859113431 pixels complete
2023-09-20 13:50:12,003 (pygeoprocessing.routing.routing) Task._call(1093) INFO (fill pits): 1289435392 of 1859113431 pixels complete
2023-09-20 13:50:23,043 (pygeoprocessing.routing.routing) Task._call(1093) INFO (fill pits): 1324976000 of 1859113431 pixels complete
2023-09-20 13:50:34,000 (pygeoprocessing.routing.routing) Task._call(1093) INFO (fill pits): 1369444096 of 1859113431 pixels complete
2023-09-20 13:50:45,005 (pygeoprocessing.routing.routing) Task._call(1093) INFO (fill pits): 1413935616 of 1859113431 pixels complete
2023-09-20 13:50:56,002 (pygeoprocessing.routing.routing) Task._call(1093) INFO (fill pits): 1467287424 of 1859113431 pixels complete
2023-09-20 13:51:07,003 (pygeoprocessing.routing.routing) Task._call(1093) INFO (fill pits): 1529495552 of 1859113431 pixels complete
2023-09-20 13:51:18,001 (pygeoprocessing.routing.routing) Task._call(1093) INFO (fill pits): 1600642176 of 1859113431 pixels complete
2023-09-20 13:51:29,000 (pygeoprocessing.routing.routing) Task._call(1093) INFO (fill pits): 1707352320 of 1859113431 pixels complete
2023-09-20 13:51:40,001 (pygeoprocessing.routing.routing) Task._call(1093) INFO (fill pits): 1822962432 of 1859113431 pixels complete
2023-09-20 13:51:42,448 (pygeoprocessing.routing.routing) Task._call(1093) INFO (fill pits): complete
2023-09-20 13:51:47,520 (pygeoprocessing.geoprocessing) geoprocessing.new_raster_from_base(1127) INFO filling new raster C:\Users\ajma\Documents\ArcGIS\Projects\230915_InVEST_v1\Output\Output_DeliniateIt\Output_DelineateIt_v4\flow_dir_d8_2023-09-20_11_51_42_vh4__v1y\plateau_distance.tif with 1859113431 – 11.01% complete
2023-09-20 13:51:52,522 (pygeoprocessing.geoprocessing) geoprocessing.new_raster_from_base(1127) INFO filling new raster C:\Users\ajma\Documents\ArcGIS\Projects\230915_InVEST_v1\Output\Output_DeliniateIt\Output_DelineateIt_v4\flow_dir_d8_2023-09-20_11_51_42_vh4__v1y\plateau_distance.tif with 1859113431 – 18.43% complete
2023-09-20 13:51:57,523 (pygeoprocessing.geoprocessing) geoprocessing.new_raster_from_base(1127) INFO filling new raster C:\Users\ajma\Documents\ArcGIS\Projects\230915_InVEST_v1\Output\Output_DeliniateIt\Output_DelineateIt_v4\flow_dir_d8_2023-09-20_11_51_42_vh4__v1y\plateau_distance.tif with 1859113431 – 24.75% complete
2023-09-20 13:52:02,526 (pygeoprocessing.geoprocessing) geoprocessing.new_raster_from_base(1127) INFO filling new raster C:\Users\ajma\Documents\ArcGIS\Projects\230915_InVEST_v1\Output\Output_DeliniateIt\Output_DelineateIt_v4\flow_dir_d8_2023-09-20_11_51_42_vh4__v1y\plateau_distance.tif with 1859113431 – 31.26% complete
2023-09-20 13:52:07,527 (pygeoprocessing.geoprocessing) geoprocessing.new_raster_from_base(1127) INFO filling new raster C:\Users\ajma\Documents\ArcGIS\Projects\230915_InVEST_v1\Output\Output_DeliniateIt\Output_DelineateIt_v4\flow_dir_d8_2023-09-20_11_51_42_vh4__v1y\plateau_distance.tif with 1859113431 – 38.15% complete
2023-09-20 13:52:12,530 (pygeoprocessing.geoprocessing) geoprocessing.new_raster_from_base(1127) INFO filling new raster C:\Users\ajma\Documents\ArcGIS\Projects\230915_InVEST_v1\Output\Output_DeliniateIt\Output_DelineateIt_v4\flow_dir_d8_2023-09-20_11_51_42_vh4__v1y\plateau_distance.tif with 1859113431 – 45.32% complete
2023-09-20 13:52:17,529 (pygeoprocessing.geoprocessing) geoprocessing.new_raster_from_base(1127) INFO filling new raster C:\Users\ajma\Documents\ArcGIS\Projects\230915_InVEST_v1\Output\Output_DeliniateIt\Output_DelineateIt_v4\flow_dir_d8_2023-09-20_11_51_42_vh4__v1y\plateau_distance.tif with 1859113431 – 51.97% complete
2023-09-20 13:52:22,530 (pygeoprocessing.geoprocessing) geoprocessing.new_raster_from_base(1127) INFO filling new raster C:\Users\ajma\Documents\ArcGIS\Projects\230915_InVEST_v1\Output\Output_DeliniateIt\Output_DelineateIt_v4\flow_dir_d8_2023-09-20_11_51_42_vh4__v1y\plateau_distance.tif with 1859113431 – 59.02% complete
2023-09-20 13:52:27,530 (pygeoprocessing.geoprocessing) geoprocessing.new_raster_from_base(1127) INFO filling new raster C:\Users\ajma\Documents\ArcGIS\Projects\230915_InVEST_v1\Output\Output_DeliniateIt\Output_DelineateIt_v4\flow_dir_d8_2023-09-20_11_51_42_vh4__v1y\plateau_distance.tif with 1859113431 – 66.20% complete
2023-09-20 13:52:32,533 (pygeoprocessing.geoprocessing) geoprocessing.new_raster_from_base(1127) INFO filling new raster C:\Users\ajma\Documents\ArcGIS\Projects\230915_InVEST_v1\Output\Output_DeliniateIt\Output_DelineateIt_v4\flow_dir_d8_2023-09-20_11_51_42_vh4__v1y\plateau_distance.tif with 1859113431 – 73.09% complete
2023-09-20 13:52:37,534 (pygeoprocessing.geoprocessing) geoprocessing.new_raster_from_base(1127) INFO filling new raster C:\Users\ajma\Documents\ArcGIS\Projects\230915_InVEST_v1\Output\Output_DeliniateIt\Output_DelineateIt_v4\flow_dir_d8_2023-09-20_11_51_42_vh4__v1y\plateau_distance.tif with 1859113431 – 79.79% complete
2023-09-20 13:52:42,536 (pygeoprocessing.geoprocessing) geoprocessing.new_raster_from_base(1127) INFO filling new raster C:\Users\ajma\Documents\ArcGIS\Projects\230915_InVEST_v1\Output\Output_DeliniateIt\Output_DelineateIt_v4\flow_dir_d8_2023-09-20_11_51_42_vh4__v1y\plateau_distance.tif with 1859113431 – 87.20% complete
2023-09-20 13:52:47,536 (pygeoprocessing.geoprocessing) geoprocessing.new_raster_from_base(1127) INFO filling new raster C:\Users\ajma\Documents\ArcGIS\Projects\230915_InVEST_v1\Output\Output_DeliniateIt\Output_DelineateIt_v4\flow_dir_d8_2023-09-20_11_51_42_vh4__v1y\plateau_distance.tif with 1859113431 – 94.67% complete
2023-09-20 13:52:55,351 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 0 of 1859113431 pixels complete
2023-09-20 13:53:06,027 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 35594496 of 1859113431 pixels complete
2023-09-20 13:53:17,014 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 53391616 of 1859113431 pixels complete
2023-09-20 13:53:28,014 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 71153920 of 1859113431 pixels complete
2023-09-20 13:53:39,063 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 88927744 of 1859113431 pixels complete
2023-09-20 13:53:50,017 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 88982016 of 1859113431 pixels complete
2023-09-20 13:54:01,094 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 106758656 of 1859113431 pixels complete
2023-09-20 13:54:12,109 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 124535808 of 1859113431 pixels complete
2023-09-20 13:54:23,032 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 142319872 of 1859113431 pixels complete
2023-09-20 13:54:34,055 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 160104704 of 1859113431 pixels complete
2023-09-20 13:54:45,045 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 177882112 of 1859113431 pixels complete
2023-09-20 13:54:56,022 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 195652096 of 1859113431 pixels complete
2023-09-20 13:55:07,008 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 213429504 of 1859113431 pixels complete
2023-09-20 13:55:18,004 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 213484032 of 1859113431 pixels complete
2023-09-20 13:55:29,108 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 231253760 of 1859113431 pixels complete
2023-09-20 13:55:40,091 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 249029120 of 1859113431 pixels complete
2023-09-20 13:55:51,019 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 266811904 of 1859113431 pixels complete
2023-09-20 13:56:02,035 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 284600832 of 1859113431 pixels complete
2023-09-20 13:56:13,029 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 302387968 of 1859113431 pixels complete
2023-09-20 13:56:25,081 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 320173056 of 1859113431 pixels complete
2023-09-20 13:56:36,024 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 355706112 of 1859113431 pixels complete
2023-09-20 13:56:47,031 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 373520384 of 1859113431 pixels complete
2023-09-20 13:56:58,048 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 391318528 of 1859113431 pixels complete
2023-09-20 13:57:09,016 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 426872576 of 1859113431 pixels complete
2023-09-20 13:57:20,038 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 444668672 of 1859113431 pixels complete
2023-09-20 13:57:31,041 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 480198400 of 1859113431 pixels complete
2023-09-20 13:57:42,488 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 498002944 of 1859113431 pixels complete
2023-09-20 13:57:53,667 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 515789824 of 1859113431 pixels complete
2023-09-20 13:58:04,049 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 533582080 of 1859113431 pixels complete
2023-09-20 13:58:15,014 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 551378688 of 1859113431 pixels complete
2023-09-20 13:58:26,031 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 586929920 of 1859113431 pixels complete
2023-09-20 13:58:37,051 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 604727296 of 1859113431 pixels complete
2023-09-20 13:58:48,027 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 640263680 of 1859113431 pixels complete
2023-09-20 13:58:59,335 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 658068992 of 1859113431 pixels complete
2023-09-20 13:59:10,057 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 675858944 of 1859113431 pixels complete
2023-09-20 13:59:21,041 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 711399680 of 1859113431 pixels complete
2023-09-20 13:59:32,041 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 729217280 of 1859113431 pixels complete
2023-09-20 13:59:43,060 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 764777984 of 1859113431 pixels complete
2023-09-20 13:59:54,136 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 800342784 of 1859113431 pixels complete
2023-09-20 14:00:05,015 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 835907328 of 1859113431 pixels complete
2023-09-20 14:00:16,047 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 853703680 of 1859113431 pixels complete
2023-09-20 14:00:27,164 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 871485184 of 1859113431 pixels complete
2023-09-20 14:00:38,009 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 907047680 of 1859113431 pixels complete
2023-09-20 14:00:49,131 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 942624512 of 1859113431 pixels complete
2023-09-20 14:01:00,041 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 978208256 of 1859113431 pixels complete
2023-09-20 14:01:11,092 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 1013758720 of 1859113431 pixels complete
2023-09-20 14:01:22,097 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 1031542272 of 1859113431 pixels complete
2023-09-20 14:01:33,011 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 1067100928 of 1859113431 pixels complete
2023-09-20 14:01:44,032 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 1102681600 of 1859113431 pixels complete
2023-09-20 14:01:55,029 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 1138269440 of 1859113431 pixels complete
2023-09-20 14:02:06,004 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 1173839872 of 1859113431 pixels complete
2023-09-20 14:02:17,028 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 1227175680 of 1859113431 pixels complete
2023-09-20 14:02:28,038 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 1262729472 of 1859113431 pixels complete
2023-09-20 14:02:39,033 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 1280527616 of 1859113431 pixels complete
2023-09-20 14:02:50,032 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 1316084992 of 1859113431 pixels complete
2023-09-20 14:03:01,096 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 1351666432 of 1859113431 pixels complete
2023-09-20 14:03:12,010 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 1387220480 of 1859113431 pixels complete
2023-09-20 14:03:23,012 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 1422788608 of 1859113431 pixels complete
2023-09-20 14:03:34,039 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 1440614144 of 1859113431 pixels complete
2023-09-20 14:03:45,012 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 1493928960 of 1859113431 pixels complete
2023-09-20 14:03:56,084 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 1529532416 of 1859113431 pixels complete
2023-09-20 14:04:07,113 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 1582889728 of 1859113431 pixels complete
2023-09-20 14:04:18,000 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): 1760678912 of 1859113431 pixels complete
2023-09-20 14:04:21,047 (pygeoprocessing.routing.routing) Task._call(1093) INFO (flow dir d8): complete

2

Hi @maulanapajie -

Did the run ever finish? It looks like it’s spending a lot of time processing the DEM itself. The log file indicates that it didn’t even start creating watersheds yet, it was still calculating flow direction. This is often the case for large, and/or high-resolution DEMs. I once delineated 30,000 watersheds in Myanmar using DelineateIt, and it took all weekend (in a few batches), but it worked.

~ Stacie

3

Dear Stacie,

Thanks for your reply!

Do you remember (ballpark) how big you Myanmar files were? My DEM is about 650 MB. Based on your suggestion, I will try to run it during the weekend and see how it works out. You also mentioned “in a few batches”. What do you mean by this, and how can I also run DelineateIt in batches?

Regarding my second idea to speed up the process, what do you think? Currently I am using the detect pour points. However, there is the option to choose the watershed outlets from the WWF HydroLakes dataset. Do you think this will make it faster?

Looking forward to your reply.

Best regards,

Maulana

4

Hi @maulanapajie,

It looks like you might be attempting to run DelineateIt on a global DEM. Is that correct? The pour points shapefile you linked to is a global dataset as well. Without going into any detail, DelineateIt will not perform well or at all on a global scale like this.

If you’re looking to do a global NDR analysis I would recommend starting with already available global watersheds. Unless of course there is a distinct need to delineate your own. But here’s a resource to view the different levels of subwatersheds. And here is where I think you can download them.

Once you have those it would be possible to write a script that iterates over each watershed and runs the NDR model on that watershed, given proper parameters and other inputs of course. Setting this type of analysis up is no small endeavor though! You’ll want to make sure you’re properly projecting data into linear units (vs degrees) along with many other things.

Sorry if I’m misinterpreting you goal.

Cheers,

Doug

5

Dear Doug,

Thanks for your reply! Yes, I decided to take or use these existing watersheds from the WWF. So I guess we can close this aspect of the question.

I want to clarify this following statement you mentioned “Once you have those it would be possible to write a script that iterates over each watershed and runs the NDR model on that watershed, given proper parameters and other inputs of course”.

My goal would be to run the NDR model on a global scale, with the hope that I would be able to obtain the NDR outputs on a secondary administrative level scale. For example, water purification for New Hampshire.

That being said, would it not be possible just to choose any watershed level from the WWF, use this as the watershed input for the NDR model, and run InVEST? Given of course that I find global data for all the other inputs that InVEST needs, for example DEM, LULC, etc. Why do I need to write an additional script?

Looking forward to your reply.

Best regards,

Maulana

6

Hi @maulanapajie, I can speak to what’s needed for this kind of analysis having just done one myself!

The problem is that the NDR model requires that your inputs are all in the same projected coordinate system, which is going to be local to a specific part of the world. So you could still run the NDR model on groups of watersheds, but you would need to group them by their coordinate system (such as UTM zones).

@dcdenu4 recommended a script because this is the kind of workflow that a script can really help with. For example, here is a loop (in pseudocode) that would accomplish the task:

For each watershed in the set of global watersheds:
    Determine the correct projected coordinate system for this watershed
    Warp your model inputs so that:
        They are clipped to the bounding box of the watershed
        They are reprojected to the correct projected coordinate system
    Run the model on these warped inputs, summarizing to secondary admin levels.

If you were trying to obtain a global raster layer of water purification, you would also need to stitch the outputs together into a single, global raster.

I will also mention that depending on the resolution of your DEM, global analyses can take a long time. The one I just completed ran at a resolution of 300m and took 260 compute hours to complete (13 hours on 20 CPU cores).

Hope this helps,
James

1 Like
7

Dear @jdouglass ,

Thanks for your reply!

Regarding to the watersheds, I took them from the WWF, chose a specific level (considering that the higher the level, the greater the detail of the watershed. I think they are based on the Pfafstetter Coding System), and then reprojected it to the same projected coordinate system as my DEM. In this regard, I am using the WGS 1984 EASE Grid Global projected system. At least in ArcGIS Pro, it is one of the few projected systems on a global scale (at least that is my understanding). You are right with respect to UTM. If I choose the UTM zones, then I would need to project different rasters depending on the zone.

Thank you also for clarifying what @dcdenu4 mentioned. So the output of my InVEST models are just 1/4 of my modelling journey. They will be used as inputs for other models. In this regard, my aim is to just do “basic” calculations with InVEST. Having a spatial resolution on a km level is also not an issue. As long as there is minimal data gaps in the output rasters and the values are somewhat reasonable, I am good to go. Moreover, what ever results I will have will not be aggregated to a watershed level, but to a country level. This is due to the constraints of the other models I mentioned.

With that in mind, do you think my existing approach makes sense? Meaning:

I avoid the use of RouteDEM and DeliniateIt completely because it takes too much time, while understanding that the watersheds ideally need to be deliniated from the respective DEM. My approach would just be to use an existing watershed so that the model runs.

I use a single raster file for my DEM and single vector file for my watersheds. Both inputs, along with other inputs needed to run the InVEST SDR and NDR models are projected to the WGS 1984 EASE Grid global projected system.

Below are the data sources for the DEM and watersheds.

  1. WWF HydroSHEDS
  2. WWF GlobalBASIN Atlas

I look forward to your comments and inputs! Please let me know if you need additional information or data so that my points can be better conveyed.

Best regards,

Maulana

8

You’re of course welcome to try it out and let us know how it goes if you do! The reason we have used UTM zones for NatCap work is because they are conformal, which preserves angles. This is useful for hydrological routing, as neighboring pixels are weighted with the assumption that all pixels are square. The EASE Grid Global projected system does not appear to be conformal, so you may see some unexpected flow directions because of this. On the other hand, you’re running the model at a fairly coarse resolution, so maybe this won’t be an issue for you at all.

No problems here … you can use whatever watersheds you would like. At a km scale, they will likely match what delineateit would generate anyways. It’s always best if your watersheds match your DEM. I imagine HydroSHEDS and GlobalBASIN match each other, but it’d be good to verify.

I’d say it’s worth trying out! At a 1km resolution, you’re likely to have a fairly long runtime, but not as long as some analyses we’ve done. We generally suggest that you try running the model at an even coarser resolution (for you, I’d suggest 10km or larger) until all data issues are worked out. This way you get get feedback on the setup of your inputs before sinking a lot of computational time into the model. Once you’re confident the model will run, then I’d suggest running it at your target 1km resolution.

Let us know if you have further questions,
James

9

Just a quick addition that there are actually quite a few global projected coordinate systems to choose from. If you’re using ArcGIS Pro, in the Coordinate System window, look at Projected Coordinate System > World. We have used several of these with InVEST, including Eckert and Equal Earth to preserve accurate areas.

~ Stacie

1 Like
10

Dear @swolny thank’s for the addition! Between Eckert and Equal Earth, which of the two would you reccomend, and more importantly why? ChatGPT at least reccomends equal earth :slight_smile: I like to think as we are using InVEST its not about “just” using but also understanding and learning. So for example in a situation like this, if I have the two options, how can I choose one over the other after seeing my results?

11

Dear @jdouglass ,

Thank you so much for your inputs and advice!

Will let you know if there are any more questions for this topic.

Best regards,

Maulana

12

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.