Urban Flood Mitigation IndexError: index out of range

Software Support
1

Hi, I just started trying to run the Urban Flood Mitigation model today. I have my curve number table, HSG raster, and land use raster (no infrastructure data). I got the following errors after the rasters were aligned, during the curve number task: (IndexError: index (128) out of range)
2020-04-07 16:30:44,673 utils.prepare_workspace(115) INFO Writing log messages to C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\InVEST-UrbanFloodRiskMitigation-log-2020-04-07–16_30_44.txt
2020-04-07 16:30:44,674 model._logged_target(1642) Level 100 Starting model with parameters:
Arguments for InVEST natcap.invest.urban_flood_risk_mitigation 3.8.0:
aoi_watersheds_path C:/ESRI/GISDATA/Dakotas/BigSioux/hydrography/huc04_bs.shp
built_infrastructure_vector_path
curve_number_table_path C:/InVEST_3.7.0_x86/seasonal_water_yield/curvenumber.csv
infrastructure_damage_loss_table_path
lulc_path C:/ESRI/GISDATA/Dakotas/BigSioux/LULC/nlcd2016utm.tif
n_workers -1
rainfall_depth 25
results_suffix 1
soils_hydrological_group_raster_path C:/ESRI/GISDATA/Dakotas/BigSioux/Soils/HSG_utm.tif
workspace_dir C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace

2020-04-07 16:30:54,142 geoprocessing.logger_callback(3239) INFO Warp 2.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:31:03,642 geoprocessing.logger_callback(3239) INFO Warp 4.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:31:14,284 geoprocessing.logger_callback(3239) INFO Warp 6.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:31:20,068 geoprocessing.logger_callback(3239) INFO Warp 7.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:31:25,875 geoprocessing.logger_callback(3239) INFO Warp 8.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:31:36,357 geoprocessing.logger_callback(3239) INFO Warp 10.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:31:45,500 geoprocessing.logger_callback(3239) INFO Warp 12.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:31:54,600 geoprocessing.logger_callback(3239) INFO Warp 14.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:32:04,021 geoprocessing.logger_callback(3239) INFO Warp 16.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:32:13,380 geoprocessing.logger_callback(3239) INFO Warp 18.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:32:22,724 geoprocessing.logger_callback(3239) INFO Warp 20.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:32:31,948 geoprocessing.logger_callback(3239) INFO Warp 22.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:32:41,398 geoprocessing.logger_callback(3239) INFO Warp 24.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:32:50,750 geoprocessing.logger_callback(3239) INFO Warp 26.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:33:00,039 geoprocessing.logger_callback(3239) INFO Warp 28.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:33:09,158 geoprocessing.logger_callback(3239) INFO Warp 30.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:33:18,278 geoprocessing.logger_callback(3239) INFO Warp 32.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:33:27,530 geoprocessing.logger_callback(3239) INFO Warp 34.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:33:36,712 geoprocessing.logger_callback(3239) INFO Warp 36.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:33:45,988 geoprocessing.logger_callback(3239) INFO Warp 38.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:33:55,240 geoprocessing.logger_callback(3239) INFO Warp 40.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:34:00,518 geoprocessing.logger_callback(3239) INFO Warp 41.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:34:10,108 geoprocessing.logger_callback(3239) INFO Warp 43.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:34:19,111 geoprocessing.logger_callback(3239) INFO Warp 45.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:34:28,440 geoprocessing.logger_callback(3239) INFO Warp 47.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:34:37,364 geoprocessing.logger_callback(3239) INFO Warp 49.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:34:46,787 geoprocessing.logger_callback(3239) INFO Warp 51.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:34:56,039 geoprocessing.logger_callback(3239) INFO Warp 53.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:35:05,209 geoprocessing.logger_callback(3239) INFO Warp 55.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:35:14,230 geoprocessing.logger_callback(3239) INFO Warp 57.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:35:23,418 geoprocessing.logger_callback(3239) INFO Warp 59.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:35:32,769 geoprocessing.logger_callback(3239) INFO Warp 61.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:35:41,922 geoprocessing.logger_callback(3239) INFO Warp 63.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:35:50,929 geoprocessing.logger_callback(3239) INFO Warp 65.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:36:00,043 geoprocessing.logger_callback(3239) INFO Warp 67.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:36:09,090 geoprocessing.logger_callback(3239) INFO Warp 69.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:36:18,206 geoprocessing.logger_callback(3239) INFO Warp 71.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:36:27,266 geoprocessing.logger_callback(3239) INFO Warp 73.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:36:36,527 geoprocessing.logger_callback(3239) INFO Warp 75.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:36:45,609 geoprocessing.logger_callback(3239) INFO Warp 77.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:36:54,646 geoprocessing.logger_callback(3239) INFO Warp 79.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:37:03,715 geoprocessing.logger_callback(3239) INFO Warp 81.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:37:13,306 geoprocessing.logger_callback(3239) INFO Warp 83.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:37:22,941 geoprocessing.logger_callback(3239) INFO Warp 85.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:37:32,530 geoprocessing.logger_callback(3239) INFO Warp 87.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:37:41,312 geoprocessing.logger_callback(3239) INFO Warp 89.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:37:49,675 geoprocessing.logger_callback(3239) INFO Warp 91.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:37:58,039 geoprocessing.logger_callback(3239) INFO Warp 93.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:38:06,442 geoprocessing.logger_callback(3239) INFO Warp 95.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:38:14,792 geoprocessing.logger_callback(3239) INFO Warp 97.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:38:23,249 geoprocessing.logger_callback(3239) INFO Warp 99.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:38:27,427 geoprocessing.logger_callback(3239) INFO Warp 100.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_lulc_1.tif
2020-04-07 16:38:27,793 geoprocessing.align_and_resize_raster_stack(752) INFO 1 of 2 aligned: aligned_lulc_1.tif
2020-04-07 16:38:32,885 geoprocessing.logger_callback(3239) INFO Warp 63.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_soils_hydrological_group_1.tif
2020-04-07 16:38:35,540 geoprocessing.logger_callback(3239) INFO Warp 100.0% complete C:\Users\KBLANN\Documents\urban_flood_risk_mitigation_workspace\temp_working_dir_not_for_humans\aligned_soils_hydrological_group_1.tif
2020-04-07 16:38:36,012 geoprocessing.align_and_resize_raster_stack(752) INFO 2 of 2 aligned: aligned_soils_hydrological_group_1.tif
2020-04-07 16:38:36,012 geoprocessing.align_and_resize_raster_stack(754) INFO aligned all 2 rasters.
2020-04-07 16:38:36,078 geoprocessing.raster_calculator(375) INFO starting stats_worker
2020-04-07 16:38:36,078 geoprocessing.raster_calculator(381) INFO started stats_worker <Thread(Thread-2, started daemon 3040)>
2020-04-07 16:38:36,094 threading.run(870) DEBUG payload is None, terminating
2020-04-07 16:38:36,094 geoprocessing.raster_calculator(486) INFO Waiting for raster stats worker result.
2020-04-07 16:38:36,094 threading.run(870) WARNING No valid pixels were received, sending None.
2020-04-07 16:38:36,094 Task.add_task(641) ERROR Something went wrong when adding task create cn raster (1), terminating taskgraph.
Traceback (most recent call last):
File “site-packages\taskgraph\Task.py”, line 607, in add_task
File “site-packages\taskgraph\Task.py”, line 1071, in _call
File “site-packages\pygeoprocessing\geoprocessing.py”, line 424, in raster_calculator
File “site-packages\natcap\invest\urban_flood_risk_mitigation.py”, line 816, in _lu_to_cn_op
File “site-packages\scipy\sparse_index.py”, line 35, in getitem
File “site-packages\scipy\sparse_index.py”, line 139, in _validate_indices
File “site-packages\scipy\sparse_index.py”, line 171, in _asindices
IndexError: index (128) out of range
2020-04-07 16:38:36,094 model._logged_target(1648) ERROR Exception while executing <function execute at 0x0DF3A9C0>
Traceback (most recent call last):
File “site-packages\natcap\invest\ui\model.py”, line 1645, in _logged_target
File “site-packages\natcap\invest\urban_flood_risk_mitigation.py”, line 250, in execute
File “site-packages\taskgraph\Task.py”, line 607, in add_task
File “site-packages\taskgraph\Task.py”, line 1071, in _call
File “site-packages\pygeoprocessing\geoprocessing.py”, line 424, in raster_calculator
File “site-packages\natcap\invest\urban_flood_risk_mitigation.py”, line 816, in _lu_to_cn_op
File “site-packages\scipy\sparse_index.py”, line 35, in getitem
File “site-packages\scipy\sparse_index.py”, line 139, in _validate_indices
File “site-packages\scipy\sparse_index.py”, line 171, in _asindices
IndexError: index (128) out of range
2020-04-07 16:38:36,094 model._logged_target(1651) INFO Execution finished
2020-04-07 16:38:36,094 utils.prepare_workspace(121) INFO Elapsed time: 7m 51.420000000000016s

2

Hi @kblann, Based on the error, it looks like your landcover raster has some pixel values of 128, and the value of 128 isn’t in your Curve Number table (curvenumber.csv). This sometimes happens with signed byte rasters in ArcGIS, or else it’s possible that the nodata value of the raster is not set. Could you check that the nodata value is set to the correct pixel value?

Let us know if that doesn’t do the trick and we can keep troubleshooting!
James

3

Thanks!

My raster attribute table did not indicate any pixel values of 128 and I wasn’t sure how to set the nodata value without seeing it in the table. So instead I added a “no data” row to my curve number table for pixel values of 128 (curve number 0 for all soil types). And that seemed to do the trick.

While I’ve got your “attention”, though–I’m sort of curious if you ever plan to add any flow accumulation / routing to this model, to model where the runoff is most likely to end up causing flooding? Thanks in advance…

4

@kblann adding 0 for 128 sounds like a good workaround!

I’m not aware of any plans to add routing to Urban Flood Risk Mitigation at this time. Maybe @RafaSchmitt would be able to chime in here?

5

Hi
I’m having the same problem with my Soil data in the UFM model.

ValueError: invalid entry in choice array
Check that the Soil Group raster does not contain values other than (1, 2, 3, 4)

Analysis of my raster shows just two classes with NODATA set to 0. Values are Unsigned Integers. (UInt16).

Biophysical Parameters (lucode, CNs) as follows:

lucode,CN_A,CN_B,CN_C,CN_D
0,30,30,30,30
1,30,30,57,63
2,30,30,74,85
3,30,30,79,84
4,30,30,85,89
5,30,30,93,95
6,30,30,47,55
8,91,91,91,94
9,90,90,90,92
10,100,100,100,100
11,100,100,100,100

Any other suggestions as to what I can do.
Thanks

6

Hi @hcambridge,

Thanks for posting with this issue. Do you think you could attach the logfile here? It would also be helpful if you could attach or share the soil group raster as well. Usually this error occurs because the soil raster has a rogue value outside of the { 1, 2, 3, 4 } set.

Best,

Doug

7

Thanks for taking a look.
Howard

InVEST-UrbanFloodRiskMitigation-log-2020-06-24–13_35_42.txt (19.8 KB)

sdat_1566_1_20200615_090645355-NAKURU-reproj3.tif (56.3 KB)

8

Hi @hcambridge

I think you have the same issue as this user: Urban Flood Risk Mitigation - Soil Group Raster preparation error? - #6 by dcdenu4.

Sorry for the delay here. It looks like there is a bug that we’ll need to fix. In the meantime can you try setting the nodata value on your Soilgroups_1_3_4 raster to something other than 0 ? Maybe 7 ?
Please try that with InVEST 3.8.4 and let me know if it works.

Could you also try this and report back?

Thanks,

Doug

9

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.