Urban Flood Risk Mitigation - Soil Group Raster preparation error?

Milkiman · June 15, 2020, 9:55am

Hello!

First of all, I know that a similar topic already has been posted (Urban Flood Mitigation Error - ValueError: invalid entry in choice array) whereas a result, I think you did include the error message that I see now.

After initially running my model, the tool aborted after a few seconds with the error message of 'Value Error encountered: Invalid entry in choice array. Check that the Soil Group Raster does not contain values other than (1,2,3,4).

The problem is, that my soil group raster DOES NOT contain any other values than 1-4.

Screencap where you can see the attribute table of the soil raster:

Any idea what the problem could be? Maybe the error lies somewhere else?
I further tried creating a new, artificial soil raster file containing just 1 single soil type for the whole area of interest (only containing value 3, just like the example data), but ended up getting the same error nonetheless, so I suspect the problem might be a different one?

Log details:

Initializing…
Showing messages with level INFO and higher
2020-06-15 11:53:34,886 utils.prepare_workspace(111) INFO Writing log messages to D:/InVEST/The Hague Runs/Flooding/Run/Test1\InVEST-UrbanFloodRiskMitigation-log-2020-06-15–11_53_34.txt
2020-06-15 11:53:34,887 model._logged_target(1629) Level 100 Starting model with parameters:
Arguments for InVEST natcap.invest.urban_flood_risk_mitigation 3.8.2:
aoi_watersheds_path D:/InVEST/The Hague Runs/Flooding/Data/MiniHague.shp
built_infrastructure_vector_path
curve_number_table_path D:/InVEST/The Hague Runs/Flooding/Data/Biophysical Table.csv
infrastructure_damage_loss_table_path
lulc_path D:/InVEST/The Hague Runs/Flooding/Data/LULC_10x10_Hague.tif
n_workers -1
rainfall_depth 8
results_suffix test
soils_hydrological_group_raster_path D:/InVEST/The Hague Runs/Flooding/Data/Soilgroups_NA.tif
workspace_dir D:/InVEST/The Hague Runs/Flooding/Run/Test1

2020-06-15 11:53:35,006 geoprocessing.raster_calculator(375) INFO starting stats_worker
2020-06-15 11:53:35,007 geoprocessing.raster_calculator(381) INFO started stats_worker <Thread(Thread-15, started daemon 16624)>
2020-06-15 11:53:35,032 geoprocessing.raster_calculator(486) INFO Waiting for raster stats worker result.
2020-06-15 11:53:35,033 threading.run(870) WARNING No valid pixels were received, sending None.
2020-06-15 11:53:35,033 Task.add_task(626) ERROR Something went wrong when adding task create cn raster (1), terminating taskgraph.
Traceback (most recent call last):
File “site-packages\natcap\invest\urban_flood_risk_mitigation.py”, line 828, in _lu_to_cn_op
File “<array_function internals>”, line 6, in choose
File “site-packages\numpy\core\fromnumeric.py”, line 433, in choose
File “site-packages\numpy\core\fromnumeric.py”, line 61, in _wrapfunc
ValueError: invalid entry in choice array

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “site-packages\taskgraph\Task.py”, line 592, in add_task
File “site-packages\taskgraph\Task.py”, line 1069, in _call
File “site-packages\pygeoprocessing\geoprocessing.py”, line 424, in raster_calculator
File “site-packages\natcap\invest\urban_flood_risk_mitigation.py”, line 832, in _lu_to_cn_op
ValueError: invalid entry in choice array
Check that the Soil Group raster does not contain values other than (1, 2, 3, 4)
2020-06-15 11:53:35,034 model._logged_target(1635) ERROR Exception while executing <function execute at 0x0FFA1C00>
Traceback (most recent call last):
File “site-packages\natcap\invest\urban_flood_risk_mitigation.py”, line 828, in _lu_to_cn_op
File “<array_function internals>”, line 6, in choose
File “site-packages\numpy\core\fromnumeric.py”, line 433, in choose
File “site-packages\numpy\core\fromnumeric.py”, line 61, in _wrapfunc
ValueError: invalid entry in choice array

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “site-packages\natcap\invest\ui\model.py”, line 1632, in _logged_target
File “site-packages\natcap\invest\urban_flood_risk_mitigation.py”, line 248, in execute
File “site-packages\taskgraph\Task.py”, line 592, in add_task
File “site-packages\taskgraph\Task.py”, line 1069, in _call
File “site-packages\pygeoprocessing\geoprocessing.py”, line 424, in raster_calculator
File “site-packages\natcap\invest\urban_flood_risk_mitigation.py”, line 832, in _lu_to_cn_op
ValueError: invalid entry in choice array
Check that the Soil Group raster does not contain values other than (1, 2, 3, 4)
2020-06-15 11:53:35,035 model._logged_target(1638) INFO Execution finished
2020-06-15 11:53:35,041 utils.prepare_workspace(117) INFO Elapsed time: 0.16s
2020-06-15 11:53:35,058 execution.run(78) ERROR Target <function InVESTModel.execute_model.._logged_target at 0x01695A98> failed with exception
Traceback (most recent call last):
File “site-packages\natcap\invest\urban_flood_risk_mitigation.py”, line 828, in _lu_to_cn_op
File “<array_function internals>”, line 6, in choose
File “site-packages\numpy\core\fromnumeric.py”, line 433, in choose
File “site-packages\numpy\core\fromnumeric.py”, line 61, in _wrapfunc
ValueError: invalid entry in choice array

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “site-packages\natcap\invest\ui\execution.py”, line 68, in run
File “site-packages\natcap\invest\ui\model.py”, line 1632, in _logged_target
File “site-packages\natcap\invest\urban_flood_risk_mitigation.py”, line 248, in execute
File “site-packages\taskgraph\Task.py”, line 592, in add_task
File “site-packages\taskgraph\Task.py”, line 1069, in _call
File “site-packages\pygeoprocessing\geoprocessing.py”, line 424, in raster_calculator
File “site-packages\natcap\invest\urban_flood_risk_mitigation.py”, line 832, in _lu_to_cn_op
ValueError: invalid entry in choice array
Check that the Soil Group raster does not contain values other than (1, 2, 3, 4)
2020-06-15 11:53:35,063 execution.run(83) INFO Execution finished

Thank you!

dcdenu4 · June 15, 2020, 7:45pm

Hi @Milkiman

Thanks for posting and sorry you’re having trouble here. Would you be willing to share the LULC and Soil Group Raster inputs you are using? If you prefer not to upload here you can use a link to a shared drive or email me at ddenu@stanford.edu.

Doug

Milkiman · June 16, 2020, 1:00pm

Hey, thanks for your help!

Yeah no worries, I will upload it here.

LULC_10x10_Hague.tif (2.0 MB)

Soilgroups_1_3_4.tif (1.0 MB)

dcdenu4 · June 16, 2020, 2:57pm

Thanks @Milkiman,

I don’t see anything that would cause concern, could you provide the rest of your inputs for the model run? This could be a bug and running the model with your inputs would really help find it.

Thanks!

Milkiman · June 16, 2020, 4:17pm

Oh weird!

Yes, so the watershed vector is just a placeholder shape file, as I yet have to finalize the actual watershed, but the final file should be somewhat similar in its extent/form…

The CSV file is already final and DOES work with the provided example data (as I CAN successfully run the example model with my own csv file), so I do not expect the error to be in there.

Watershed_ex.zip (2.0 KB)

Biophysical Table.csv (1.2 KB)

If you can successfully run the model with all of my data, the problem might be on my PC (although I did try with InVEST version 3.8.2 and 3.8.4, with a fresh install already, but same error always)

Thanks!

dcdenu4 · June 24, 2020, 1:46pm

Hi @Milkiman

Sorry for the delay here. It looks like there is a bug that we’ll need to fix. In the meantime can you try setting the nodata value on your Soilgroups_1_3_4 raster to something other than 0? Maybe 7?

Please try that with InVEST 3.8.4 and let me know if it works.

Doug

hcambridge · June 24, 2020, 2:42pm

Thanks for your help. I reclassified the soils group layer Class 0–>7 and managed to get it to work.
Howard

Milkiman · June 25, 2020, 12:42pm

Thank you so far, good to know!

As of now, if I reclassify the nodata values to 7 using f.ex. the reclassify tool in ArcMap or using the raster calculator, I still get the same error as usual. Only if I assign nodata to either 1,2,3 or 4, I manage to get the model to run successfully.

For now I can work with that, but if anybody has any other input, feel free to comment. Maybe I need to reclassify the data in another manner, considering @hcambridge got it to run with the value 7? What spatial analysis software are you using?

hcambridge · June 25, 2020, 12:57pm

I used QGIS to reclassify @Milkiman .
Howard

dcdenu4 · June 26, 2020, 3:00pm

@Milkiman,

I’m happy to take a look at your reclassified soil data raster that’s still not working for you, to make sure there’s not something I overlooked.

Doug

Milkiman · June 29, 2020, 5:43pm

@dcdenu4

I finally got it to work now by reclassifying all NoData values of [Soilgroups_1_3_4] first to some number (e.g. 7), and then reclassifying all 7 values back to NoData (All with the reclassifying tool of ArcMap). I am not sure on why it works now, but I will upload the file so you may be able to find the reason for further troubleshooting…

Soilgroups_1_3_4_reclass.tif (2.0 MB)

DGowtham · August 11, 2021, 12:25pm

hi sir,
I am also working on Urban flood mitigation model and facing a similar problem.

my log file is

2021-08-11 17:30:39,384 utils.prepare_workspace(123) INFO Writing log messages to E:/M.Arch Urban Regeneration/Semester - 02/MAR - 257 Dissertation - 2/Diss_Invest/UrbanFloodMitigationModel/Result1108\InVEST-UrbanFloodRiskMitigation-log-2021-08-11–17_30_39.txt
2021-08-11 17:30:39,393 model._logged_target(1646) Level 100 Starting model with parameters:
Arguments for InVEST natcap.invest.urban_flood_risk_mitigation 3.9.0:
aoi_watersheds_path E:/M.Arch Urban Regeneration/Semester - 02/MAR - 257 Dissertation - 2/Diss_Invest/RouteDEM/Vector_Watershed.shp
built_infrastructure_vector_path
curve_number_table_path E:/M.Arch Urban Regeneration/Semester - 02/MAR - 257 Dissertation - 2/Diss_Invest/UrbanFloodMitigationModel/Bisphysical table.csv
infrastructure_damage_loss_table_path
lulc_path E:/M.Arch Urban Regeneration/Semester - 02/MAR - 257 Dissertation - 2/Diss_Invest/UrbanFloodMitigationModel/LULC_Invest.tif
n_workers -1
rainfall_depth 1200
results_suffix UFMM_1108
soils_hydrological_group_raster_path E:/M.Arch Urban Regeneration/Semester - 02/MAR - 257 Dissertation - 2/Diss_Invest/UrbanFloodMitigationModel/Soilgroup_ProjectRaster2.tif
workspace_dir E:/M.Arch Urban Regeneration/Semester - 02/MAR - 257 Dissertation - 2/Diss_Invest/UrbanFloodMitigationModel/Result1108

2021-08-11 17:30:39,398 validation._wrapped_validate_func(911) DEBUG Using ARG_SPEC for validation
2021-08-11 17:30:39,398 validation._wrapped_validate_func(915) INFO Starting whole-model validation with ARGS_SPEC
2021-08-11 17:30:39,401 validation.timeout(625) DEBUG Starting file checking thread with timeout=5
2021-08-11 17:30:39,472 validation.timeout(637) DEBUG File checking thread completed.
2021-08-11 17:30:39,473 validation.timeout(625) DEBUG Starting file checking thread with timeout=5
2021-08-11 17:30:39,757 validation.timeout(637) DEBUG File checking thread completed.
2021-08-11 17:30:39,759 validation.timeout(625) DEBUG Starting file checking thread with timeout=5
2021-08-11 17:30:40,027 validation.timeout(637) DEBUG File checking thread completed.
2021-08-11 17:30:40,028 validation.timeout(625) DEBUG Starting file checking thread with timeout=5
2021-08-11 17:30:40,163 validation.timeout(637) DEBUG File checking thread completed.
2021-08-11 17:30:40,164 validation.timeout(625) DEBUG Starting file checking thread with timeout=5
2021-08-11 17:30:40,179 validation.timeout(637) DEBUG File checking thread completed.
2021-08-11 17:30:40,558 validation._wrapped_validate_func(962) DEBUG Validation warnings: []
2021-08-11 17:30:42,087 geoprocessing.align_and_resize_raster_stack(795) INFO 1 of 2 aligned: aligned_lulc_UFMM_1108.tif
2021-08-11 17:30:42,296 geoprocessing.align_and_resize_raster_stack(795) INFO 2 of 2 aligned: aligned_soils_hydrological_group_UFMM_1108.tif
2021-08-11 17:30:42,297 geoprocessing.align_and_resize_raster_stack(797) INFO aligned all 2 rasters.
2021-08-11 17:30:42,647 utils.build_lookup_from_csv(528) WARNING Empty or NaN values were found in the table: E:/M.Arch Urban Regeneration/Semester - 02/MAR - 257 Dissertation - 2/Diss_Invest/UrbanFloodMitigationModel/Bisphysical table.csv.
2021-08-11 17:30:42,676 utils.build_lookup_from_csv(535) WARNING Encountered an entirely blank row on line(s) [nan, nan, nan, nan, nan]. Dropping rows from table.
2021-08-11 17:30:42,978 geoprocessing.raster_calculator(391) INFO starting stats_worker
2021-08-11 17:30:43,016 threading.run(870) DEBUG stats worker PID: 10644
2021-08-11 17:30:43,015 geoprocessing.raster_calculator(397) INFO started stats_worker <Thread(Thread-154, started daemon 12732)>
2021-08-11 17:30:43,391 geoprocessing.raster_calculator(507) INFO Waiting for raster stats worker result.
2021-08-11 17:30:43,404 threading.run(870) WARNING No valid pixels were received, sending None.
2021-08-11 17:30:43,406 Task.add_task(781) ERROR Something went wrong when adding task create Curve Number raster (1), terminating taskgraph.
Traceback (most recent call last):
File “site-packages\natcap\invest\urban_flood_risk_mitigation.py”, line 789, in _lu_to_cn_op
File “<array_function internals>”, line 6, in choose
File “site-packages\numpy\core\fromnumeric.py”, line 430, in choose
File “site-packages\numpy\core\fromnumeric.py”, line 58, in _wrapfunc
ValueError: invalid entry in choice array

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “site-packages\taskgraph\Task.py”, line 747, in add_task
File “site-packages\taskgraph\Task.py”, line 1234, in _call
File “site-packages\pygeoprocessing\geoprocessing.py”, line 438, in raster_calculator
File “site-packages\natcap\invest\urban_flood_risk_mitigation.py”, line 794, in _lu_to_cn_op
ValueError: invalid entry in choice array
Check that the Soil Group raster does not contain values other than (1, 2, 3, 4)
2021-08-11 17:30:43,431 model._logged_target(1652) ERROR Exception while executing <function execute at 0x0000016EE2677CA8>
Traceback (most recent call last):
File “site-packages\natcap\invest\urban_flood_risk_mitigation.py”, line 789, in _lu_to_cn_op
File “<array_function internals>”, line 6, in choose
File “site-packages\numpy\core\fromnumeric.py”, line 430, in choose
File “site-packages\numpy\core\fromnumeric.py”, line 58, in _wrapfunc
ValueError: invalid entry in choice array

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “site-packages\natcap\invest\ui\model.py”, line 1649, in _logged_target
File “site-packages\natcap\invest\urban_flood_risk_mitigation.py”, line 241, in execute
File “site-packages\taskgraph\Task.py”, line 747, in add_task
File “site-packages\taskgraph\Task.py”, line 1234, in _call
File “site-packages\pygeoprocessing\geoprocessing.py”, line 438, in raster_calculator
File “site-packages\natcap\invest\urban_flood_risk_mitigation.py”, line 794, in _lu_to_cn_op
ValueError: invalid entry in choice array
Check that the Soil Group raster does not contain values other than (1, 2, 3, 4)
2021-08-11 17:30:43,441 model._logged_target(1655) INFO Execution finished
2021-08-11 17:30:43,469 utils.prepare_workspace(129) INFO Elapsed time: 4.08s

Bisphysical table.csv (190 Bytes)
LULC_Invest.tif (194.8 KB)
Soilgroup_ProjectRaster2.tif (3.4 KB)

And also please explain about setting no data value soil group?

Thanks in advance
Regards,
Gowtham.

dcdenu4 · August 11, 2021, 1:38pm

Hi @DGowtham ,

It looks like your soil group raster has values outside of the allowed range. The soil group raster must have values of 1, 2, 3, 4 only. The rest should be nodata. Per the users guide:

Soils Hydrological Group Raster (required). Raster of categorical hydrological groups. Pixel values must be limited to 1, 2, 3, or 4, which correspond to soil hydrologic group A, B, C, or D, respectively (used to derive the CN number)

https://storage.googleapis.com/releases.naturalcapitalproject.org/invest-userguide/latest/urban_flood_mitigation.html#data-needs

Under the Appendix of the User Guide section there will be quite a bit of useful information regarding the soil group raster:
https://storage.googleapis.com/releases.naturalcapitalproject.org/invest-userguide/latest/urban_flood_mitigation.html#appendix-data-sources-and-guidance-for-parameter-selection

Hopefully this is helpful,

Doug

DGowtham · August 12, 2021, 1:48pm

Hi sir,
Thanks for your quick reply. I downloaded and cliped the soil raster from HYSOGs250m this layer only. Here the pixel value of soil raster is coming between 3 to 14 as u see from my attachement. I dont know how to change that value and run the model. Please help me out.

Regards,
Gowtham.

dcdenu4 · August 12, 2021, 3:11pm

Hi @DGowtham ,

I was able to find this resource for the HYSOGs250m raster:
https://daac.ornl.gov/SOILS/guides/Global_Hydrologic_Soil_Group.html

In this document they have a table for what the values relate to:

From this, I think that our guidance from the UG can be used.

There’s also this post from @swolny about converting soil groups for the Seasonal Water Yield model: Urban Flood Risk Mitigation model - converting dual soil-groups (e.g., A/D) to 1-4

Cheers,

Doug

DGowtham · August 14, 2021, 5:26pm

Thanks for ur response sir. It helped me a lot.

Topic		Replies	Views
Urban Flood Mitigation Error - ValueError: invalid entry in choice array NatCap Software Support urban-flood	8	1263	June 26, 2020
Urban flood risk mitigation, soil group raster error? NatCap Software Support urban-flood	5	425	May 5, 2023
Urban Flood Risk Mitigation Model Problem in Soil Group Raster NatCap Software Support urban-flood	7	421	December 22, 2022
Urban flood risk mitigation model error NatCap Software Support urban-flood	2	180	September 28, 2023
Urban Flood Risk Mitigation (v. 3.9.0) TypeError NatCap Software Support urban-flood	6	331	June 1, 2021

Urban Flood Risk Mitigation - Soil Group Raster preparation error?

Related Topics