It looks like InVEST is still having problems with 8 bit signed rasters in version 3.8.9. This time we’re using Seasonal Water Yield and running into two related issues.
1/ The climate zone raster is 8 bit signed, with a NoData value of -128, which led to this error:
ValueError: The following 1 raster values [128] from “D:/Colombia/Golfo_de_Morrosquillo/output_CV_tmp\cache_dir\cz_aligned_t1.tif” do not have corresponding entries in the value_map: {11: 10.1, 12: 10.0, 14: 3.7}
Adding 128 to the climate zone rain events table (with rain event values of 0) got me past that error.
2/ Next came this one:
File “site-packages\natcap\invest\seasonal_water_yield\seasonal_water_yield.py”, line 1039, in _calculate_curve_number_raster
File “site-packages\pygeoprocessing\geoprocessing.py”, line 424, in raster_calculator
File “site-packages\natcap\invest\seasonal_water_yield\seasonal_water_yield.py”, line 1029, in cn_op
KeyError: 128
This seems to be caused by the soil group raster, which also is type 8 bit signed with NoData = -128. I tried adding 128 to the biophysical table, but got the same error. So converted the soil group raster to 32 bit signed, with a NoData value of -2147483647 and it ran fine.
Attached are both of the related log files, I can provide the data if it’s useful. Thanks!
Is it possible to share this data so I can replicate and work through these bugs you’re seeing? I can’t help but feel that we recently made fixes to some of these, but obviously not!
Thanks for sharing the data. One thing that I just double checked was the fact that GDAL Byte is unsigned (values 0 - 255). Looking at the soils and climate raster in QGIS, I see that they are both Byte - Eight bit unsigned integer and that nodata is not defined. pygeoprocessing shows nodata as set to -128 and QGIS shows the +128 values that should be negative nodata in the rasters. So it seems like this is where a lot of the mix up is happening. The GDAL dataset ( or datasource ) is happy to set nodata as -128, but when writing that data, it transforms it to +128 (I’m guessing).
In an interesting turn of events I ran the data on both the 3.8.9 and development 3.9 build only to find that the 3.9 dev build completes without fault. I’m going to spend a little time figuring out why that’s the case.
BUT, as a work around for the problems you’re seeing, if you define those raster to have nodata values that are in the 0 - 255 range you should be good. You can also change the raster types to int16 or another signed type.
I’m also just realizing you already got to the work around part and were just informing us of the 8 bit raster type issue! So, perhaps it’s not an issue at all since GDAL Byte is unsigned? Though, I am curious how our dev 3.9 build is seemingly handling the issue.
Update: @swolny
Just to follow up, it looks like Pygeoprocessing 2.0 fixed the unsigned / signed byte issue. This will be reflected in the 3.9 release, currently 3.8.9 uses Pygeoprocessing < 2.0.