Debugging all nan in n export output

Software Support
4

Okay, so here’s what I know so far…

In my environment that has natcap-invest 3.14.0, gdal 3.6.4, and libgdal 3.7.0, and calling

ndr.execute({
    'workspace_dir': './local_data_test_1/',
    'dem_path': dem_path,
    'lulc_path': lucode_raster_path,
    'runoff_proxy_path': runoff_proxy_path,
    'watersheds_path': watersheds_path,
    'biophysical_table_path': biophysical_table_path,
    'calc_p': False,
    'calc_n': True,
    'results_suffix': f"test_1",
    'threshold_flow_accumulation': 1000,
    'k_param': 2,
    'subsurface_critical_length_n': 5,
    'subsurface_eff_n': 0.5,
    'n_workers': -1,
})

i.e. with n_workers set to -1 on @dcdenu4 's suggestion on this thread, I get

Warning 1: the input vector layer has a SRS, but the source raster dataset does not.
Cutline results may be incorrect.

which, with hindsight, is just a warning. I tracked this down to my runoff proxy raster missing a defined CRS. Adding that metadata, ndr.execute runs without error or warning in some 2.6 s. However, the watershed_results_ndr gpkg output has nan in the n export columns. These are the outputs that motivated this post, with ic_factor being produced but ndr_n not. On @esoth 's suggestion (above) I looked at the effective_retention_n intermediate output as well. I’ve now highlighted where these two outputs have nan:

ndr_3_14_0_nans
ndr_3_14_0_nans1043×394 51.3 KB

The red boundaries are the subcatchments I’ve specified to be of interest. (The black boundary you can partially see is where I’d defined input LULC). Clearly the ic_factor has nan along the stream network, but also around the uppermost reaches around the perimeter of the watershed. The effective_retention_n intermediate, however, is riddled with nan. (NB these plots have 1 where data is nan, 0 otherwise).

Now I switch back to my environment that has natcap-invest 3.12.1, gdal 3.6.2, and libgdal 3.6.2 installed and run again. Now, instead of the warning about a source raster missing an SRS, I get

ERROR 1: ./local_data_test_1/intermediate_outputs/ic_factor_test_1.tif, band 1: Failed to compute statistics, no valid pixels found in sampling.

(NB this is when giving it input that is missing a defined SRS in an input raster).

So now “fixing” the missing SRS issue, I get the same ERROR 1 again after the model runs for 3+ minutes. Differences in output with this version include that d_dn, dist_to_channel, ic_factor, as well as ndr_n are all nan. The export columns in the watershed_results_ndr gpkg file are now zero rather than nan. The effective_retention_n intermediate output is also entirely nan now.

Whatever the issue with my inputs, they cause quite different degrees of problem between these two versions of invest. I think it’s time to beg that help from Doug and will package up my inputs, if that’s still okay. I’ve about run out of “ah ha, I think it will be … oh, it wasn’t that …”