NDR Model -- fill pits creates filled DEM with one value

[CarolynK]

What is the issue or question you have?

I am trying to run the NDR model in InVEST, but it ultimately fails by returning this error code: “ValueError: cannot convert float NaN to integer.” By going through the intermediate outputs, my best guess is that it fails because the “fill pits” procedure creates a filled DEM with basically one elevation value, even though the terrain should have a varied range. Attached are summary tables of the intermediate DEMs output by the NDR model (I will try to attach images below). The first is “aligned_dem,” which looks as expected and has a varied range. The next output, “filled_dem,” does not reflect the terrain of the aligned DEM, with most pixels assigned a value of 2124.

Summary of data:

 aligned_dem
 Min.   : 663.0             
 1st Qu.: 817.0             
 Median : 899.0             
 Mean   : 924.4             
 3rd Qu.: 968.0             
 Max.   :2083.0             
 NA's   :41671

 filled_dem
 Min.   : 693              
 1st Qu.:2124              
 Median :2124              
 Mean   :2123              
 3rd Qu.:2124              
 Max.   :2124              
 NA's   :37845

Any suggestions on why this might be happening and how I can correct it would be greatly appreciated!

What do you expect to happen?

I expect the filled DEM to have a similar range of values to the aligned DEM. I believe the NDR model is failing to make streams because of the uniformity of the filled DEM.

What have you tried so far?

I have tried running this model with a larger and smaller (shown here) spatial range. I’ve tried coarse (1km) and finer (100m, shown here) resolution inputs. I’ve tried an unfilled DEM, a filled DEM (using InVEST’s DelineateIt tool), a pre-processed filled DEM (from WWF Hydrosheds Hydrologically Conditioned DEMs, shown here), and burning lakes/rivers into the DEM as NAs using my land use classification (shown here). All these had similar intermediate outputs to the example shown here.

Attach the logfile here:

InVEST-natcap.invest.ndr.ndr-log-2023-01-10–09_33_57.txt (11.7 KB)

[CarolynK]

Aligned DEM:

[CarolynK]

Filled DEM:

[swolny]

Hello @CarolynK -

Well you sure have tried a variety of iterations, thank you for describing your process. I’ve used the freshwater models many times, with many different DEM sources and not had this experience, so it’s curious that it’s happening consistently with the sources that you’ve tried.

Probably easiest is if you can send me all of your model inputs, so I can try to replicate the error, and look at what’s happening to the DEM. You can share the data in any way that works for you, and if you’d rather not post it (or a link to it) here, you can send it to swolny@stanford.edu.

~ Stacie

[CarolynK]

I sent you an email with the model inputs. Thanks for your quick response!

[swolny]

Thanks for sending your data @CarolynK.

The specific error that NDR produces is

ValueError: cannot convert float NaN to integer

This is happening because all of your input rasters’ NoData values are not set to a valid data value, they are set to “nan”. When I save new versions of the input rasters, assigning a value of -9999 to the NoData value, the model runs successfully.

The model doesn’t produce an intermediate file for the pit-filled DEM (which is actually surprising), so I can’t look at that specifically, but the resulting rasters look ok to me, regarding flow accumulation, stream creation and variation in N export.

For more information, please see this recent thread where we talk about the same NaN issue.

~ Stacie

[CarolynK]

Hi @swolny, thanks for your help!

It’s worth noting that I was processing rasters in R (package terra), and though I replaced NoData values with -9999, NaN values were re-introduced in the exporting process. For the reference of others working in R, the fix was to specify the data type of each exported raster as integer (e.g. writeRaster(x, filename, datatype="INT4S")). By doing this, I was able to eliminate NaN values and run the model successfully.

[swolny]

Great @CarolynK, glad it got resolved, and thank you for letting us know how you fixed it!

~ Stacie