Coastal Vulnerability input data coordinate system issues

NatCap Software Support
1

When I run this model I keep receiving a “Type Error” message that says is not a string. I have no idea what to do or what it is related to. I was getting value errors at first but I fixed that and now I am getting this message.

Update: log file is included below.
2019-11-06 10:21:59,793 utils.prepare_workspace(109) INFO Writing log messages to C:\Users\cwallace1\Pictures\School Stuff\outputs\InVEST-Coastal-Vulnerability-log-2019-11-06–10_21_59.txt
2019-11-06 10:21:59,799 model._logged_target(1633) Level 100 Starting model with parameters:
Arguments for InVEST natcap.invest.coastal_vulnerability 3.7.0:
aoi_vector_path C:/Users/cwallace1/Documents/New Research Data/AOI_CK_11_7.shp
dem_averaging_radius 5000
dem_path C:/Users/cwallace1/Documents/New Research Data/ned38/ned38/ned38.ovr
geomorphology_fill_value 3
geomorphology_vector_path C:/Users/cwallace1/Documents/New Research Data/Gmms_1.shp
habitat_table_path C:/Users/cwallace1/Documents/New Research Data/geo1.csv
landmass_vector_path C:/Users/cwallace1/Documents/New Research Data/LP_Final.shp
max_fetch_distance 1200
model_resolution 3000
n_workers -1
population_radius
population_raster_path
results_suffix
shelf_contour_vector_path C:/Users/cwallace1/Documents/New Research Data/conshe/conshe.shp
slr_field H
slr_vector_path C:/Users/cwallace1/Documents/New Research Data/CVInputs_GulfWide_JMS/SLR_WGS84.shp
workspace_dir C:\Users\cwallace1\Pictures\School Stuff\outputs
wwiii_vector_path C:/Users/cwallace1/Pictures/School Stuff/InVEST_3.7.0_x86/CoastalProtection/Input/WaveWatchIII.shp

2019-11-06 10:21:59,818 coastal_vulnerability.execute(95) INFO Validating arguments
2019-11-06 10:21:59,842 validation._wrapped_validate_func(148) DEBUG Validation warnings: []
2019-11-06 10:21:59,928 coastal_vulnerability.calculate_relief_exposure(1136) INFO Calculating relief exposure
2019-11-06 10:21:59,937 Task.add_task(636) ERROR Something went wrong when adding task calculate relief exposure (7), terminating taskgraph.
Traceback (most recent call last):
File “Z:\opt\atlassian\pipelines\agent\build\env\lib\site-packages\taskgraph\Task.py”, line 602, in add_task
File “Z:\opt\atlassian\pipelines\agent\build\env\lib\site-packages\taskgraph\Task.py”, line 1068, in _call
File “Z:\opt\atlassian\pipelines\agent\build\env\lib\site-packages\natcap\invest\coastal_vulnerability.py”, line 1152, in calculate_relief_exposure
File “Z:\opt\atlassian\pipelines\agent\build\env\lib\site-packages\natcap\invest\coastal_vulnerability.py”, line 1763, in clip_and_project_raster
File “Z:\opt\atlassian\pipelines\agent\build\env\lib\site-packages\pygeoprocessing\geoprocessing.py”, line 2683, in transform_bounding_box
File “C:\Python27\lib\site-packages\osgeo\osr.py”, line 765, in ImportFromWkt
TypeError: not a string
2019-11-06 10:21:59,938 model._logged_target(1639) ERROR Exception while executing <function execute at 0x0F00F8F0>
Traceback (most recent call last):
File “Z:\opt\atlassian\pipelines\agent\build\env\lib\site-packages\natcap\invest\ui\model.py”, line 1636, in _logged_target
File “Z:\opt\atlassian\pipelines\agent\build\env\lib\site-packages\natcap\invest\coastal_vulnerability.py”, line 277, in execute
File “Z:\opt\atlassian\pipelines\agent\build\env\lib\site-packages\taskgraph\Task.py”, line 602, in add_task
File “Z:\opt\atlassian\pipelines\agent\build\env\lib\site-packages\taskgraph\Task.py”, line 1068, in _call
File “Z:\opt\atlassian\pipelines\agent\build\env\lib\site-packages\natcap\invest\coastal_vulnerability.py”, line 1152, in calculate_relief_exposure
File “Z:\opt\atlassian\pipelines\agent\build\env\lib\site-packages\natcap\invest\coastal_vulnerability.py”, line 1763, in clip_and_project_raster
File “Z:\opt\atlassian\pipelines\agent\build\env\lib\site-packages\pygeoprocessing\geoprocessing.py”, line 2683, in transform_bounding_box
File “C:\Python27\lib\site-packages\osgeo\osr.py”, line 765, in ImportFromWkt
TypeError: not a string
2019-11-06 10:21:59,938 model._logged_target(1642) INFO Execution finished
2019-11-06 10:21:59,940 utils.prepare_workspace(115) INFO Elapsed time: 0.14s

2

Hi @Cwallace3 -

Please post the whole log file (.txt) that the model produces in the output Workspace, that will give us more information to potentially help us debug your problem. Thanks.

~ Stacie

1 Like
3

@Cwallace3, could you check that your shore points vector and your DEM both have spatial references defined? The error message makes it sound like at least one of these doesn’t have a defined SRS.

If this isn’t it, could you share your inputs with us? Google drive, dropbox, etc. usually work well.

Thanks,
James

4

Both of the files have defined spatial references. I have attached my files below.

https://drive.google.com/drive/folders/1JZpqw0izzGjNwmIKizfYWIKuD466ltOP

5

Ah, thanks for sharing your inputs! I believe the issue here is that the DEM input is an ovr file, which doesn’t actually have any spatial reference information in it (at least none that GDAL can detect):

Driver: GTiff/GeoTIFF
Files: /mnt/c/Users/jdouglass/Downloads/2019-11-11-cwallace-cv-error/Cedar_Key_DEM.ovr
Size is 1356, 1255
Coordinate System is `'
Image Structure Metadata:
  COMPRESSION=DEFLATE
  INTERLEAVE=BAND
Corner Coordinates:
Upper Left  (    0.0,    0.0)
Lower Left  (    0.0, 1255.0)
Upper Right ( 1356.0,    0.0)
Lower Right ( 1356.0, 1255.0)
Center      (  678.0,  627.5)
Band 1 Block=128x128 Type=Byte, ColorInterp=Gray
  NoData Value=-128
  Overviews: 678x628, 339x314, 170x157

Typically this .ovr file is derived from the DEM dataset itself. I suspect that if you were to provide the original DEM input to the model, things should run just fine.

Let us know how this goes!

6

I keep getting the same error messages.

7

Hmm … well, could you share your non-OVR DEM so I can try to reproduce the issue on my end?

Thanks!
James

8

Not a problem, the file is “Ned38”
https://drive.google.com/drive/folders/1JZpqw0izzGjNwmIKizfYWIKuD466ltOP

9

OK! I see the file there, but it’s another overview file, which also does not have any spatial information. Is this raster from NED? If so, your NED download should have come with one or more GeoTiffs, all of which should have spatial information. Could you send the GeoTiff for this area?

10

I uploaded the whole file folder from NED to the google drive folder. Thank you for taking the time to help me with this.

11

OK, thanks! So, the raster you need (with the required spatial information) is inside your google drive folder at ned38/ned38/ned38/hdr.adf. If you provide that file (hdr.adf) to the InVEST CV model, it should work at least get past the error you mentioned above.

12

Ok so that worked but now Im getting a value error.

2019-11-14 10:39:34,572 utils.prepare_workspace(109) INFO Writing log messages to C:\Users\cwallace1\Pictures\School Stuff\outputs\InVEST-Coastal-Vulnerability-log-2019-11-14–10_39_34.txt
2019-11-14 10:39:34,575 model._logged_target(1633) Level 100 Starting model with parameters:
Arguments for InVEST natcap.invest.coastal_vulnerability 3.7.0:
aoi_vector_path C:/Users/cwallace1/Documents/New Research Data/AOI_CK_11_7.shp
dem_averaging_radius 5000
dem_path C:/Users/cwallace1/Documents/New Research Data/ned38/ned38/ned38/hdr.adf
geomorphology_fill_value 1
geomorphology_vector_path
habitat_table_path C:/Users/cwallace1/Documents/New Research Data/Habitats/geo1.csv
landmass_vector_path C:/Users/cwallace1/Documents/New Research Data/LP_Final.shp
max_fetch_distance 1200
model_resolution 3000
n_workers -1
population_radius
population_raster_path
results_suffix
shelf_contour_vector_path C:/Users/cwallace1/Documents/New Research Data/conshe/conshe.shp
slr_field
slr_vector_path
workspace_dir C:\Users\cwallace1\Pictures\School Stuff\outputs
wwiii_vector_path C:/Users/cwallace1/Pictures/School Stuff/InVEST_3.7.0_x86/CoastalProtection/Input/WaveWatchIII.shp

2019-11-14 10:39:34,653 coastal_vulnerability.execute(95) INFO Validating arguments
2019-11-14 10:39:34,724 validation._wrapped_validate_func(148) DEBUG Validation warnings: []
2019-11-14 10:39:34,877 coastal_vulnerability.search_for_habitat(1306) INFO Searching for Shrub in proximity to shore points
2019-11-14 10:39:34,897 Task.add_task(636) ERROR Something went wrong when adding task searching for Shrub (8), terminating taskgraph.
Traceback (most recent call last):
File “Z:\opt\atlassian\pipelines\agent\build\env\lib\site-packages\taskgraph\Task.py”, line 602, in add_task
File “Z:\opt\atlassian\pipelines\agent\build\env\lib\site-packages\taskgraph\Task.py”, line 1068, in _call
File “Z:\opt\atlassian\pipelines\agent\build\env\lib\site-packages\natcap\invest\coastal_vulnerability.py”, line 1317, in search_for_habitat
File “Z:\opt\atlassian\pipelines\agent\build\env\lib\site-packages\pygeoprocessing\geoprocessing.py”, line 1473, in get_vector_info
ValueError: Could not open C:/Users/cwallace1/Documents/New Research Data/Habitats\Habitat/Shrub1.shp as a gdal.OF_VECTOR
2019-11-14 10:39:34,897 model._logged_target(1639) ERROR Exception while executing <function execute at 0x0FA328F0>
Traceback (most recent call last):
File “Z:\opt\atlassian\pipelines\agent\build\env\lib\site-packages\natcap\invest\ui\model.py”, line 1636, in _logged_target
File “Z:\opt\atlassian\pipelines\agent\build\env\lib\site-packages\natcap\invest\coastal_vulnerability.py”, line 284, in execute
File “Z:\opt\atlassian\pipelines\agent\build\env\lib\site-packages\natcap\invest\coastal_vulnerability.py”, line 1276, in _schedule_habitat_tasks
File “Z:\opt\atlassian\pipelines\agent\build\env\lib\site-packages\taskgraph\Task.py”, line 602, in add_task
File “Z:\opt\atlassian\pipelines\agent\build\env\lib\site-packages\taskgraph\Task.py”, line 1068, in _call
File “Z:\opt\atlassian\pipelines\agent\build\env\lib\site-packages\natcap\invest\coastal_vulnerability.py”, line 1317, in search_for_habitat
File “Z:\opt\atlassian\pipelines\agent\build\env\lib\site-packages\pygeoprocessing\geoprocessing.py”, line 1473, in get_vector_info
ValueError: Could not open C:/Users/cwallace1/Documents/New Research Data/Habitats\Habitat/Shrub1.shp as a gdal.OF_VECTOR
2019-11-14 10:39:34,897 model._logged_target(1642) INFO Execution finished
2019-11-14 10:39:34,897 utils.prepare_workspace(115) INFO Elapsed time: 0.32s

13

So, this error is saying that GDAL can’t open your habitat vector Shrub1.shp as a vector. Are you able to open it in ArcMap or QGIS? It’s worth noting that with ESRI Shapefiles, all of the component files are important to include … so be sure to have Shrub1.shp alongside Shrub1.dbf, Shrub1.shx and whatever other files also are named Shrub1.*.

14

Yes I can open the files in ArcGis. All “habitat”.* are all in the same folder as well.

15

Huh, well that is strange indeed. My only other thought related to Arc is that maybe InVEST isn’t able to open the files if they’re also open in Arc or some other program?

As an alternative, could you try exporting the files to a different vector format and see if that helps?

16

Now I keep receiving either an OverFlowError message or Value Error: cannot float NaN to integer

2019-11-20 12:33:30,921 utils.prepare_workspace(109) INFO Writing log messages to C:\Users\cwallace1\Documents\New Research Data\Cwallace_Invest_Inputs\InVEST-Coastal-Vulnerability-log-2019-11-20–12_33_30.txt
2019-11-20 12:33:30,923 model._logged_target(1633) Level 100 Starting model with parameters:
Arguments for InVEST natcap.invest.coastal_vulnerability 3.7.0:
aoi_vector_path C:/Users/cwallace1/Documents/New Research Data/AOI_CK_11_7.shp
dem_averaging_radius 200
dem_path C:/Users/cwallace1/Documents/New Research Data/ned38/ned38/ned38/hdr.adf
geomorphology_fill_value 1
geomorphology_vector_path C:/Users/cwallace1/Documents/New Research Data/Cwallace_Invest_Inputs/Geomorph_11_14.shp
habitat_table_path C:/Users/cwallace1/Documents/New Research Data/Habitats/geo_1.csv
landmass_vector_path C:/Users/cwallace1/Documents/New Research Data/LP_Final.shp
max_fetch_distance 1200
model_resolution 10
n_workers -1
population_radius
population_raster_path
results_suffix
shelf_contour_vector_path C:/Users/cwallace1/Documents/New Research Data/conshe/conshe.shp
slr_field UNKNOWN
slr_vector_path
workspace_dir C:\Users\cwallace1\Documents\New Research Data\Cwallace_Invest_Inputs
wwiii_vector_path C:/Users/cwallace1/Pictures/School Stuff/InVEST_3.7.0_x86/CoastalProtection/Input/WaveWatchIII.shp

2019-11-20 12:33:30,940 coastal_vulnerability.execute(95) INFO Validating arguments
2019-11-20 12:33:30,954 validation._wrapped_validate_func(148) DEBUG Validation warnings: []
2019-11-20 12:33:31,266 geoprocessing.reproject_vector(1647) INFO starting reprojection
2019-11-20 12:33:31,279 geoprocessing.reproject_vector(1690) INFO reprojection 100.0% complete on clipped_projected_landmass.gpkg
2019-11-20 12:33:31,345 coastal_vulnerability.prepare_landmass_line_index(406) INFO preparing landmass geometry
2019-11-20 12:33:31,494 coastal_vulnerability.prepare_landmass_line_index(443) INFO indexing geometry of landmass
2019-11-20 12:33:31,516 coastal_vulnerability.interpolate_shore_points(474) INFO creating shore points along edge of landmass
2019-11-20 12:33:31,618 coastal_vulnerability.interpolate_wwiii_to_shore(573) INFO Building spatial index for Wave Watch III points
2019-11-20 12:33:39,828 coastal_vulnerability.interpolate_wwiii_to_shore(617) INFO Interpolating Wave Watch III data to shore points
2019-11-20 12:33:39,855 coastal_vulnerability.calculate_wind_exposure(812) INFO Calculating wind exposure
2019-11-20 12:33:40,144 coastal_vulnerability.calculate_wind_exposure(866) INFO Casting rays
2019-11-20 12:33:40,217 coastal_vulnerability.calculate_wave_exposure(1002) INFO Calculating wave exposure
2019-11-20 12:33:40,243 coastal_vulnerability.calculate_surge_exposure(1051) INFO Calculating surge potential
2019-11-20 12:33:40,283 coastal_vulnerability.calculate_relief_exposure(1136) INFO Calculating relief exposure
2019-11-20 12:33:40,292 utils._log_gdal_errors(66) ERROR [errno 1] latitude or longitude exceeded limits
2019-11-20 12:33:40,292 utils._log_gdal_errors(66) ERROR [errno 1] latitude or longitude exceeded limits
2019-11-20 12:33:40,292 utils._log_gdal_errors(66) ERROR [errno 1] latitude or longitude exceeded limits
2019-11-20 12:33:40,292 utils._log_gdal_errors(66) ERROR [errno 1] latitude or longitude exceeded limits
2019-11-20 12:33:40,292 utils._log_gdal_errors(66) ERROR [errno 1] latitude or longitude exceeded limits
2019-11-20 12:33:40,292 utils._log_gdal_errors(66) ERROR [errno 1] latitude or longitude exceeded limits
2019-11-20 12:33:40,292 utils._log_gdal_errors(66) ERROR [errno 1] latitude or longitude exceeded limits
2019-11-20 12:33:40,292 utils._log_gdal_errors(66) ERROR [errno 1] latitude or longitude exceeded limits
2019-11-20 12:33:40,292 utils._log_gdal_errors(66) ERROR [errno 1] latitude or longitude exceeded limits
2019-11-20 12:33:40,292 utils._log_gdal_errors(66) ERROR [errno 1] latitude or longitude exceeded limits
2019-11-20 12:33:40,292 utils._log_gdal_errors(66) ERROR [errno 1] latitude or longitude exceeded limits
2019-11-20 12:33:40,292 utils._log_gdal_errors(66) ERROR [errno 1] latitude or longitude exceeded limits
2019-11-20 12:33:40,292 utils._log_gdal_errors(66) ERROR [errno 1] latitude or longitude exceeded limits
2019-11-20 12:33:40,293 utils._log_gdal_errors(66) ERROR [errno 1] latitude or longitude exceeded limits
2019-11-20 12:33:40,293 utils._log_gdal_errors(66) ERROR [errno 1] latitude or longitude exceeded limits
2019-11-20 12:33:40,293 utils._log_gdal_errors(66) ERROR [errno 1] latitude or longitude exceeded limits
2019-11-20 12:33:40,293 utils._log_gdal_errors(66) ERROR [errno 1] latitude or longitude exceeded limits
2019-11-20 12:33:40,293 utils._log_gdal_errors(66) ERROR [errno 1] latitude or longitude exceeded limits
2019-11-20 12:33:40,293 utils._log_gdal_errors(66) ERROR [errno 1] latitude or longitude exceeded limits
2019-11-20 12:33:40,293 utils._log_gdal_errors(66) ERROR [errno 1] Reprojection failed, err = -14, further errors will be suppressed on the transform object.
2019-11-20 12:33:40,298 Task.add_task(636) ERROR Something went wrong when adding task calculate relief exposure (7), terminating taskgraph.
Traceback (most recent call last):
File “Z:\opt\atlassian\pipelines\agent\build\env\lib\site-packages\taskgraph\Task.py”, line 602, in add_task
File “Z:\opt\atlassian\pipelines\agent\build\env\lib\site-packages\taskgraph\Task.py”, line 1068, in _call
File “Z:\opt\atlassian\pipelines\agent\build\env\lib\site-packages\natcap\invest\coastal_vulnerability.py”, line 1152, in calculate_relief_exposure
File “Z:\opt\atlassian\pipelines\agent\build\env\lib\site-packages\natcap\invest\coastal_vulnerability.py”, line 1778, in clip_and_project_raster
File “Z:\opt\atlassian\pipelines\agent\build\env\lib\site-packages\pygeoprocessing\geoprocessing.py”, line 1852, in warp_raster
ValueError: cannot convert float NaN to integer
2019-11-20 12:33:40,299 model._logged_target(1639) ERROR Exception while executing <function execute at 0x0FC768F0>
Traceback (most recent call last):
File “Z:\opt\atlassian\pipelines\agent\build\env\lib\site-packages\natcap\invest\ui\model.py”, line 1636, in _logged_target
File “Z:\opt\atlassian\pipelines\agent\build\env\lib\site-packages\natcap\invest\coastal_vulnerability.py”, line 277, in execute
File “Z:\opt\atlassian\pipelines\agent\build\env\lib\site-packages\taskgraph\Task.py”, line 602, in add_task
File “Z:\opt\atlassian\pipelines\agent\build\env\lib\site-packages\taskgraph\Task.py”, line 1068, in _call
File “Z:\opt\atlassian\pipelines\agent\build\env\lib\site-packages\natcap\invest\coastal_vulnerability.py”, line 1152, in calculate_relief_exposure
File “Z:\opt\atlassian\pipelines\agent\build\env\lib\site-packages\natcap\invest\coastal_vulnerability.py”, line 1778, in clip_and_project_raster
File “Z:\opt\atlassian\pipelines\agent\build\env\lib\site-packages\pygeoprocessing\geoprocessing.py”, line 1852, in warp_raster
ValueError: cannot convert float NaN to integer
2019-11-20 12:33:40,299 model._logged_target(1642) INFO Execution finished
2019-11-20 12:33:40,302 utils.prepare_workspace(115) INFO Elapsed time: 9.38s

17

Neat! That’s not the sort of error we see every day :slight_smile: Are all of your inputs in the Google Drive? It’ll be helpful to try to reproduce the error on our end.

18

Oh Boy! Yes all of my inputs are on the google drive

19

Alright, well, I can’t open the vectors in the google drive (using either Arc or QGIS) to verify this, but based on the traceback and the source code, I would guess that your AOI is either not projected at all or is not projected in a linear coordinate system.

The errors you see a bit farther up in the logfile:

These make me think that there’s a projections issue going on. In CV, files are reprojected to the coordinate system of the AOI, so if that isn’t set or is set incorrectly, that might lead to the ValueError you mention.

Anyways, could you take a look at your projections and verify that they make sense?

Thanks!
James

20

I would actually expect the model to raise a validation error right away if the AOI is not projected in a linear coordinate system using meters, so I wonder if the AOI has an incorrectly defined coordinate system.

Anyway, when sharing shapefiles, please be sure to always include all the files with the same basename, not just the .shp

Finally, consider using a much larger model resolution until all these data problems are sorted out. In your log it is currently set at 10 meters.