HRA generates empty output

Hi Everyone,

I have to admit that I don’t have experience with python but I know more about R. My overall aim is to run the hra python code via R. But firstly I wanted to make sure the hra code and my data were running using Python via Pycharm before running the model via R. My Invest environment is set up is natcap.invest 3.9.1.
However, when I set up my data and run the hra code using PyCharm, the code is running with my data and no error message is generated. But when I am opening the summary statistic from the output file there is no values in most of columns and the raster are just big squares even when I visualised them using the hra-dashboard. I attached the summary statistics csv file and one of the raster as example.
SUMMARY_STATISTICS.csv (998 Bytes)
TOTAL_RISK_Ecosystem.tif (2.3 KB)
I am not sure where the problem comes from, but something must not be quite right :sweat_smile: any suggestion would be welcome :slightly_smiling_face:.

Thanking you in advance
Morgane

Hi @Morgane,

Thanks for the post and interesting use case! With your PyCharm setup can you run HRA with the sample data and verify that the results look reasonable? That’ll be a good first step in telling us if it’s a model / environment setup problem or maybe a data input problem.

Best,

Doug

Hi @dcdenu4
Thank you for getting back to me so promptly.
So I just tried with the sample data I had the warning as you can see on the following image


and I obtained the following outputs as example
SUMMARY_STATISTICS.csv (6.9 KB)
TOTAL_RISK_Ecosystem.tif (2.7 KB)

I am not sure whether or not I should have a warning when I use the sampling data ? :sweat_smile:

Cheers,
Morgane

Thanks @Morgane for running that and sharing some outputs. Those warnings are okay to ignore.

Could you share the script you’re using to run HRA or talk about how you created it? Is it possible you could update your invest package to the latest 3.10.1 version ( and dependencies? )

Are you hoping to do batch runs of HRA?

Best,

Doug

Hi @dcdenu4

Thank you for your answer and your help :slightly_smiling_face:.
I attached the HRA zip file
HRA.zip (40.1 KB)
to this message with my hra.py and validation.py and a txt file called “Invest_envs_ installation” explaining how I installed the environment and ran hra.py the code lunching Pycharm from Anaconda. There are also comments at the begging of both hra.py and validation.py file regarding the bit of code I had to change to solve the error message I got. The hra.py also contains a bit of code (from line 27 to 40) related to my data input. To input my data I followed the instructions available on this page InVEST Model Entry Points — InVEST 3.10.2.post6+gd0716b0e.d20220208 documentation.

Regarding running batches I am not sure yet but yes I guess so because I am working on assessing the cumulative risk of climate change, renewable energies and fisheries from a ecosystem scale down to a local one for my PhD. But I am just starting to understand how the model works and runs and how I could run it using R as well :sweat_smile:.
Kind regards
Morgane

Hi @Morgane,

Thanks for sharing. I didn’t know you could program with Python in RStudio, cool!

I would highly recommend updating or creating a new conda environment with the latest version of InVEST, which is now 3.10.2! Each release will have important bugfixes and updating your InVEST version to the latest will help keep up with those. All recent versions of InVEST should be backwards compatible, so updating InVEST should not cause major breaks to your code.

I noticed that you were editing the source code for hra.py to run your data. If you just want to run your data in a scripting environment without needing to edit the source code for the model we would recommend the following workflow.

  1. Download the latest version of InVEST
  2. Run the HRA model user interface and from the menu toolbar select Development->Save to python script
  3. Open the python script that is created

Running that python script in Pycharm or RStudio with your conda env activated will execute the HRA model with your inputs. There shouldn’t be a need to edit the source code directly until you want to change the HRA model in some way to add new functionality.

Let us know if that makes sense and if you have anymore questions.

Cheers,

Doug

Hi @Morgane,

I don’t know very much about integrating Python and R code together, so am not sure what the best approach would be to making edits to the HRA source. I certainly recommend updating to the latest InVEST version. You can do that by either updating InVEST in your current conda environment, or sometimes a more trouble free approach is creating a new conda env. I believe how you were setting up your conda env looked okay to me, it seemed to reflect our preferred method mentioned here: Installing the InVEST Python Package — InVEST 3.10.2.post6+gd0716b0e.d20220208 documentation.

I will mention that HRA is currently getting a refactor that will fix some bugs and ambiguities in the code. So keep an eye out for that in an upcoming InVEST release. We try to release monthly on average.

Sorry I can’t be more help about the Python and R integration,

Doug

Hi Doug,

Thank you for your help I will dig up the R and Python method.
However I installed the lasted version of Invest and ran it using the software


And it is still telling me that the model ran successfully and output are empty and I am not getting any error message. So maybe there is a problem with my data but I don’t know where is the problem.

Would you mind let me know if there is any problem with how my data are defined or if I am missing something ? I attached them to in the following zip file HRA_INPUT.zip (1.7 MB)
Thanking you in advance for your time and help :slightly_smiling_face:

Morgane

Hi @dcdenu4,

Sorry to ask for help again. I think there is a problem with my data because I updated my env with the lasted invest version and the model is running with sample data but I still have empty output and no error message when I run it with my own data. I attached my data in the following Zip file
HRA_INPUT.zip (1.7 MB) is there any chance you could look at it to see if there is any problem coming from it.

Thanking you fro help and your time.
Morgane

Hi @Morgane , sorry for the delay, Doug is offline for a little while.

The extent of your input data (habitats, subregions, etc) is -4.0,51.0 : 4.0,59.50

And the units are in meters according to the coordinate system defined for those layers (UTM Zone 10N)

But the resolution of analysis you chose is 7000 meters. So zero grid cells of width & height 7000 meters fit inside your area of interest.

Maybe you didn’t really intend for the extent of all those layers to be so small. Perhaps they have an incorrectly defined coordinate system? Do the numbers -4.0,51.0 : 4.0,59.50 make more sense as decimal degrees?

Hi @dave,

Thank you for getting back to me so quickly.
I am not sure if these number make sense, but it looks like I have coordinate system problem.
I used the coordinate the same coordinate system as the sample data which transmecator UTM Zone if I remember correctly. But would you recommend using another given I am using data in the North Sea maybe British National that is in meters units as well?

Thank you
Morgane

Yes, you should use a coordinate system that makes sense for your local area. But also be aware of a very common pitfall of “re-defining” the existing projection instead of actually “re-projecting” to a different system.

Always look at your data in GIS, ideally add another layer from a different source that you can use for reference, such as borders for neighboring countries.

If you’re in the North Sea, then your data with bounding boxes of -4.0,51.0 : 4.0,59.50 are surely meant to represent longitude, latitude degrees, right? So you probably want to first “re-define” the coordinate system to geographic coordinates (e.g. WGS84, or whatever coordinate system was in place originally) and then “re-project” to a new local projected coordinate system that uses meters for units.

Hi @Dave,

Thank you very much. Yes the number are longitude, latitude and degree.
Yes I was thinking of using WGS84 as well but I was wandering if you would have a different opinion. I am going to re define and re-project my data and see if it is tackling the problem.

Thank you for your help :slightly_smiling_face:.

Morgane