Yep, this is a software issue and we’re working on it! I don’t have much to report at the moment other than that the fix is in progress.
Hi @swolny and @jdouglass
Thank you for the answers! I appreciate it.
Yes, you’re right Stacie. I also working with the large area. However, I already try to install and run the model by using the latest version (3.8.2) and it’s still shows the same error. How about you, have you try to run using the latest version?
Here I attached the screenshot after running the model. Do you think it’s related with the threshold flow accumulation?
Once again, thank you.
Armyanda
Hi @armyanda92 -
The log file isn’t posted, but I bet if you looked further up in it, you’d see the same LZW error as before. And no, it doesn’t seem to be related to the threshold flow accumulation value. I have tried 3.8.2 and had the same problem, but a coworker was able to successfully run her data with it, that’s why I thought it was worth trying. Sorry that didn’t help.
While the software team is working on a fix, the only thing I can recommend is trying InVEST 3.7, which did not have this problem. Of course, there have been other changes and bug fixes between 3.7 and 3.8, but I don’t believe that the SDR calculations have substantially changed. It would at least allow you to run the model and start to work with outputs. Here’s a link to where you can download version 3.7, if you want to try that.
~ Stacie
Hi @swolny
Thank you so much for your answer. It really give me the insight!
I will try to run with the InVEST 3.7 then.
If you do not mind, please kindly let me know if you already success to run your SDR model using the latest version or if you know if there is any upgrade from the InVEST model to fix the problems.
Once again, thank you for the discussion.
Best,
Armyanda
Hi @armyanda92 -
The latest (3.8.2) version of SDR does not work for me with large datasets, it probably runs fine if the area of interest is small and/or the input data (particularly the DEM) is not large in size, but I haven’t had time to test this yet. Since it is easy enough to install multiple versions of InVEST, it does not hurt to try 3.8.2, but if that does not work, then I do recommend 3.7.
The software team is working on fixing the problem, but there is not an upgrade yet. When new versions are released, they will announce it on this forum, so keep an eye out!
~ Stacie
Hi @swolny
I already try to run using the InVEST 3.7.0 but it still shows an error. Although the error from this version was different with the 3.8.2.
Here I attached the model log files (.txt). If you do not mind, would you like to check it and give some advice of the error data? It will really helps me!
In the meantime, I will try to run with the small and/or the input data using the 3.8.2 version.
Once again, thank you so much for your help!
Best,
Armyanda
InVEST-Sediment-Delivery-Ratio-Model-(SDR)-log-2020-05-21–21_13_37.txt (251.2 KB)
Hi @armyanda92 -
So this is the error:
ValueError: The following 2 raster values [4 6] from “D:\RESEARCH\WRI-Indonesia\Waves-RZWP3K-Project\InVEST\SDR\Trial-SDR-Sumsel\RESULT-SDR-Sumsel\SDR-Sumsel-1i\intermediate_outputs\churn_dir_not_for_humans\aligned_lulc.tif” do not have corresponding entries in the value_map: {1L: 0.1, 2L: 0.121, 3L: 0.034, 5L: 0.41200000000000003, 8L: 0.025, 9L: 0.0}
Sorry this error message isn’t more clear, but it means that there are two values in your land cover map (4 and 6) that are not in the biophysical table. All values in lc96_resample_copy.tif must have values in Biophysical_table_6class.csv, as is noted in the Data Needs section of the User Guide. Try adding these entries to your biophysical table and see if it helps.
~ Stacie
Hi @swolny
I checked the values between the land cover map and the biophysical table, it shows that all the land cover classes are already listed in the biophysical table.
Here I attached the screenshot of my land cover and biophysical data. If you do not mind and have time, please kindly review it. I highly appreciate it!
Best,
Armyanda
Hi @armyanda92,
If the LULC raster is not too big you can certainly upload it here so we can take a look. If it is too large you could share it via google drive, email, or another sharing method.
Best,
Doug
Hi @dcdenu4
Now I try to run the SDR model using InVEST 3.8.2 with a smaller study area (almost the same as the sample data). However, I also find an error that I think it’s also related with the land cover map.
Here I attached the land cover map (.tif) and biophysical table (.csv). Please kindly check and review it. It will really help me.
Thank you for your concern!
Best,
Armyanda
Biophysical_table_5cl.csv (138 Bytes)
alor_lulc_96_utm.tif (6.8 KB)
Hi @armyanda92 -
When you have a new error, please post the log file from that run, it will have information specific to this error - thanks.
Looking at your data, I bet the error is related to a missing value of 1 in the biophysical table. Your land cover map has values 1-6, while the biophysical table only has values 2-6. So you need to add usle_c and usle_p data for value 1 to the biophysical table as well.
~ Stacie
1 Like
Hi @swolny
Thank you for your suggestion. I’m really sorry to not upload the model log file (.txt)
The value 1 is blank data in the land use raster. Hence, I deleted it and make it just 5 classes.
Here I attached the model log file (.txt). If you do not mind, please kindly give me another suggestion regards the error.
I highly appreciate all your answers. It really helps my work!
Best,
Armyanda
InVEST-Sediment-Delivery-Ratio-Model-(SDR)-log-2020-05-26–19_46_24.txt (19.4 KB)
Hi @armyanda92
Thanks for including the files. This is a bit strange because when looking at your LULC you provided it appears that you do indeed still have “1” values that are not in the Biophysical table CSV.
However, in the log file, it says that the missing value from the biophysical table is “4”, which I currently see in the CSV you provided… Moreover it says the Biophysical table has values: {8: 0.025, 3: 0.034, 1: 0.1, 5: 0.41200000000000003, 2: 0.121}, which is not what I see in the CSV.
Do you think you could zip up and share your input workspace that you are using so I can take a look at this more closely to make sure we’re not missing something obvious and too make sure there is not an underlying bug in the model?
Thanks and sorry for the issues,
Doug
Hi @dcdenu4
Thank you for your response.
Aah, I see!
Here I attached all my input data. Please kindly have a look and give me a suggestion for my future work.
I highly appreciate your help. It really helps me!
Best,
Armyanda
Alor_input_data.zip (3.2 MB)
Hi @armyanda92,
There was indeed a strange bug that we’ll work on fixing for future InVEST releases. In the meantime, I uploaded a revised Biophysical Table and LULC input that should work around the issue.
Biophysical Table Changes
- Swapped
descriptionandlucodecolumn so thatlucodeis first column
LULC Changes
- Set pixel values of 1 equal to 5 (as I believe you mentioned above… )
Biophysical_table_5cl-swapped.csv (308 Bytes)
alor_lulc_96_utm-remove-one.tif (274.3 KB)
Hopefully this should now work as expected for you. Let us know if it doesn’t!
Doug
1 Like
Hi @dcdenu4
Thank you very much for your help! It really helps my work.
I already try to run the model using the landcover raster data and biophysical table from you. It works now!
However, I have some questions for you, Doug. I see in the LULC raster data and there is no description table of the landcover name. So is it necessary or not to put the landcover name in the raster data? (I put the screenshot)
My next question is regarding the result. Do you think is it possible to see the amount of sediment export and retention from each of the land cover?
Once again, thank you for your help!
Best,
Armyanda
Hello @armyanda92 -
For InVEST, it is not necessary to have the land cover description in the raster. It certainly makes it easier to use if you work with a GIS program like ArcGIS that supports raster attribute tables, but programs like QGIS very unfortunately don’t support raster attribute tables at all. Since you do use ArcGIS, you can add a column with the description to the raster attribute table yourself.
The model does not output the amount of export and retention by land cover, but you can use a GIS tool like Zonal Statistics to easily calculate this.
~ Stacie
1 Like
Hi @swolny
Well noted!
Thank you for your answers, Stacie. It really helps me to process the data and interpret the result.
Once again, thank you for all your answer regarding my questions!
I’m sorry for the late response.
Best,
Armyanda
Hello again!
Thank you for all your response regards to my issues lately.
As I told you, I succeded to run the model using the data from Doug. It’s called the Alor district (the study area).
Currently, I try to run using a larger area called South Sumatera.
The total area of Alor district is 73000 ha meanwhile the South Sumatera is 8638590 ha. So it is totally different.
When I try to run use the latest version 3.8.4 it shows the same error which is “LZWDecode:Corrupted LZW table”.
Do you think it is because of the size of study area (in hectare) or the data size (megabyte)?
Because for the South Sumatera, the data size could be up to 2gb, meanwhile the Alor only around 50mb.
Thank you for your time in response to my question!
Best,
Armyanda
Hi Armyanda,
Sorry about the LZWDecode:Corrupted error you’re running into. We are still working on this error as mentioned above. I will see if I can get a development build to you soon that might help.
Best,
Doug