=>

This is a simple piece of code to automatically download the products provided by the French Sentinel collaborative ground segment named PEPS : https://peps.cnes.fr. PEPS is mirroring all the Sentinel data provided by ESA, and is providing a simplified access.This code was written thanks to the precious help of one my colleagues at CNES Jérôme Gasperi who developed the "rocket" interface which is used by Peps.This code relies on python 2.7 and on the curl utility. Because of that, I guess it only works with linux.The tool is available on my github repository : https://github.com/olivierhagolle/peps_download

## Examples

This software is still quite basic, but if you have an account at PEPS, you may download products using command lines like

• python ./peps_download.py -c S2 -l 'Toulouse' -a peps.txt -d 2015-11-01 -f 2015-12-01 which downloads the Sentinel-2 products above Toulouse, acquired in November 2015.
• python ./peps_download.py -c S2 --lon 1 --lat 43.5 -a peps.txt -d 2015-11-01 -f 2015-12-01 which downloads the Sentinel-2 products above --lon 1 --lat 43.5 (~Toulouse), acquired in November 2015.
• python ./peps_download.py -c S1 --lonmin 1 --lonmax 2 --latmin 43 --latmax 44 -a peps.txt -d 2015-11-01 -f 2015-12-01 which downloads the Sentinel-1 products in latitude, longitude box around Toulouse, acquired in November 2015.

## Authentification

The file peps.txt must contain your email address and your password on the same line, such as follows :your.email@address.fr top_secretTo get an account : https://peps.cnes.fr/rocket/#/register

## 39 thoughts on “How to automatically download Sentinel data from PEPS collaborative ground segment”

1. Christina dit :

I am fairly unfamiliar with Python and am having issues getting this code and the Sentinel_download.py code to work. I keep getting syntax errors for the input line (i.e. the decimal degrees for the longitude and latitude and the file path). I have added back slashes and apostrophes in the file path to get it to recognize it as a string, but there continue to be errors. Is there any other formatting that needs to be done to the line? I guess I assumed I could strictly copy and paste your input line and change to suit my parameters. Merci!

1. Olivier Hagolle dit :

Dear Christina,\nI lack information to be sure, but it seems you are first launching python, then pasting the command lines, Which is wrong.\nYou do not need to start python, these command lines are shell command lines.\nBest regards,\nOlivier

2. Denis dit :

1. Olivier Hagolle dit :

Thanks for your kind words, Denis,\n\nYes it is possible to access directly to the tiles, I have tried to add it in my sentinel-2 download module (with the -t option), but it is nor working yet.\nThe issue is that you have to download all the files one by one, so it is a little complicated.\n\nFor that, you need to use Url like this : (provided in an other comment by vince) \nhttps://scihub.copernicus.eu/dhus/odata/v1/Products('141ea4dd-2c99-4ffb-ac07-f5775e08b58f')/Nodes('S2A_OPER_PRD_MSIL1C_PDMC_20160209T011325_R008_V20160208T104841_20160208T104841.SAFE')/Nodes('GRANULE')/Nodes('S2A_OPER_MSI_L1C_TL_SGS__20160208T194222_A003294_T31TCJ_N02.01')/Nodes('IMG_DATA')/Nodes('S2A_OPER_MSI_L1C_TL_SGS__20160208T194222_A003294_T31TCJ_B02.jp2')/$value\n\nSol, if I find some time I will do it.\n\n\nOn your second point, of course Sentinel-2 data will be made available on GEE, and it should start quickly. But you have to be aware that GEE can use your code and applications and does not guarantee the platform will still be available next year. \n\nAnd finally, starting this summer, Theia will distribute atmospherically corrected data, by tiles, on selected regions such as Tunisia ;-)\n\nBest regards,\nOlivier 1. Olivier Hagolle dit : Bonjour Denis,\nMy little download tool : Sentinel-download, now allows to download specific tiles instead of whole S2 products thanks to the -t option. I have finally found some time to finish it. I have not tested it much, though, please tell me if you find some bugs.\nOlivier 1. Denis dit : Bonjour Olivier,I've just tried... and didn't manage to make it. I can't determine whether the bug is in your script or between my keyboard and my screen, so I post the results here ; I might have misunderstood or been too fast on something.My command is :python Sentinel_download.py --lat 34.75 --lon 10.8 -a apihub.txt -t 32SPF -d 20151225 -f 20151229I tried it both with the guest account and with my personal account (created october, 1st 2015). I also tried with a bigger time period.I browsed the archive on the scihub interface. The download link given for the whole 20151228 S2 scene on my site ishttps://scihub.copernicus.eu/dhus/odata/v1/Products('83297ffc-7c04-49f4-b80f-4f88ee77e1c3')/$valueThe http requests, give first error codes 401 and then correct code 200. I get the query_results.xml but I don't find file_dir.xml neither do I find granule_dir.xml...At the end I'm said "Tile 32SPF is not available within product"..Any idea ?Many thanks in advance,Denis

1. Denis dit :

By the way, with the command :python Sentinel_download.py --lat 43.6 --lon 1.44 -a apihub.txt -o 51 -t 31TCJ -d 20160101 -f 20160228I manage to get image files (I stopped the download after the first successful download of a .jp2 file).I tried to add the row number for my image of interest :python Sentinel_download.py --lat 34.75 --lon 10.8 -a apihub\ $copy$.txt -o 122 -t 32SPF -d 20151225 -f 20151229but I get the same result as in the comment above...

2. Olivier Hagolle dit :

Dear Denis, I tried the command :

python Sentinel_download.py --lat 34.75 --lon 10.8 -a apihub.txt -t 32SPF -d 20151225 -f 20151229

And what I get is this message :========================================Tile 32SPF is not available within product========================================And in fact, the tile number and the coordinates do not belong to the same place. For the Cap Bon, you are two degrees below. The following command line works for me.

python Sentinel_download.py --lat 36.75 --lon 10.8 -a apihub.txt -t 32SPF -d 20151225 -f 20151229

It was easy for me to find, I did the same error in one of my first tests...A bientôt,Olivier

3. Denis dit :

Perfect !How could I have set up this 34.75 ? Now it works, the answer has no interest ! Many thanks again Olivier.Denis

3. Denis dit :

Waw !Thank you so much Olivier. One comment and you develop a tool.I'll test it as soon as possible. I'm very interested in downloading a specific tile... over Tunisia ! ;-)Denis

4. Hallo Olivier,is it possible to run that script under Windows? I have installed python 2.7- it came with ArcGIS -, but no skills in using python. Is it possible that you could give me a short hint how to start the script? Would be a great help!Thank you, greetingsFrank

1. Olivier Hagolle dit :

Dear Frank, you have several examples above. It is supposed to work on windows, it needs installing aria2 software.\nAs I am not a windows user, I am not sure it works as well as on linux. Some users reported issues.\nBest regards,\nOlivier

5. Denis dit :

Dear Olivier,I'm back on trying to download tiles on our site, and I was facing a strange behaviour with the same working command as those days :python Sentinel_download.py --lat 36.75 --lon 10.8 -a apihub.txt -t 32SPF -d 2015-12-25 -f 2015-12-29It seems that the date format has change somwhere because with the following command, it works :python Sentinel_download.py --lat 36.75 --lon 10.8 -a apihub.txt -t 32SPF -d 20160316 -f 20160318Hope this helps,Denis

6. Olivier Hagolle dit :

Thanks Denis, you are right, I figured it out some time ago. I had changed the examples in many places blog, github) and in the description with -h option. But not in the comments. I need to correct it too.Olivier

7. Mario dit :

I am using peps but I am getting: python peps_download.py -c S2 -a /home/atos/cron/peps.txt -21 --latmax=41.9744 --lonmax=-8.7556curl -k -o search.json https://peps.cnes.fr/resto/api/collections/S2/search.json?axRecords=500 % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0Traceback (most recent call last): File "peps_download.py", line 130, in with open('search.json') as data_file:IOError: [Errno 2] No such file or directory: 'search.json'What is the problem?This used to work...

1. Olivier Hagolle dit :

It seems to work on my side. Maybe Peps was in maintenance ?\nPlease try again.\nOlivier

8. Denis dit :

Dear Olivier,We have a trouble with the sentinel download script. In our lab, we have installed a script on a server which automatically downloads latest image of our 3 tiles of interest.At first the script was launched once a day (now once a week).We had a correct behaviour since the end of october.For the 32SPF tile, the trouble appeared somewhere after the 24th of october image.The behaviour now is that the script creates an empty directory tree (the GRANULE directory contains an empty directory), and runs endlessly.Any idea ?Many thanks in advance,Denis

1. Denis dit :

Thanks to a private communication with Olivier, we went to the following solution : using the peps_download tool to download from the PEPS repository. It's ok for dates after the 24th of october since PEPS has tiles from then.Caution : before the 24th of octobre, the peps_download tool will download >5Go files (full images).Denis

9. Bogoljub dit :

>python download_SentFromPEPS.py -c S2ST --lon -114 --lat 55 -a peps.txt -d 2017-03-14 -f 2017-14-03I am trying to download data for an area in Canada. Following are errors i receive.'lon' is not recognized as an internal or external command,operable program or batch file.'startDate' is not recognized as an internal or external 'completionDate' is not recognized as an internal or 'maxRecords' is not recognized as an internal or external commandValue for "lat" must be numericwhat is it that i am missing in my command??

1. Olivier Hagolle dit :

Hi the date provided in the -f format has a wrong format, which should be YYYY-MM-DD. It is the cause of the problem. (I recognise the error message is wrong...)

10. Max dit :

Hi, thanks for the software, it might help me a lot for my research. But for now, I keep getting errors about date/location. Have I made something wrong ? Here is what I type : python peps_download.py -c S1 -p GRD -lonmin 6 --latmin 44.918 --lonmax 7.173 --lonmax 7.173 --latmax 45.559 -a peps.txt -d 2017-02-15 -f 2017-03-15And I get : 'startDate' n'est pas reconnu en tant que commande interneou externe, un programme exécutable ou un fichier de commandes.'completionDate' n'est pas reconnu en tant que commande interneou externe, un programme exécutable ou un fichier de commandes.'maxRecords' n'est pas reconnu en tant que commande interneou externe, un programme exécutable ou un fichier de commandes.'productType' n'est pas reconnu en tant que commande interneou externe, un programme exécutable ou un fichier de commandes.'sensorMode' n'est pas reconnu en tant que commande interneou externe, un programme exécutable ou un fichier de commandes.No product matches the criteriaThe ESA copernicus data hub seems to be under maintenance, which might explain for the last message, but not the first. Thanks a lot,Max

1. Max dit :

Btw, I saw the comment above, but the date does not seem to be a problem here

2. Olivier Hagolle dit :

Hi, \n\nthis type of error corresponds to a URL non understood\nI found it : you used -lonmin instead of --lonmin ;)\n\nBest regards,\nOlivier

1. Max dit :

Hi, You are right, I also wrote --lonmax twice. But those mistakes only apperead when I copied by hand my command line in this writing box 😉 I get a new one when correcting everything (and even copy-pasting example files) : Value for "box" must follow the pattern ^[0-9\.\,\-]*\$If I only specify a value for --lat, I get the previous error messages for " 'xxx' is not recognised "and : "Value for "lat" must be numeric"Thanks!Max

11. Inti dit :

1. Olivier Hagolle dit :

12. Inti dit :

Hello Olivier,Something weird is when I try another of the examples:python ./peps_download.py -c S2 --lon 1 --lat 43.5 -a peps.txt -d 2015-11-01 -f 2015-12-01 it gives the following error message:curl -k -o search.json https://peps.cnes.fr/resto/api/collections/S2/search.json?lat=43.500000\&lon=1.000000\&startDate=2015-11-01\&completionDate=2015-12-01\&maxRecords=500 % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed100 68 100 68 0 0 43 0 0:00:01 0:00:01 --:--:-- 43'lon' is not recognized as an internal or external command,operable program or batch file.'startDate' is not recognized as an internal or external command,operable program or batch file.'completionDate' is not recognized as an internal or external command,operable program or batch file.'maxRecords' is not recognized as an internal or external command,operable program or batch file.Value for "lat" must be numericWhat could be the problem? I tried all your examples in the README file and they don't work for me except for: python ./peps_download.py -c S2 -l 'Toulouse' -a peps.txt -d 2015-11-01 -f 2015-12-01Many thanks for your help

13. Yinat dit :

Hi Olivier,I tried to downloas Sentinel-1 data from peps with the following command : python ./peps_download.py -c S1 --lonmin 1 --lonmax 2 --latmin 43 --latmax 44 -a peps.txt -d 2015-11-01 -f 2015-12-01but i got this error : curl -k -o search.json https://peps.cnes.fr/resto/api/collections/S1/search.json?box=1.0,43.0,2.0,44.0\&startDate=2015-11-01\&completionDate=2015-12-01\&maxRecords=500sh: 1: curl: not foundTraceback (most recent call last): File "./peps_download.py", line 172, in with open(options.search_json_file) as data_file: IOError: [Errno 2] No such file or directory: 'search.json'I tried other examples with Sentinel-2 but always the same error.I verified that the library json exists in python but it seems not to be the problem.Did you know where is the problem ?Best regards

14. Yinat dit :

Problem fixed by installing curl!

1. Olivier Hagolle dit :

I was just writing that answer, but you found it !\nBest regards,\nOlivier

15. Yinat dit :

1. Olivier Hagolle dit :

Thanks for the feedback.\nI am not using S1 a lot... it was a wrong variable name. It has been corrected and pushed to github.\nFor next time, please use the "issues" of github, rather than commenting here, it is much easier to handle for me.\nOlivier

16. hbluo dit :

I input the command 'python Sentinel_download.py --lat 44.5 --lon 80 -a apihub.txt -d 20170101 -f 20171231 -o 158 -n -s S1A*SLC'. Then the errors ' File "Sentinel_download.py", line 7 ^SyntaxError: invalid syntax' are shown. How to solve the problem.Thanks

17. samira dit :

Hi Olivier,how it is possible to download data using our own geometry vector as follow:WKT,Name,description,timestamp,begin,end,altitudeMode,tessellate,extrude,visibility,drawOrder,icon"POLYGON Z ((91.0 23.5165536384341 0,90.5 24.0033838608002 0,90.2 24.5100469313974 0,90.15 25.0147145030644 0,90.5 25.5970699400967 0,91.0 25.5548536418031 0,91.5 25.5558223234974 0,92.0 25.7244080499758 0,92.5 25.9714642899991 0,93.0 26.0605811505251 0,93.5 26.1825057872809 0,94.0 26.1738683362909 0,94.5 26.3692228599101 0,95.0 26.9677550408619 0,95.5 27.0837961513984 0,96.0 27.2495743623361 0,96.5 27.2404969432799 0,96.65 27.4978440327206 0,96.5 28.4877353473867 0,96.0 28.8041171174123 0,95.5 28.8852730958532 0,95.0 28.4993551762164 0,94.5 27.7318637043648 0,94.0 27.4446587122329 0,93.5 27.3449597847392 0,93.0 27.1893206203557 0,92.5 27.1597488266727 0,92.0 26.9202623137292 0,91.5 26.7568665914695 0,91.0 26.7602952361474 0,90.5 26.7203546118003 0,90.0 26.5817349371244 0,89.5 26.4711751788167 0,89.23 25.506668929819 0,89.23 24.4934552653587 0,89.25 23.405933984027 0,89.5 22.9903128006586 0,90.0 21.0925157070459 0,90.5 21.0928410280981 0,91.0 21.2477977548221 0,91.25 21.4894497201735 0,91.5 22.0117069833903 0,91.5 22.5036642933011 0,91.0 23.5165536384341 0))",Region polygon - Brahmaputra river,,,,,,1,0,-1,,thanks in advance for your help.

1. Olivier Hagolle dit :

Hi, \npeps_download does not allow that (sorry).\nI would advise to find the list of tiles you need, and launch peps_download several times in a loop for each tile, with -t option.\nOlivier

18. samira dit :

if I want to download special tile like S1A_EW_GRDM_1SDH_20190318T120356_20190318T120456_026392_02F3FD_AF64how can i do this?i must use this as a command line?python Sentinel_download.py -a apihub.txt -s S1A_EW_GRDM_1SDH_20190318T120356_20190318T120456_026392_02F3FD_AF64

19. Frank dit :

Hi Olivier,
I have recently installed this script but every time I run it, it gives the error 'error with password file', but in the password file I have filled in my email and my PEPS password on the same line, so I do not understand where this error comes from.

20. Frank dit :

The problem I just mentioned has been solved. Now, however, using the examples you mentioned, mainly python ./peps_download.py -c S2 -l 'Toulouse' -a peps.txt -d 2015-11-01 -f 2015-12-01, it gives the error 'no product corresponds to selection data'. How is this solved?

Ce site utilise Akismet pour réduire les indésirables. En savoir plus sur comment les données de vos commentaires sont utilisées.