A simple walkthrough to download datasets from NASA’s earth datasets (GES DISC)
Download single or multi dataset links from a single text file.
NASA’s GES DISC(Goddard Earth Sciences Data and Information Services Center) is a very popular platform among people who use satellite data of creating, testing or implementing their ML algorithms in the SPACE tech domain.
The NASA GES DISC is one of the twelve Distributed Active Archive Centers (DAACs) established as part of the Earth Observation System Data and Information System (EOSDIS). The Earth Science Data and Information System (ESDIS) Project, which oversees the EOSDIS, has directed GES DISC to establish and sponsor a User Working Group (UWG).
The purpose of the GES DISC UWG is to engage science users in the planning, development, and operation of GES DISC, and to represent the science user community in reviewing and guiding GES DISC evolution in order to better serve the community into the future. (courtesy disc.gsfc.nasa.gov)
Disclaimer: The following is for windows OS
Firstly, you should have chocolatey installed. It can be installed by the following code.
powershell.exe Set-ExecutionPolicy Bypass -Scope Process -Force; [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072; iex ((New-Object System.Net.WebClient).DownloadString(‘https://chocolatey.org/install.ps1'))
Just copy and paste the above code on your command prompt opened as administrator and let it download.
To install wget, open a new cmd window in admin mode and type the following.
choco install wget
If you want to upgrade the wget package, you may use,
choco upgrade wget
Now as we have wget in our system we can continue with downloading the dataset.
The first step is to make sure you have setup your Earthdata account. Select the data that you want to download (you may either search or browse the different datasets present). For the walkthrough I am going to use the OCO-2 Level 2 bias-corrected XCO2 dataset.
Click on the button which says “Subset/Get Data”.
In the new applet opened, you may chose the “Download Method”(Original files/File Subset) depending on your requirement.
Select the required Date range.
Select the region/area you want the data of. (you may use the drag option or enter the coordinates of the region of which u need the data)
Do keep in mind, longer the date range or larger the region size, higher will be the size of data.
And then click on “Get data”
The links will be generated and now we can download the links by clicking on the “Download links list”.
Open command prompt as admin, navigate to your C:\ drive and enter the following set of codes in the same order(a notepad will be opened you may close it).
NUL > .urs_cookies
Now navigate into the folder where you want the data to be downloaded and paste the following code changing:
- <yourusername> to your Earthdata account username
- <yourpassword> to your Earthdata account password
- <locationofthetextfile> to the location of text file previously downloaded
wget — load-cookies C:\.urs_cookies — save-cookies C:\.urs_cookies — auth-no-challenge=on — keep-session-cookies — user=<yourusername> — password=<yourpassword> — content-disposition — no-check-certificate -i <locationofthetextfile>
here is an example:
All the all files in the downloaded list of links will be downloaded.
There you go. The requested data has been downloaded. Have fun visualizing these data. All the best!