Regional Datasets

1. Explanation of Directory Structure

Regional datasets can be found in: /esd/esd/data/atmospheric_data/gridded/regional/
The contained subfolders (one for each dataset) contain the higher resolution (0.5°x0.5° or higher) data, but geographical coverage varies between the datasets. In each data product folder you will find subfolders [dataset version]/[data]/[temporal resolution]/[2d surface or 3d variables] where the actual data is kept and [dataset version]/[doc] where you will find the documentation. This includes the official documentation and publications as well as ESD notes, which sum up everything briefly (what variables are available, what has been downloaded at Tübingen, when, from where, etc.). This summary should provide you with everything you need to know if you're unsure about which dataset you want to use.

2. Use of Datasets

  1.   When using these data, please do not write any derivatives or output in the source folders. They serve as a source only.  

  2. If you are interested in variables that were not downloaded (or a different version of the datasets), be consistent with the existing data structure and naming, update the documentation and make sure others have permissions to use it using the command chmod -R 777 [name of folder containing new files]. If you are not sure, talk to Sebastian. This way, we can keep data structure and documentation clean and avoid redundancy.

  3. Please consult the websites of the dataset publishers(found in the ESD documentation) to see the terms of use and find out how they would like their work to be acknowledged.

3. Processing Scripts

Python template for dataset processing (interfaces with bash, requires CDO, NCO and NCL):

 

prep.py

 

This will unzip files (as downloaded), combine files and creates means where neccessary, interpolate to desired resolution (for comparison with model output), saves the new files, rezip originals and removes temporary files. 

NOTE:
The NARR dataset requires interpolation with NCL by using the following script (paths are passed from prep.py):

 

narr_rw.ncl