cmiputil.esgfdatainfo module

Supporting module for esgfsearch.

This class holds and maintains search result of ESGF dataset obtained via RESTful API issued by esgfsearch.ESGFSearch.doSearch() method.

One instance of this class corresponds to one search result, and a list of instances is set as the esgfsearch.ESGFSearch.datainfo attribute.

This class also have some methods to access OPeNDAP catalog and retrieve additional information, search local (pre-downloaded) files to match the search result, etc.

Example

>>> from cmiputil import esgfdatainfo
>>> import urllib3, json
>>> keywords = {
... 'distrib':'true',
... 'type':'Dataset',
... 'format':r'application/solr+json',
... 'offset':0,
... 'replica':'False'}
>>> params = {
... 'experiment_id':'piControl',
... 'variable_id':'tas',
... 'table_id':'Amon',
... 'source_id':'BCC-CSM2-MR'}
>>> base_url = 'http://esgf-node.llnl.gov/esg-search/' 'search'
>>> params.update(keywords)
>>> http = urllib3.PoolManager()
>>> r = http.request('GET', base_url, fields=params)
>>> result = json.loads(r.data.decode())
>>> attrs = result['response']['docs'][0]
>>> dinfo = esgfdatainfo.ESGFDataInfo(attrs)
>>> dinfo.id
'CMIP6.CMIP.BCC.BCC-CSM2-MR.piControl.r1i1p1f1.Amon.tas.gn.v20181016|cmip.bcc.cma.cn'

Actually, doing search as above is done by esgfsearch.ESGFSearch.

class esgfdatainfo.ESGFDataInfo(attribs={})

Bases: collections.abc.MutableMapping

Holds and maintains search result of ESGF dataset obtained via RESTful API.

Among attributes obtained from one search result, you can access several useful ones via managedAttribs.

Since this class inherits MutableMapping ABC, you can access an instance of this class as mapping, such as datainfo['source_id'].

cat_url

URL of OPeNDAP catalog

data_url

URL of dataset

local_files

Paths of local file corresponding to the search result.

Parameters

attribs (dict) – attributes to be set, see setFrom():.

findLocalFile(base_dir)

Find local (pre-downloaded) files corresponds to the search result.

See Local data store section in esgfsearch.

getDDS()

Get OPeNDAP DDS (Dataset Descriptor Structure).

Must be called after getDataURL().

Example of DDS:

Dataset {
    Float64 lat[lat = 160];
    Float64 lat_bnds[lat = 160][bnds = 2];
    Float64 lon[lon = 320];
    Float64 lon_bnds[lon = 320][bnds = 2];
    Float64 height;
    Float64 time[time = 8412];
    Float64 time_bnds[time = 8412][bnds = 2];
    Grid {
     ARRAY:
        Float32 tas[time = 8412][lat = 160][lon = 320];
     MAPS:
        Float64 time[time = 8412];
        Float64 lat[lat = 160];
        Float64 lon[lon = 320];
    } tas;
} CMIP6.CMIP.MRI.MRI-ESM2-0.piControl.r1i1p1f1.Amon.tas.gn.tas.20190222.aggregation.1;

getDataURL(aggregate)

Get URL(s) of dataset by accessing the OPeNDAP Catalog.

Results are set as data_url

Parameters

aggregate (bool) – retrieve aggregated dataset, or not.

setFrom(attribs)

Set attributes from one ESGF RESTful API search result.

Parameters

attribs (dict) – attributes to be set.

property managedAttribs

dict of useful global attributes.