WCS-Server_Introduction

WCS Server for CF-NetCDF
Mitglied in der Helmholtz-Gemeinschaft
An Overview
AQCoP Meeting, August 2011 | M. Decker, M. Schultz, K. Hoijarvi, R.B. Husar
Introduction

Mitglied in der Helmholtz-Gemeinschaft
Functioning software package that implements a WCS
compliant data server delivering NetCDF files
 Based on Python
 Runs on Windows and Linux
 Supports WCS 1.1.2 Protocol
 Supports NetCDF 3 and NetCDF 4 large file formats
 Supports station data from SQL database
14. Februar 2017
2
WCS gives you...

Metadata about...



Extraction of...



Mitglied in der Helmholtz-Gemeinschaft


the service
available datasets
a rectangular area (boundingbox)
individual variables (fields)
individual points or ranges in time
(subsetting of other dimensions)
...the foundation for using interoperable web applications
14. Februar 2017
3
Mitglied in der Helmholtz-Gemeinschaft
High-Level view of WCS
14. Februar 2017
4
Mitglied in der Helmholtz-Gemeinschaft
Example metadata response (GetCapabilities)
14. Februar 2017
5
Software Dependencies
Operating system
Python
Python web framework
Python XML parser
Python numerical package
Logging Backend for web.py
(optional)
Mitglied in der Helmholtz-Gemeinschaft
Python netcdf interface
Windows (>= XP)
GNU/Linux
>= 2.6 (but not 3 - yet)
web.py (http://webpy.org/)
Lxml (http://lxml.de/)
NumPy (http://numpy.scipy.org/)
WsgiLog
(http://pypi.python.org/pypi/wsgilog/)
[currently part of
server package]
PyNIO
(http://www.pyngl.uc
ar.edu/Nio.shtml)
More??
14. Februar 2017
6
Mitglied in der Helmholtz-Gemeinschaft
Server Concept
14. Februar 2017
7
Server Directory Structure
OWS
datafed
NetCDF interface code
web
web interface code
Mitglied in der Helmholtz-Gemeinschaft
static
14. Februar 2017
web home (index.html, ...)
cache
temp dir
testprovider
example data dir
...
other data dirs
8
Development

Code is OpenSource under The MIT License
 Collaboration Tools:



Esipfed-Wiki: documentation
SourceForge project page: forum, bugtracker
DARCS distributed version control system: code

Mitglied in der Helmholtz-Gemeinschaft
(currently) two main repositories: WUSTL and FZ Juelich
 easy to set up your own full-featured repository
 further information available on the SourceForge project page

Test-driven development process


unit tests: OWS/test/unit.py
integration tests: OWS/test/integ.py
14. Februar 2017
9
Getting Started I: Installation

install 3rd party software (see dependencies)

get WCS server archive from SourceForge project page


Mitglied in der Helmholtz-Gemeinschaft

put archive contents into sensible location for server operation


Windows: win32-ows-*.zip
Linux: linux-ows-*.tar.gz
for example /usr/local/ows (Linux) or C:\ows (Windows)
add server dir to PYTHONPATH environment variable
14. Februar 2017
10
Getting Started II: test it

example data dir included: testprovider


create NetCDF test cubes


Mitglied in der Helmholtz-Gemeinschaft
call web/owsadmin.py wcs_prepare testprovider
start the server:


call web/static/testprovider/create_all.py
extract metadata for operation:


contains example files (index.html, wcs_capabilities.conf)
that you can adapt for your own data
web/ows.py 1080
check if it works:

http://localhost:1080/testprovider
14. Februar 2017
11
Getting Started III: serving your data (NetCDF)

create a data directory in web/static/

populate it with (CF-compliant) NetCDF files
 configure metadata



Mitglied in der Helmholtz-Gemeinschaft

run metadata extractor:



create wcs_capabilities.conf
create human-readable index.html(optional, but encouraged)
you can use the example files from testprovider
owsadmin.py wcs_prepare <data dir name>
needs to be re-run after data dir content has been updated
also adapt the global index.html in web/static
14. Februar 2017
12
Getting Started IV: serving station data (SQL)

create a data directory in web/static/

create SQL views to:



edit configuration:



Mitglied in der Helmholtz-Gemeinschaft
location data: loc_code, lat, lon + optional
parameter data: loc_code, lat, lon, datetime, parameter, flags

SQL database connection
location SQL view name + fields
time periodicity for each coverage
SQL data view name + fields for each parameter

configure metadata as with NetCDF data
The real work is done inside the SQL database when creatring SQL
views, the rest is just listing the names!
14. Februar 2017
13
Getting Started V: running the server

start the standalone server:


data will be available via WCS at

Mitglied in der Helmholtz-Gemeinschaft
ows.py [port number] (default port: 1080)
http://localhost:<port>/<data dir>?...
14. Februar 2017
14
Mitglied in der Helmholtz-Gemeinschaft
Thank you for your attention!
14. Februar 2017
15
Mitglied in der Helmholtz-Gemeinschaft
14. Februar 2017
16