Rights statement: Under a Creative Commons license
Final published version, 2.14 MB, PDF document
Available under license: CC BY
Research output: Contribution to Journal/Magazine › Journal article › peer-review
Research output: Contribution to Journal/Magazine › Journal article › peer-review
}
TY - JOUR
T1 - Web technologies for environmental big data
AU - Vitolo, Claudia
AU - Elkhatib, Yehia
AU - Reusser, Dominik
AU - Macleod, Christopher J.A.
AU - Buytaert, Wouter
N1 - Under a Creative Commons license
PY - 2015/1
Y1 - 2015/1
N2 - Recent evolutions in computing science and web technology provide the environmental community with continuously expanding resources for data collection and analysis that pose unprecedented challenges to the design of analysis methods, workflows, and interaction with data sets. In the light of the recent UK Research Council funded Environmental Virtual Observatory pilot project, this paper gives an overview of currently available implementations related to web-based technologies for processing large and heterogeneous datasets and discuss their relevance within the context of environmental data processing, simulation and prediction. We found that, the processing of the simple datasets used in the pilot proved to be relatively straightforward using a combination of R, RPy2, PyWPS and PostgreSQL. However, the use of NoSQL databases and more versatile frameworks such as OGC standard based implementations may provide a wider and more flexible set of features that particularly facilitate working with larger volumes and more heterogeneous data sources.
AB - Recent evolutions in computing science and web technology provide the environmental community with continuously expanding resources for data collection and analysis that pose unprecedented challenges to the design of analysis methods, workflows, and interaction with data sets. In the light of the recent UK Research Council funded Environmental Virtual Observatory pilot project, this paper gives an overview of currently available implementations related to web-based technologies for processing large and heterogeneous datasets and discuss their relevance within the context of environmental data processing, simulation and prediction. We found that, the processing of the simple datasets used in the pilot proved to be relatively straightforward using a combination of R, RPy2, PyWPS and PostgreSQL. However, the use of NoSQL databases and more versatile frameworks such as OGC standard based implementations may provide a wider and more flexible set of features that particularly facilitate working with larger volumes and more heterogeneous data sources.
KW - web-based modelling
KW - big data
KW - web services
KW - OGC standards
U2 - 10.1016/j.envsoft.2014.10.007
DO - 10.1016/j.envsoft.2014.10.007
M3 - Journal article
VL - 63
SP - 185
EP - 198
JO - Environmental Modelling and Software
JF - Environmental Modelling and Software
SN - 1873-6726
ER -