Approaching Zero-ETL with FOSS

Leaving aside complexities like the Enterprise Integration Patterns, we can consider most integrations as a form of advanced ETL: Extract, Transform, and Load. We extract data from a data store or service. Then we transform it from an input to an output format. And finally we push or load that transformed data into some output channel. It is the easiness to connect with the input and output channels what makes the ETL need a proper integration framework. Complex integrations will combine these three steps differently. But the outcome is always to move information from one place to another, connecting different systems. Where the information may be a full dataset or just a triggered event. I already tackled the issue of choosing the right integration tool from an engineer’s perspective and what variables to take into account. But when we are talking about data science and data analysis, there is a requirement that goes on top of all of the previous: the accessibility and easiness of usage of the tool.

May 3, 2023 · 5 min · delawen

Cosas que aprendí en la GeoCamp

Año tras año, la GeoCamp se consolida como el geosarao imprescindible para hacer brainstorming sobre las últimas novedades en geocosas. Publicado originalmente en el blog de GeoInquietos Sevilla. La foto de grup del Geocamp ES 2016 a Barcelona #geocampes pic.twitter.com/H44a413tM0 — Geo Inquiets (@geoinquiets) 22 de octubre de 2016 Todo el mundo tiene sus cinco minutos de fama, incluso si no llevas nada preparado, raro será que no participes en alguna discusión. Battery just about to die so this concludes my #geocampes tweeting. Here’s the remaining lineup pic.twitter.com/3eRxBt5NsF — Ed Freyfogle (@freyfogle) 22 de octubre de 2016

November 13, 2016 · 3 min · delawen

JIIDE 2015 - Sevilla

This post was originally posted on the blog of a former company. But since they have decided to violate my authorship rights, here is a copy of it. Last week I attended the JIIDE conference, that took place here in Sevilla. This is the official conference for both portuguese and spanish spatial data infraestructures. The presentations were diverse and rich in content and there were working groups for INSPIRE and conformance running in parallel. Trends on GeoSpatial You could see some trends in how SDIs are evolving through all the Iberian Peninsula. Geograma explained to us that hiding data behind paywalls or registering sites makes us less compliant. But on the other hand, maybe it doesn’t matter because as José Fernández ( IECA) showed us, data is going more and more open and free. Why should someone pay for data generated on a public administration? It has already been payed by taxes and a paywall is just another stone on the way of generating added value to the data. And above all this, every country has a different payment and access system, so it is virtually impossible to query the same data on different countries easily, which was one of the goals for INSPIRE.

November 13, 2015 · 4 min · delawen

SIGLibre 9, Girona

This post was originally posted on the blog of a former company. But since they have decided to violate my authorship rights, here is a copy of it. The 9th SIGLibre conference in Girona starts with bad news: no wifi. But that didn’t stop us to talk about geo free stuff. SIGLibre Topics Almost all the plenary talks touched topics like smart citizens, open data, crowd data, crowd sourcing,… It is quite clear this is an emerging subject that we are going to use more and more. But, is this really a business market or is this just something to research and contribute to the community? ...

April 22, 2015 · 6 min · delawen

Jornadas SIGLibre Girona VII

Estas últimas Jornadas SIGLibre de Girona han girado en torno a servicios cloud y datos abiertos. Desde el apoteósico inicio con las ponencias plenarias, con parte destacada de Sergi Morales ( ExportosenTI), el resto de las charlas han ido rodando todas en el mismo tema. Algunas quizás un poco más críticas, como F. Puga desde CartoLab cuando nos pidió que no olvidáramos que no todo el mundo tiene acceso global a internet, y que muchos millones de personas, las cuales no tienen nuestro nivel tecnológico, también tienen necesidades GIS. ...

March 8, 2013 · 5 min · delawen

What is GeoNetwork?

GeoNetwork is a server side application that allows you to maintain a geographic referenced metadata catalogue. This means: a search portal that allows to view metadata combined with maps. GeoNetwork logo Based on Free and Open Source Software, it strictly follows different standards for metadata, from Inspire to OGC. It implements the CSW interface to be able to interact with generic clients looking for data. It also has built-in harvesters to connect to other servers and populate data. ...

October 28, 2012 · 1 min · delawen

The importance of open data

I’ve been thinking for a while about writing about the importance of open data, but is with the advertising given to Google Map Maker when I really understood the urgency of the matter. Can you imagine a country with so poor geographic data that even the government doesn’t known which cities and towns do they have? How could they invest on roads, literacy, drinking water or even know that there are people who live there? How could they collect taxes or… count votes in elections!? Can you imagine that a battalion of soldiers use maps that are wrong and establish a base in the nearest country? An absurdity that happened recently on the border between [Nicaragua and Costa Rica](https://www. elmundo.es/america/2011/04/19/noticias/1303174548.html), which almost causes an international conflict. ...

April 21, 2011 · 4 min · delawen

FOSS4G 2015 - Seoul

This post was originally posted on the blog of a former company. But since they have decided to violate my authorship rights, here is a copy of it. Last week I had the privilege to attend the main osgeo conference: the FOSS4G. This time it took place on Seoul, Korea. Exotic place I strongly recommend to visit, but better to focus on non-cultural surprises on this post. It’s impossible to write everything on a single blog post, as it was impossible to assist to all the interesting parallel threads that run on those short five days. But let me guide you through my steps so I can share part of my experience until we get access to the full videos of that awesome week.

7 min