Leaving aside complexities like the Enterprise Integration Patterns, we can consider most integrations as a form of advanced ETL: Extract, Transform, and Load. We extract data from a data store or service. Then we transform it from an input to an output format. And finally we push or load that transformed data into some output channel. It is the easiness to connect with the input and output channels what makes the ETL need a proper integration framework.
Complex integrations will combine these three steps differently. But the outcome is always to move information from one place to another, connecting different systems. Where the information may be a full dataset or just a triggered event.
I already tackled the issue of choosing the right integration tool from an engineer’s perspective and what variables to take into account. But when we are talking about data science and data analysis, there is a requirement that goes on top of all of the previous: the accessibility and easiness of usage of the tool.
Continue reading “Approaching Zero-ETL with FOSS”
Año tras año, la GeoCamp se consolida como el geosarao imprescindible para hacer brainstorming sobre las últimas novedades en geocosas.
Publicado originalmente en el blog de GeoInquietos Sevilla.
Todo el mundo tiene sus cinco minutos de fama, incluso si no llevas nada preparado, raro será que no participes en alguna discusión.
Continue reading “Cosas que aprendí en la GeoCamp”
This post was originally posted on the blog of a former company. But since they have decided to violate my authorship rights, here is a copy of it.
Last week I attended the JIIDE conference, that took place here in Sevilla. This is the official conference for both portuguese and spanish spatial data infraestructures. The presentations were diverse and rich in content and there were working groups for INSPIRE and conformance running in parallel.
Trends on GeoSpatial
You could see some trends in how SDIs are evolving through all the Iberian Peninsula. Geograma explained to us that hiding data behind paywalls or registering sites makes us less compliant. But on the other hand, maybe it doesn’t matter because as José Fernández (IECA) showed us, data is going more and more open and free. Why should someone pay for data generated on a public administration? It has already been payed by taxes and a paywall is just another stone on the way of generating added value to the data. And above all this, every country has a different payment and access system, so it is virtually impossible to query the same data on different countries easily, which was one of the goals for INSPIRE.
Continue reading “JIIDE 2015 – Sevilla”