Approaching Zero-ETL with FOSS

Leaving aside complexities like the Enterprise Integration Patterns, we can consider most integrations as a form of advanced ETL: Extract, Transform, and Load. We extract data from a data store or service. Then we transform it from an input to an output format. And finally we push or load that transformed data into some output channel. It is the easiness to connect with the input and output channels what makes the ETL need a proper integration framework. Complex integrations will combine these three steps differently. But the outcome is always to move information from one place to another, connecting different systems. Where the information may be a full dataset or just a triggered event. I already tackled the issue of choosing the right integration tool from an engineer’s perspective and what variables to take into account. But when we are talking about data science and data analysis, there is a requirement that goes on top of all of the previous: the accessibility and easiness of usage of the tool.

May 3, 2023 · 5 min · delawen
imagen

No Code Integrations

On this article we will explore how to do no code and low code integrations based on Apache Camel. “Every paradigm including data flow, programming by example, and programming through analogies brings its own set of affordances and obstacles.” Alexander Repenning - DOI reference number: 10.18293/VLSS2017-010 We are going to use Kaoto, which just made its 1.0.0 release. On this release, the Kaoto team has focused on the no-code graphical canvas to make sure the user experience is as smooth as possible.

April 24, 2023 · 5 min · delawen
unikitty

In the FOSS trenches

After almost 20 years of being involved in the Free and Open Source Software (FOSS) community, and having gone through different associations and foundations, I would like to give my perspective on its sustainability. I have seen how companies get closer and further from FOSS as they evolve, and how different FOSS entities have overcome challenges. This is not a light matter and the contents of this article are not only opinionated, but a mere scratch on the surface. My intention here is merely to try to open a debate I feel is stagnant. ...

November 30, 2022 · 18 min · delawen
no-code

Kaoto: Integrate without limits

I would like to present you with an ETL and integration editor Rachel and I have been working on for the past year with the initial help of Zineb: Kaoto. What is Kaoto? Kaoto is an integration editor to create and deploy integrations in a low-code way and no-code way based on Apache Camel. It combines a source code editor and a drag and drop graphical space synchronized with each other. It can be run both as standalone and as a service (SaaS). With the no-code mode, the user can build the entire integration orchestration with the drag and drop function. Kaoto has a step catalog with a list of all available building blocks that the users may want to transform data or integrate with services.

April 2, 2022 · 3 min · delawen

Are online events the new normal?

Some of you may know I am going to co-chair the next FOSS4G. ( Come Hell or High Water, but we are going to have a FOSS4G next year!) I have also participated on the organization of the last ApacheCON and some other minor online events. I have also spoken on many online events by now.

October 11, 2020 · 16 min
IMG_20201006_132333

What's all this integration agitation, anyway?

You may have heard any or all of these keywords before: middleware, integration, orchestration. And you may be wondering why and when to use them. Take a walk with me to understand when and how integration frameworks are useful. Imagine you are in charge of solving a new need of your company. There is no complete software stack for what you need. You will have to involve your team to create something new. Even if you reuse some components, you have to make them interact and talk to each other. You are an experienced software engineer and have solved previously many of the requirements with some components you are already familiar with. But now you have to orchestrate all these components together and make them work like a clock. Now you need a proper integration. You want all of them to cooperate smoothly in your architecture.

October 6, 2020 · 8 min · delawen

From Cat to Hat

On June I joined Red Hat as a Senior Software Engineer.

July 19, 2019 · 4 min

Cosas que aprendí en la GeoCamp

Año tras año, la GeoCamp se consolida como el geosarao imprescindible para hacer brainstorming sobre las últimas novedades en geocosas. Publicado originalmente en el blog de GeoInquietos Sevilla. La foto de grup del Geocamp ES 2016 a Barcelona #geocampes pic.twitter.com/H44a413tM0 > > — Geo Inquiets (@geoinquiets) 22 de octubre de 2016 Todo el mundo tiene sus cinco minutos de fama, incluso si no llevas nada preparado, raro será que no participes en alguna discusión. Battery just about to die so this concludes my #geocampes tweeting. Here’s the remaining lineup pic.twitter.com/3eRxBt5NsF > > — Ed Freyfogle (@freyfogle) 22 de octubre de 2016

November 13, 2016 · 3 min

FOSS4G Argentina 2016 (Buenos Aires)

This post was originally posted on the blog of a former company. But since they have decided to violate my authorship rights, here is a copy of it. English version is lost. Photo of an asado Este año tuve el privilegio de poder asistir a la FOSS4G Argentina 2016, que se celebró en el IGN de Buenos Aires. Qué puedo contaros de Argentina que no sepáis ya. https://www.instagram.com/p/BD1thIpqcTI ...

April 27, 2016 · 5 min

GeoNetwork from Scratch II : Attack of the IDEs

This post was originally posted on the blog of a former company. But since they have decided to violate my authorship rights, here is a copy of it. We have already seen how to compile and run a basic GeoNetwork instance. Although we know that real developers will probably skip this step too, for new developers in GeoNetwork, it will be relief to have an IDE to work with. I know that many GeoNetwork developers use NetBeans or Intellij but as I am used to work with Eclipse, that’s what we are going to explore on this post. First of all: Eclipse has better support for Maven projects on each version. So, to avoid headaches, just download the latest eclipse available.Eclipse has many installer tutorials, so I won’t stop here explaining how to run eclipse. I will just assume you know how to do it. To run GeoNetwork from eclipse is very very easy. Just right click on the Package Explorer view to import -> As Maven Project over the folder you already had cloned on the last post:

November 16, 2015 · 3 min