Approaching Zero-ETL with FOSS

Leaving aside complexities like the Enterprise Integration Patterns, we can consider most integrations as a form of advanced ETL: Extract, Transform, and Load. We extract data from a data store or service. Then we transform it from an input to an output format. And finally we push or load that transformed data into some output channel. It is the easiness to connect with the input and output channels what makes the ETL need a proper integration framework.

Complex integrations will combine these three steps differently. But the outcome is always to move information from one place to another, connecting different systems. Where the information may be a full dataset or just a triggered event.

I already tackled the issue of choosing the right integration tool from an engineer’s perspective and what variables to take into account. But when we are talking about data science and data analysis, there is a requirement that goes on top of all of the previous: the accessibility and easiness of usage of the tool.

Continuar leyendo “Approaching Zero-ETL with FOSS”

No Code Integrations

Integration is mostly about being everywhere and nowhere at once: interoperability without a clunky user interface or spaghetti code. Can we get no code or low code integrations?

On this article we will explore how to do no code and low code integrations based on Apache Camel.

“Every paradigm including data flow, programming by example, and programming through analogies brings its own set of affordances and obstacles.”

Alexander Repenning – DOI reference number: 10.18293/VLSS2017-010

We are going to use Kaoto, which just made its 1.0.0 release. On this release, the Kaoto team has focused on the no-code graphical canvas to make sure the user experience is as smooth as possible.

Continuar leyendo “No Code Integrations”

Bungee jumping into Quarkus: blindfolded but happy

A year ago I started with a couple of friends a new project based on Quarkus to create a visual editor for integrations called Kaoto.

As responsible of the backend side, I obviously chose Java to do it. Coming from the Java 8 world with shy traces of Java 11, I decided to jump directly to Quarkus on Java 17 (unstable at the time) with Reactive and explore the serverless possibilities while, at the same time, keep the over-engineering and the over-fanciness of new features as reasonable as possible.

On this article I will discuss the good and the bad of this experience. I am not a Quarkus developer, I am a developer that used Quarkus. And as any average developer that starts with a new technology, I obviously skipped the documentation and just bungee jumped into the framework, blindfolded and without safe nets.

Continuar leyendo “Bungee jumping into Quarkus: blindfolded but happy”
es_ESEspañol