Tag: integration

No Code Integrations

Integration is mostly about being everywhere and nowhere at once: interoperability without a clunky user interface or spaghetti code. Can we get no code or low code integrations?

On this article we will explore how to do no code and low code integrations based on Apache Camel.

“Every paradigm including data flow, programming by example, and programming through analogies brings its own set of affordances and obstacles.”

Alexander Repenning – DOI reference number: 10.18293/VLSS2017-010

We are going to use Kaoto, which just made its 1.0.0 release. On this release, the Kaoto team has focused on the no-code graphical canvas to make sure the user experience is as smooth as possible.

Apache Camel

As we have discussed previously on this blog, Apache Camel is an integration framework. This means Camel will help you orchestrate and compose your services in a decoupled way.

Camel has its own Domain Specific Language (DSL) that translates a simple Camel route into code you can run.

Simplified diagram on how Apache Camel works
Simplified Diagram of Apache Camel converting DSL to integrations

Camel Quarkus

Whether we decide to write our camel routes in Java, Javascript, Groovy, YAML, XML,… In Camel they all get transpiled to Java code for deployment. While usually this is not a problem by itself, it is true that Java uses sometimes a bigger memory footprint that would be needed.

But we can overcome this with the Quarkus build for Apache Camel. Camel Quarkus looks exactly the same from the user or developer perspective. But it brings all the developer joy of native builds and deployments to the integration world. This is done with the Quarkus extensions on Apache Camel.

Simplified diagram on how Apache Camel Quarkus works
The Quarkus version of Camel allows you to deploy both using native and Quarkus runtimes

Camel K and Kamelets

But Camel offers much more than just route integrations. We also have Kamelets that aim to simplify our integration definitions. These kamelets, or camel route snippets, act as meta-steps in an integration, hiding complexities in otherwise pretty simple orchestrations.

We also have Camel K to help in our integration management. It is a lightweight integration framework that runs natively on Kubernetes. It has been specifically designed for serverless and microservice architectures.

Simplified diagram on how Apache Camel K works
Simplified diagram on how Camel K wraps Apache Camel to provide k-native deployments.

No Code Integrations with Kaoto

The issue of making integrations frameworks accessible and easy to use is not new. There was been many different approaches to the same problem.

This is where we decided to try a new approach and create Kaoto as a way of creating no code and low code integrations.

The obvious first step on this diagram for us was some kind of Visual Editor at the beginning of the previously defined workflows, that would allow people to integrate without writing a single line of code.

Simplified diagram on how Kaoto works providing no code and low code integrations.
Simplified diagram on how a low code/no code editor works generating the source code graphically.

The primitives or building blocks of Kaoto are usually steps in the Apache Camel integration DSL: Kamelets, Camel Components, or Enterprise Integration Patterns.

We wanted Kaoto to have a simple user interface that provides building-blocks in a drag-and-drop style manipulation. But we wanted users to be able to manipulate the source code to be as transparent as possible about what they are building.

“Well, this is all fine and well, but the problem with visual programming languages is that you can’t have more than 50 visual primitives on the screen at the same time. How are you going to write an operating system?”

Peter Deutsch
What does Low Code look like?

Low code allows the user to view and interact with the source code to deploy. At the same time, the user will be able to focus on the features being implemented without really knowing how to write the source code. The source code may look as an adjacent add-on or a guide to help new users get familiar with concepts.

Good low code editors will also show some kind of visual aid to understand what the code is implementing. On our case, Kaoto has a visual workflow showing how the components get connected to each other.

Example of low code integrations.

On the above example, the user starts by writing part of the code on an empty template. Once the user stops writing, Kaoto fills the gaps in the code to make it a valid source code. Also, the user can drag and drop on the graphical side to build the integration, while the source code gets updated with the changes.

What does No Code look like?

On a no code solution, there is no need to interact or even see the source code at any point. The user can focus on bringing integration capabilities to their architecture without worrying about implementation details.

How to build an integration with no code

You can see on this video how the user can build the integration and deploy it just by using the graphical space. Drag and drop steps, select steps from a catalog, setup the configuration properties in a HTML forms and clicking the deploy button.

You see full examples of how to use Kaoto in the workshop section you can test using any of the quickstart options.

Kaoto: Integrate without limits

I would like to present you with an ETL and integration editor Rachel and I have been working on for the past year with the initial help of Zineb: Kaoto.

What is Kaoto?

Kaoto is an integration editor to create and deploy integrations in a low-code way and no-code way based on Apache Camel. It combines a source code editor and a drag and drop graphical space synchronized with each other. It can be run both as standalone and as a service (SaaS).

With the no-code mode, the user can build the entire integration orchestration with the drag and drop function. Kaoto has a step catalog with a list of all available building blocks that the users may want to transform data or integrate with services.

The source code will be available for users curious to know what they are going to deploy. But in no case they have to understand or even see that code.

Example of building block drag and drop

With the low code mode, users can learn how to create integrations and, at the same time, control what they are deploying. They can use both the drag and drop and the source code editor, that will autocomplete what the user won’t or don’t know how to write. Meanwhile the graphical space will show the integration being built and the drag and drop will still be available for them to complete or accelerate the development.

Example of low code integrations.

Kaoto can help users start with Apache Camel and slowly build their knowledge. All the source code generated is clean and viewable in any IDE.

Customizing Kaoto

Each building block type can have its own microfrontend. This is useful when you add your own building blocks to your Kaoto instance. But it can also help adapt Kaoto to different kinds of users, hiding or extending certain details important for each use-case. Extensions can be manuals and helpers for Kaoto.

When used as a service, the extensions and the list of available building blocks are settings that can be stored in the cloud. Therefore, administrator users can modify this configuration, which will refresh live in the users playgrounds. In addition, as this configuration can be in the cloud, different users can share the configuration. This can help organizations accommodate different groups of users, offering different configurations depending on the profile.

What is on the roadmap?

We started the development focused on Kamelet Bindings, but we soon realized we could add more languages. Edition of Apache Camel routes (in yaml form) and Kamelet definitions are next in development queue. We are also working on translating from other integration languages to Camel DSL. This can help users migrate to Apache Camel.

We will soon have one-click support for cloud-native Apache Camel deployments via Camel-K. Matej is close to having an operator for Kubernetes clusters which will simplify even more the installation of Kaoto in the cloud.

You can quickly test it via docker as described on the quickstart. Make sure your docker images have access to internet to be able to access the default remote configuration!

You can find more information on the webpage of the project.

en_GBEnglish (UK)
es_ESEspañol en_GBEnglish (UK)