GeoNetwork from Scratch II : Attack of the IDEs

We have already seen how to compile and run a basic GeoNetwork instance. Although we know that real developers will probably skip this step too, for new developers in GeoNetwork, it will be relief to have an IDE to work with. I know that many GeoNetwork developers use NetBeans or Intellij but as I am used to work with Eclipse, that’s what we are going to explore on this post.

First of all: Eclipse has better support for Maven projects on each version. So, to avoid headaches, just download the latest eclipse available.Eclipse has many installer tutorials, so I won’t stop here explaining how to run eclipse. I will just assume you know how to do it.

To run GeoNetwork from eclipse is very very easy. Just right click on the Package Explorer view to import -> As Maven Project over the folder you already had cloned on the last post:

Import As Maven Project
Import As Maven Project
Import As Maven Project
Import As Maven Project

There is still something Eclipse does not support right about GeoNetwork: we have a classes folder that Eclipse tends to misconfigure. So, go to that folder, right click and remove as source folder. To do this, go to the “web-app” project and right click on src/main/webapp/WEB-INF/classes. Select Build Path > Remove From BuildPath.

Then, completely remove the folder from the source code. Don’t worry, it’s git, you can recover it later. You can also do this by right-clicking on the folder and selecting Delete. Yes, you are sure you want to delete folder “classes“.

Now, update as maven project right clicking on the project “web-app” and selecting Maven > Update Project …

update project

Once this finishes, you can restore the folder we previously removed. Go to the “web-app” project, right click on src/main/webapp/WEB-INF and select Replace With >HEAD Revision. Yes, you are sure.

Congratulations! You are ready to use Eclipse to modify GeoNetwork.

But wait, how do we run GeoNetwork inside Eclipse to be able to debug?

We have several approaches here. Remember the jetty command to run GeoNetwork from the console? It is available also inside Eclipse (right click on web-app and Run As > Maven Build) and you can add some maven variables to be able to run in parallel a debug watch to debug your code. You can also set up a Tomcat server inside Eclipse and run GeoNetwork from it. This second option is more easy for beginners, so that’s what we are going to do now.

First, you have to create a Tomcat server inside Eclipse. So, search for the “Servers” tab and right click on it. Select New > Server. You will see a windows offering different types of servers. We will select the Tomcat v.7.0 Server one. You will probably won’t have any server runtime environment configured for it, but you can “Add…” a new one. There are many tutorials[1][2] for this, so we won’t stop here.

On the following window, you can select which applications to run. Obviously, you choose the one called “web-app” and Finish.

Now, you will have a new Server on the Servers tab. select it (left-click) and click on the green arrow just on the top of that tab. You will see on the “Console” tab all the output of GeoNetwork starting up. Once it is started, you can enter GeoNetwork the same way as before, using http://localhost:8080/geonetwork

Have fun customizing GeoNetwork!

JIIDE 2015 – Sevilla

Last week I attended the JIIDE conference, that took place here in Sevilla. This is the official conference for both portuguese and spanish spatial data infraestructures. The presentations were diverse and rich in content and there were working groups for INSPIRE and conformance running in parallel.

You could see some trends in how SDIs are evolving through all the Iberian Peninsula. Geograma explained to us that hiding data behind paywalls or registering sites makes us less compliant. But on the other hand, maybe it doesn’t matter because as José Fernández (IECA) showed us, data is going more and more open and free. Why should someone pay for data generated on a public administration? It has already been payed by taxes and a paywall is just another stone on the way of generating added value to the data. And above all this, every country has a different payment and access system, so it is virtually impossible to query the same data on different countries easily, which was one of the goals for INSPIRE.

Transparency, interoperability, quality, conflation,… keywords through all the conference. As an example of conflation and reusability, IECA was created by the union of the geospatial information department and the statistics department of the government of Andalucía. This allows them to localize statistic data that, once the privacy details are removed, can be easily shared. Creative Commons is the main license for all their products.

On a statal level, now we have the CNIG, who unifies all the data from Spain and allows us to download (or buy) data. Here, the map is not the central issue, but just another product you can use.

And still, there are many things INSPIRE has yet to solve. There are a lot of abstract requirements the nodes are not sure how to solve. All data has to have quality metadata associated to it, but, is there any quality minimum required for the data? How close the scales should be? What precision? And above all, how are the different public administrations supposed to handle all this without specific financing from Europe? Or what is worst: why do Europe ignore the conclusions reached by several working groups on different countries? Why reinventing the wheel?

There was another subject running through all the conference: why do SDIs have less users than open data portals? Is it because the type of data? Is it because we don’t focus on usability? Why do they choose data from worse quality (or not government certified)? It looks like we have to work more on usability and user interaction.

Should SDI focus on developing applications around the data? Or should they just focus on being a data repository third party companies can query to generate added value? Should we merge with the opendata portals even if that means lose part of the focus on spatial data?

Javier López, explained the problem about persistent identifiers. We have to assume that the entities generating data will not be persistent. But their data should survive those entities and we should be able to trace back who created the data and who have been maintaining it. How to achieve this without being too dependent on some specific platform? How to create a standard that survives through the years?

We also had the visit of Rodrigo Barriga Vargas (IPGH) who told us about GeoSUR, an initiative to create, conform and share quality spatial data in America. He told us how lucky we are to have INSPIRE as a gubernamental initiative to force us to follow standards.

But the best thing was to see how GeoNetwork is being used more and more and we have happy users advocating that it is the only SDI that makes sense.

GeoNetwork From Scratch I : The Phantom Catalog

GeoNetwork never has been an easy software to work with. But specially after the 3.0 version release, many things have changed. On this series of posts we will try to help new developers start with GeoNetwork.

The source code of GeoNetwork is available on a public repository on Github. This means that you can clone, fork and propose pushes of your custom changes. If you are not familiar with repositories of code or git, you should check this quick manual.

GeoNetwork is built using maven version 3+. It is written on Java and requires version 7 or more. It works both with OpenJDK or the Oracle version.You will need git and maven installed on your local machine to work. There are several ways to install this on your local machine; for example if you have a Debian based OS (like Ubuntu), you can install them with just this command:

sudo apt-get install maven git

Make sure you installed maven version 3!!

$ mvn –version
Apache Maven 3.2.1 (ea8b2b07643dbb1b84b6d16e1f08391b666bc1e9; 2014-02-14T18:37:52+01:00)
Maven home: ….

Remember that this will also install java on your system. You can check that the version is the right one with the following command:

java -version

So, the very first step once you have your environment set up is clone the GeoNetwork repository on your local machine. That can be done on the command line using the following command inside an empty folder where the source code will be populated:

cd yourEmptyFolder
git clone
git submodule init
git submodule update

As you can see, all the source code shown on github is also available on your local machine now.

The source code of GeoNetwork is split on several smaller maven projects. To run GeoNetwork, you have to build all of them and run the project named “web“. If you are familiar to maven, you will probably have guessed that you have to run a package install command on the root folder of GeoNetwork source code. But if you try that, maven will warn you that for building GeoNetwork you need more memory than the default memory provided to maven. This means, you will have to export the maven options to increase the memory like this:

export MAVEN_OPTS=”-Xmx512M -XX:MaxPermSize=256M”

At this point we are not interested in running the tests, so you can skip them using the parameter “-DskipTests”:

mvn package install -DskipTests

At the end of this build (which can take long, depending on your network connection, as it has many third party libraries), you will see something like this:

[INFO] ————————————————————————
[INFO] Reactor Summary:
[INFO] GeoNetwork opensource ……………………….. SUCCESS [ 3.111 s]
[INFO] common utils ……………………………….. SUCCESS [ 13.678 s]
[INFO] Caching xslt module …………………………. SUCCESS [ 7.607 s]
[INFO] ArcSDE module (dummy-api) ……………………. SUCCESS [ 7.860 s]
[INFO] GeoNetwork domain …………………………… SUCCESS [ 33.785 s]
[INFO] Oaipmh modules ……………………………… SUCCESS [ 0.833 s]
[INFO] GeoNetwork Events …………………………… SUCCESS [ 0.654 s]
[INFO] GeoNetwork schema plugins ……………………. SUCCESS [ 4.646 s]
[INFO] GeoNetwork schema plugins core ……………….. SUCCESS [ 5.338 s]
[INFO] GeoNetwork schema plugin for ISO19139/119 standards SUCCESS [ 8.432 s]
[INFO] GeoNetwork core …………………………….. SUCCESS [ 16.304 s]
[INFO] GeoNetwork schema plugin for Dublin Core records retrieved by CSW SUCCESS [ 5.031 s]
[INFO] GeoNetwork schema plugin for Dublin Core standard . SUCCESS [ 8.419 s]
[INFO] GeoNetwork schema plugin for ISO19110 standard …. SUCCESS [ 3.627 s]
[INFO] GeoNetwork CSW server ……………………….. SUCCESS [ 5.546 s]
[INFO] GeoNetwork harvesters ……………………….. SUCCESS [ 3.888 s]
[INFO] GeoNetwork health monitor ……………………. SUCCESS [ 2.489 s]
[INFO] GeoNetwork services …………………………. SUCCESS [ 8.597 s]
[INFO] Geonetwork Web Resources 4 Java ………………. SUCCESS [ 5.261 s]
[INFO] Cobweb Customizations ……………………….. SUCCESS [ 4.226 s]
[INFO] GeoNetwork INSPIRE Atom ……………………… SUCCESS [ 3.990 s]
[INFO] Tests for schema plugins …………………….. SUCCESS [ 2.334 s]
[INFO] GeoNetwork user interface module ……………… SUCCESS [ 35.356 s]
[INFO] JS API and Service documentation ……………… SUCCESS [ 21.203 s]
[INFO] GeoNetwork web client module …………………. SUCCESS [ 47.484 s]
[INFO] GeoNetwork Web module ……………………….. SUCCESS [ 48.490 s]
[INFO] GeoNetwork E2E Javascript Tests ………………. SUCCESS [ 1.645 s]
[INFO] ————————————————————————
[INFO] ————————————————————————
[INFO] Total time: 02:19 min (Wall Clock)
[INFO] Finished at: 2015-07-17T10:36:43+01:00
[INFO] Final Memory: 232M/441M
[INFO] ————————————————————————

This will generate a war file you can use in any Java Application Container (server) like Tomcat on web/target/geonetwork.war

Congratulations! You are ready to run GeoNetwork. To do this, just go to the web folder and run jetty in there:

cd web; mvn jetty:run

After jetty starts, you can see your running GeoNetwork by opening a browser and enter to http://localhost:8080/geonetwork

Continue in Attack of the IDEs

FOSS4G 2015 – Seoul

Last week I had the privilege to attend the main osgeo conference: the FOSS4G. This time it took place on Seoul, Korea. Exotic place I strongly recommend to visit, but better to focus on non-cultural surprises on this post.

It’s impossible to write everything on a single blog post, as it was impossible to assist to all the interesting parallel threads that run on those short five days. But let me guide you through my steps so I can share part of my experience until we get access to the full videos of that awesome week.

First there were the workshops. I have to confess something: my first options for workshops were cancelled. But it doesn’t matter, as I had problems deciding on first place. So, I started with the WS02 workshop:

Exploring the Sensor Observation Service Standard Enhanced by IstSOS Special Features

Let me summarize it: if you want to work with sensors, take a very close look at IstSOS. Combining PostGIS, Python, GDAL and Apache you get not only a complete sensor service, but a complete sensor data management system. Easy to use, easy to install, easy to everything!


Quality Assurance is integrated on IstSOS to make sure the sensor doesn’t go mad. This process are sometimes done asynchronously to conflate your own sensor data with other sources. It doesn’t just return a basic boolean, but a statistic value of confidence based on previous data.

Another interesting feature is the virtual sensors, which allows you to create your “own” “sensors” based on data from real physical sensors. Useful to conflate data and offer it on a unique endpoint as sensor.

As if this workshop wasn’t good enough, we got some real arduino and sensors to play with them.


Once we finished this amazing workshop, I went to assist Florent Gravin on the GeoNetwork 3 workshop in the afternoon:


WS08 Build your own data portal using GeoNetwork 3

Besides some Windows issues (surprise!), I think that the workshop was very successful. Assistants not only built their own user interface style for GeoNetwork, but asked us about advanced features and how to implement them.


As Florent said: “We are the experts of GeoNetwork, so if you have any questions, it is now or never.” And we had questions asked.

Jet-lagged day finished after this workshop. So we went to the hotel to try to rest a bit more and start with renewed strength the following day, which I started with:

WS19 Beyond GeoServer Basics

If you already run a GeoServer, most of this workshop is already known to you. But if you don’t… bad you missed it! There were a lot of fancy features to play with, like regexp on parameters for sql layers (surprised that very few knew what regexp meant!).

And, of course, WPS already useful in production now. You can even use WPS inside SLD definitions!

After refreshing knowledge on GeoServer, I went for the last workshop:

WS21 OpenSource 3D GIS

Yes, you are right, 3D GIS has too much hype right now. But Oslandia did a great job on this workshop. If you don’t have experience with 3D, I strongly advise you to try to follow the workshop, posted on github.


We even had some docker introduction. And the second day was finished, but we still had energy to went for the last one. Or some more beers:


I was already on the first slot of the conference, during the Opening Ceremony, when I first noticed that we talked too much about open instead of free.


Besides that, Venkatesh made very clear that we need more Geo4All, specially on developing countries.

Very interesting the national SDI of Korea presentation about the workflow of open source. While I don’t fully agree with their vision, it is always refreshing to see new perspectives on this subject. Maybe we should make more clear that free software can have a professional customer support as good as any closed source software. Probably even better in most cases, as there are more companies that can offer it.

Now the conference divided on so many threads at the same time that it was impossible to follow them all. QGIS (new useful plugins to develop) and MapServer (now faster than ever) before lunch and the sensor and crowdsourcing session after lunch. There, between the ISA server for indoor spatial data, Apache Spark, Ontologies and CartoDB, we presented Cobweb. The day couldn’t finish without a very productive and promising BOFH session about crowdsourcing… and more!

Thursday was a hard day: jet lag started to go away, but taking #geobeers every night does not help much.

Alysa Wright started with a cool presentation about why and how did she start on the spatial world. We should never forget that, as a community, we come from very different places and perspectives. And still, we share a common view about freedom.

On the last session of the day, I learned about OpenDroneMap. I couldn’t stop asking myself why didn’t I knew about this incredible project before. Where drone software is even more awesome than the drone hardware.

Finally, Paul Ramsey talked about sustainability of free and open software and why we should care about who maintains it.

After the keynote, more social drinks with lots of fun:

Last day of conference is here. Friday tastes bitterweet. And not because of the food! María Brovelli starts with a keynote about crowdsourcing and VGI and how they can change our community. Because even if we don’t deal directly with the data, the software is also affected by all this.         The next keynote was from Marco Hugentobler, talking about the qGis community from a developer’s perspective. Because, yes, “developers are human and not everyone is aware of that”. What a surprise! In any case, never trust someone with the “expert” label, just trust the community.           After the coffee break, I attended a very instructive presentation from Henrik Lund Pedersen about caching fresh data. Is it possible to have a cached and updated server for GIS data?

During lunch we had some lightning random talks and I couldn’t stop myself. Please, use free and not open:

Jody Garnett talked about documentation and why we should be very careful when writing it: not only to help advanced developers but also to not scare newbies. Never use the word “easy”, for example. If you use it and the reader don’t find it easy, he will go away to look for something easier for him. And maybe the original documentation was easy, it’s just that it hasn’t been updated with new features.

Then we had another Cobweb session on the main grand ballroom, this time by Panos, talking about the mobile part of the framework.

And FOSS4G was finishing fast! But not before our president Jeff gave his vision about OsGeo and where we should focus on. Build community is the key. And shaking hands with people wearing suits.

On top of all the final party, GeoCat gave the first Most Innovative Developer Award. We hope to repeat it year after year encouraging developers in osgeo software.

Before finishing this post, I want to say a very big thank you to all the sponsors. Without them, this is not possible.

See you next year in Bonn!


Published first in GeoCat.