The Directorate for Maritime Affairs improves data quality and business intelligence with Talend

Quality controls directly integrated into data flows, data consolidated and distributed to different departments, and streamlined deployments help the Directorate for Maritime Affairs and its partners to optimize business intelligence
The Talend solution meets all our needs and, above all else, makes us independent. We are often surprised by the wealth of the new versions, allowing us to plan future extensions.
Michel Lollichon, Project Director at the Directorate for Maritime Affairs

The Directorate-General for Infrastructure, Transportation and the Sea (DGITM) of the French Ministry of Ecology, Sustainable Development and Energy is responsible for all matters relating to land and sea transportation (except for road safety) and airport planning. It is composed of three departments, including the Directorate for Maritime Affairs, which is responsible for maritime safety, maritime training and education, monitoring of seafarers, development of the national flag, recreational policies and nautical activities.

A constant source of important information and applications ​​since its inception, the Sub-Directorate for Maritime Information Systems (SDSI) is a service of the Directorate for Maritime Affairs, responsible for creating and maintaining IT applications and providing guidance for other services. It includes about 50 individuals whose duties range from project management assistance to IT project management and providing user support.

A significant pool of data to enhance

The Directorate for Maritime Affairs manages and produces a lot of data via its business intelligence information system: statistics on commercial vessels (e.g., number of merchant ships or controlled fishing vessels), recreational permits or the activities of seafarers.

During the development of the BI system in 2007, it sought a data integration solution capable of being populated by heterogeneous data. This solution had to replace the former COBOL data extraction system, the maintenance of which became increasingly cumbersome as analysis needs grew. In compliance with the global directives of the Ministry, the Directorate began testing open source solutions.

At first, it downloaded Talend Open Studio for Data Integration, which shared the same technological foundations (Java) as the Directorate’s new information system. After a series of tests, the Directorate decided to deploy Talend Data Integration to enjoy collaborative work features and version management.

Our goal was to create a data warehouse (based on Sybase IQ) from heterogeneous sources – 14 in total, each of which are based on different data base technology, particularly Oracle – in order to apply various consistency and quality controls,” explains Michel Lollichon, Project Director at the Directorate for Maritime Affairs. “This data warehouse was constructed based on several shared repositories – the vessel, the parties involved (sailor, owner, shellfish industry, etc.), the profession and activity of the sailor. While the data volumes to process are not necessarily significant, the data – very dispersed – must be consolidated to obtain a cross-disciplinary vision and improve business intelligence.

Talend Data Integration to guide decision-making

Quality controls were directly integrated into data flows. When the Talend solution detects an anomaly or inconsistency, it triggers an alert in order to correct it. Generally speaking, all data is logged, which enables the fast processing of large volumes.

The consolidation of this data enables us to complete dashboards that support the decision-making of the Directorate for Maritime Affairs (e.g. statistics on maritime employment or monitoring of vessel activity). We also distribute them to the services of the Interregional Sea Departments, the Sea and Coast Commissions, and other administrations according to the specific or specialized needs,” adds Michel Lollichon.

At the same time, the Directorate transferred numerous batch processing series for the consolidation, update, export and distribution of data to Talend, which were previously based on COBOL. The objective was to streamline this processing on which data exchanges with external partners are based (such as the National Establishment for Disabled Seamen, which is the social security regime for seamen). In all, six data exchange interfaces were developed to facilitate exchanges with partners who have different information systems.  Talend enables the use of Web Services to send or receive data in automatic exchanges.

User-friendliness, portability and robustness of the components

Today, most of this processing is in production, the first of which were implemented in 2009. The statistics are returned via an information portal based on SpagoBI, which should eventually grant access to approximately 1,000 users of the Ministry of Ecology.

The Directorate for Maritime Affairs stresses how easy the Talend solution is to use, in addition to the wealth of components inherently offered. “The user interface is easy to learn and does not require extensive Java knowledge or experience. The range of components is very rich, but it is also easy to create new ones, which was required in our case to meet specific needs,” comments Alexandre Olliet, Head of Technical Projects appointed by Capgemini. “In addition, the solution is regularly improved and enriched via the availability of new versions and Talend takes the feedback from its users into consideration in these developments. This increases the sustainability of its solutions, which is essential for the Directorate since Talend Enterprise Data Integration has become a strategic component of its information system.

Meanwhile, the Directorate for Maritime Affairs appreciates the portability of the Talend solution. “The Talend deployment process is based on well-structured and documented servers, which enable us to streamline our deployments. The administration console helps us manage users to whom access rights are granted.  After some testing operations, in-production deployment is a simple export operation from Talend servers. We perfectly control our deliveries and deployments and we know how to quickly correct any deviations or anomalies,” continues Alexandre Olliet.

Lastly, the Directorate for Maritime Affairs appreciates the clear and easy-to-create documentation which enabled the deployment of good practices. This contributed to making activities sustainable by facilitating knowledge and skill transfers.

The Talend solution meets all our needs and, above all else, makes us independent. We are often surprised by the wealth of the new versions, allowing us to plan future extensions, such as interapplication business data exchanges. With Talend, we can broaden the spectrum of analysis of our department and our partners,” concludes Michel Lollichon.