10 trends in data intelligence for 2017

With the members of the strategic committee of the 2017 Data Intelligence Forum, we have identified 10 key trends in data intelligence. This field is so dynamic that we had to make some tough choices.

You can find more information about the 2017 Data Intelligence Forum to help you manage your data, including registration and application details, at https://www.documation.fr/en/data-forum/ .

Many thanks to Bruce Epstein for his collaboration on the English version of this document.

1/ System interoperability

We might assume that this problem is already solved, given that systems are already interoperable with respect to data exchange. Many protocols have been developed over the past few decades, APIs allow us to connect dataservices, ETLs provide connection between databases, semantic engines approach natural language. The “plumbing” exists and is regularly enhanced with new component, each more rapid and higher performance.

But data intelligence advancement has opened new territories that attract interest from research labs, software vendors and other companies with respect to data interoperability between human and intelligent machine or between two intelligent machines. Communicating knowledge and concepts is a very active research field, and will remain so for several years to come.

In this field, we also see progress that may be less ambitious but more pragmatic, to raise the abstraction level of exchanges between intelligent machines, and to provide a higher layer allowing us at first to operate independently from directly managing the various data exchange solutions.

2/ The rise of data storytelling

Data exchanges among experts and datascientists results in nontrivial amounts of time being devoted to analysis and understanding of the exchanged data, with a raft of interpretations, rediscoveries, and errors.

Data storytelling provides knowledge about this analysis and understanding process, easily transferable with the data. In a world where rapidity is often key to success, this gain of time and energy can be a decisive factor.

Data visualization platforms increasingly integrate data storytelling functions, reinforcing collaboration around data. Datascience education now insists on the importance of storytelling.

We knew that software developers were not particularly fond of narrative, finding it increasingly difficult to transmit their knowledge even as their languages have become more legible. Hopefully datascientists will be more prone to take advantage of storytelling.

3/ Liberating and protecting data

For many years two opposing schools of thought have been at odds: reinforcement of private data protection, versus growth of personalized services.

On the legislative front, France now has a more robust framework than its Anglo-Saxon partners. However the major players have become masters of the art of obtaining individual consent by providing them with practically illegible legalese-based text, presented at a moment that is not conducive to thoughtful, objective reading.

On the personalized services front, knowledge of each individual situation has become indispensable. But in this area, capturing private data is often a question of ease. Behavior modeling is a way to effectively handle anonymous users and thus limit the need to obtain personal data. However, use of such data remains essential in certain areas such as healthcare, security, education, etc.

The purpose of self-data, upon which FING (Fondation internet nouvelle génération, Next Generation Internet Foundation) and VRM (vendor relationship management) work, is to return control of one’s data to their owner, who can then control their usage in any circumstances.

Technologies are keenly awaited by the market to manage this delicate balance between freedom and protection of private data. Many initiatives are emerging, especially around the blockchain concept, which hope to provide a response.

4/ More “human-like” AI

The development of new AI techniques such as deep learning usher in the start of a new ear where AI can compete with humans on relatively complex intellectual tasks.

Automation of intellectual work might revolutionize the service world, very soon. Gartner says that one third of all the world’s jobs might be affected over the next 10 years. In a country like France, more than 80% of current activity is in the service sector and thus will be directly impacted by this phenomenon.

More “human-like” AI is AI capable of better serving humans, better advising them, better orienting them, making them more secure. Progress in intellectually-enhanced humans, in understanding natural language – even idiomatic, displaying emotion, are areas where startups emerge every day.

5/ Better use of user experience data

Capturing and using user experience data has always been a major goal of companies looking to feed fresh relevant data to their marketing, design, maintenance, purchasing departments (to name just a few).

The development of Internet, and mobile access, strongly contributed to the emergence of experience data capture techniques across all the steps experienced by the user. Newly connected objects will further reinforce this phenomenon.

Data storage and query mechanisms regularly offer new capabilities for prices that are continually more affordable. We see daily progress in data visualization: data visualization studios offer increasingly powerful platforms for datascientists to explore their data.

But it is on the holistic front that progress in using user experience data is most interesting. Most systems were previously limited to user categorization or alert generation and had to pass further control to an expert. Recent progress across available technologies offers the possibility of a richer, more relevant usage of user experience data. This is a burning issue for companies.

6/ Thirst for “collaborativity” in data

We have already seen “collaborativity” in the world of open source, bringing together thousands of contributors into common projects, we have also seen it in certain social networks, allowing everyone to express themselves openly on any topic of observation or thought, we have also seen it in the great encyclopedia projects bringing together knowledge from around the world. But in a world of open data, the movement has long remained reduced to a catalogue of highly independent data sources.

The most relevant data are always the result of a strong aggregation of numerous data sources. Progress in digitalization techniques along with the obvious will of companies and administrative agencies to automate their processing further strengthen the need for collaborativity in data.

Development of data-intelligence techniques allows us to process, understand, and generate data with unequalled power and is thus eager for new data sources and collaborativity in data.

7/ The quest for data truthfulness

Ensuring that the data we have at our disposal are as close as possible to reality is a highly natural reflex. Who could dispute the need to correct an error or an inaccuracy once we are aware of it, and who would wish to work with contaminated data?

Based on this principle mechanisms are being developed for data security, accuracy, traceability, and integrity.

But over the past few years we have also witnessed another phenomenon with noSQL technology, which has opened Pandora’s box by giving up data integrity in favor of speed. The world of Big Data is populated with data that are not guaranteed to be complete, accurate, or even up-to-date, all of which increases our need for alternative techniques allowing us to work with imperfect data.

Two completely complementary schools of thought evolve in parallel with opposing technologies: “clean before processing” and “process as well as we can with what we have”. But in both cases the need to manage uncertainty is exponential.

8/ The increasingly central role of Business Intelligence

Over time, we have all become sources of business data for companies. All of our browsing, movements, applications downloaded on our phones, exchanges on social media, and our mail, are sources of business information.

But we are also data suppliers for our own companies. Development of specialized applications offers new possibilities for continuous input to the enterprise information system. Employees, customers, suppliers with mobiles are connected to the information system. These collection and interaction channels are more structured, more inclusive, and faster. They are thus cheaper and higher performance than analysis of social media. They constitute a new step in the transformation of business intelligence that we have seen over the past several years, with the addition of datascience.

It is in the area of mobile business intelligence that progress is most striking, the smartphone having increasingly become a mobile office, in a way that PCs or even tablets have not been able to become as dominant.

This issue has entered the realm of behavior, as companies needing to collect data are no longer reluctant to pay their employees, customers or suppliers to compensate data collection or installation of proprietary applications.

9/ Geolocation everywhere

After low-power Bluetooth, Wi-Fi, ultrasound and Beacon, retailers have a choice of technology with which to geolocate their visitors inside their stores. Internet has the advantage of ease of capturing the customer’s browsing history, but its lack of ability to provide the customer the opportunity to actually touch the product is decreasing its superiority with respect to collecting customer knowledge compared to the bricks-and-mortar store.

The impact of geolocation greatly surpasses the scope of retail business. Every day, new applications arrive and interest physical site managers in culture, transportation, urban areas, sports, healthcare, etc.

Solutions based on geolocation are being developed to offer products and services, supply information and commentary, put people in touch with each other, optimize flows, manage security, etc. Knowing that a physician is located just a few meters away from an accident might save a life.

Geolocation is required for the new revolution enabling artificial intelligence in the concept of “enhanced society”.

10/ Data discovery: the hidden treasure in your data

It is no longer necessary to convince companies that their data constitute an inadequately exploited treasure.

“Data discovery” systems combining search engines, indexation and semantic analysis are constantly being developed and enhanced. Data lakes are being formed in each company to accept increasing quantities of data, thus offering concrete experiences for these new technologies.

The data economy is becoming increasingly developed, as observed by the report issued last summer by the Franco-British task force on this topic.

In this area, we are hearing the desire expressed by companies to extract value from their data. They want not only to know what data they own, detect relevant data to make their projections, improve their processes and develop new business sources, but also to be able to assign a financial value to these data.

Data-driven governance solutions and MDM (master data management) are available and increasingly enhanced with new possibilities. Data valuation solutions and data production and exploitation ERPs are eagerly awaited across the sector.

Jean Pierre MALLE