Is unstructured data worth your time? What do Qlik luminaries think?

Did you know it would take 1.2 million mosquitoes, each sucking once, to completely drain the blood in an average human? My latest post about the covid-19 publicly available app has 5 815 views up to now. The text analytics market value will be US$ 18.25 Bn by 2025. Interesting, isn’t it?

Do you think those numbers are correct? Is it even important? How many posts and articles do you read-only because they contain numbers in the headline or first one or two sentences?

People are fascinated by numbers. I believe there are many theories why it is so – about the evolution, genes, collective behavior… I like numbers, too. It’s comfortable to know exactly how many breads I should buy, to see numbers when comparing two possible scenarios before making the decision where to go for a vacation, to watch the reach of my posts in social media.

“If you can’t measure it, you can’t improve it.”

Data is something I can use to improve my business, my decisions, my lifestyle…

All in all, data is here to be measured.

Measures are numbers.

Data is numbers.

Wrong.

…but somehow many people think like this. Despite the fact that 80% of all data we generate is unstructured (not sure if the number is correct, but it looks better than “much”), we are still focused on analyzing and measuring numbers.

A few years ago, at Inphinity (even before it was named Inphinity and the team was a part of Emark) we developed the Mole unstructured data connector – the first Qlik custom connector for unstructured data. To be honest, it doesn’t have hundreds of users today as we expected.

What do Qlik Luminaries think?

Why are we so focused on numbers? According to Konrad Mattheis (Deloitte): “The first reason is that there is a lot of structured data. Secondly, most human beings need and want structure – unstructured is chaos, unpredictable, for them.”

However, many leaders with years of experience in the data analytics universe discuss the importance of unstructured data (see the article by Martin Kostic) or already deal with it (Visualizing HR 660 by Dalton Ruer). If you are interested in some creative analysis, check US Presidential Speeches Analysis by Terezia Blaskova (Emark).

I can nothing else than agree with Adrian Parker (Differentia Consulting): “For the full story, we need to show both structured and unstructured data in a single tool.”

The opinion of Radovan Oresky (Emark) is quite skeptical but also confirms the value in usage all data, not only structured: “Unstructured data will surely have its part in the future (not the only one). But I am doubtful, that it will be mainstream.“

Angelika Klidas (Data literacy evangelist) confirms it, too: “I think the power is within the combination of the two, so yes I do believe that unstructured data is in combination with structured the future.”

“For the full story, we need to show both structured and unstructured data in a single tool.” (Adrian Parker)

All mentioned leads me to the conclusion that unstructured data needs to be analyzed together with structured – it should be accessible and easy to understand for the users. We are on the way to “avoid the data racism” but it is a step by step process. Let’s start by combining the data together…

Combining data together

“Getting files with unstructured data into the application is at least half of the battle.” (Dalton Ruer)

Some inspirations

Oh, yes and some inspirations how something like this can be used right now 😉:

Adrian Parker: “This solution is great for empowering users to self-serve with sensitive data recorded on unstructured documents.  Delivery accurate and rapid results. Every department and business function will have this type of data. It can reduce exposure to compliance risk, e.g. the GDPR. If the data is stored only in one place it can be found.”

Angelika Klidas: “Uhm, this is more from an HR perspective, what is handy due to the fact that from the systems I know certificates are just copied, or scanned and added in the personals file. I love it that you can add it to a list and that the PDF is saved as well. I like what I see, but I have to think a bit further… it could also work for adding budget information, or expected deals to close, etc…”

Radovan Oresky: “Review, search and semantic analysis in Construction (engineering documentation), Legal (contracts, laws), Manufacturing / High-tech (product documentation, supplier contracts), Consulting (deliverables, knowledge base).”

Me: “I personally used it a year ago for managing all those presentations I downloaded from Qonnections 😁. You can start with something straightforward like CVs and certificates, legal documents, or technical documentation. Once you become familiar with unstructured data, I think many more ideas will come.”

Download Your Files Analyzer