Enabling access and coherent analysis of Big Data can be challenging. Hundreds of millions or billions of rows of data is not something trivial to model and access easily. It's always the question of how much data can one honestly present in an app that processes it quickly enough to be relevant for analysis.
Big Data is the reference term to massive amounts of information. In the last year, we have seen a sharp decline in the price of storage. Storage devices now can carry many terabytes in the same space footprint as a regular hard disk drive. This cost of ownership reduction caused a very sharp incline in the amounts of data that individuals store regarding to production, lifestyle, sales, sports and various other metrics. Such an increase in data storage is a crucial step in conducting more coherent analysis. The downfall is that the data discovery of such a massive haystack of information is only as valid as its correct and timely correlation.
Why is analyzing Big Data challenging? Enterprise databases are storing thousands of production, sales and other various metrics under lots of different tables or other structures. A reporting application would have to correctly correlate the data and aggregate it for the specific use case. Various ERP systems have implementations for a front-end visualization of their hosted data. The shortcomings with this approach is that aggregating data from other sources will be difficult, if possible.
This is where solutions like QlikView are coming in. This article will focus on how QlikView helps millions of users dissect Big Data in daily analysis processes.
QlikView is a tool for consolidating and visualizing data from various sources. It makes reporting data from different source system, possible, within a single application.
QlikView is a very mature software that allows the creation of reporting dashboards with data from multiple sources. Almost any source system that stores tabular data can be connected. QlikView will import and model the data via its script and link it to the UI elements of the app.
The possibilities of this approach are endless. Developers basically develop a process and a UI. Once the data load (can load from multiple sources inside the same application) is completed, the data must be modelled as a relational database structure so the data can communicate.
Data modelling is required to merge, concatenate, join or link disconnected tables that are imported via script. The QlikView script editor supports all the common scripting syntaxes (If, While, For, etc.), variables, functions and the scripting language is similar to SQL.
The tables within the data model will automatically link to each other when they detect a field (column) with an identical name. It's easier to visualize this by thinking of how Microsoft Access data model works (here it's easier, just one key) and how a Pivot Table can be interacted with.
Now let's imagine that a few tens of lines of code later and a few systems connected, we now have an app.
The power of QlikView is best shown when needing to create complex visualizations. Standard dashboarding tools revolve around a few, 2-3 charts on a page, while QlikView supports hundreds of objects to be deployed within a single page, giving the possibility of complex visualization structures, like the one you see on top. A page like that contains tens of objects, while able to support many more.
Now that we see what basically goes into developing an app, let's see how an enterprise environment would work.
QlikView is not a database system. It is merely a proxy for the data it temporarily holds within its apps. When a script reload is started, it will basically run and import and model the data. Then the data stays inside your QlikView file. The data inside your open app is stored in within your RAM. This makes calculations brazing fast.
A QlikView server deployment is mainly formed of 2 important components:
QlikView Publisher: it handles application tasks, like scheduled reloads and distributions to end users
It is important to highlight that QlikView doesn't have live connections to its sources. The data within the app is as-fresh-as the latest application reload.
This is where the Publisher comes into play. Once an application is developed, then you can create scheduled reload and distribution tasks on it. This will ensure that the data is coming in regularly and that the users will always have the latest version distributed to their respective portals.
Consuming a QlikView application is very straight-forward from a user's perspective. Each employee can access an internal URL that links to the QlikView AccessPoint. This is where each user will find all the applications that are distributed to him.
From here it's just about selecting your application and opening it. This will open the desired QlikView application within your browser using Ajax and the user will be able to interact with the data the exact same way as he would if he opened it locally.
QlikView enables users to consume Big Data easier, faster and, most importantly, without predefined paths allowing data patterns and discovery to be found naturally.
By removing predefined analysis paths and giving users full control of their data research, this tool allows for discovery of data patterns in a much more social way. The functionality of creating bookmarks and sharing them across the organization for a collaborative analysis process makes analytics much more fluid. Combining the flexibility of application consumption with in-memory engine for data storage and processing, results in analysts spending more time collaborating on their processes.
Because QlikView is a stand-alone system within an enterprise environment, this ensures a protection buffer from its data sources. Importing data from multiple and various source systems within a single application using QlikView, report analysis and data discovery is more coherent and faster for the end user.
Earlier we touched on the fact that QlikView stores the data within each application. This approach safeguards the end user in case of a source system failure. If a source system is down, their applications will still function perfectly because the data resides within the apps themselves. The user has the possibility of enhancing the power of the applications' visual analytics by creating their own objects and linking them to the existing data. This removes some burden from the developers, who can focus on creating the data import and modelling and puts the user in power of the type of visual analytics his analysis requires.
QlikView focuses on the concept of business discovery, which enables users to find their own analysis paths through predefined data sets. The data structure is defined by the developer, but its content is updated on a schedule basis or on-demand.
Users are free to use their applications over multiple devices. Whether one starts his analysis on his work computer, resumes it via his iPad to share results with colleagues or just check metrics on his smartphone, QlikView enables this by rendering the apps using Ajax and adapting to the end-user consumption environment. Developers no longer must develop a certain application for PC and one for mobile use; all app visualizations are scaled according to the platform that is opening them.
QlikView allows large scale enterprise systems to create a professional stand-alone reporting platform. Modelling the data sets from multiple sources and scheduling data imports and distribution is essential for coherent metric analysis and business discovery.
This way, Big Data becomes easily available to end users. They can easily interact with it, discover new data patterns within their organizations and simplify the workplace collaboration in the analysis processes.
As Big Data matures, large scale enterprises must find faster and more direct ways of allowing analysists to perform data evaluations. By taking app-making out of the equation and focusing on linking data from various sources, business will create processes and business discovery will flow more naturally.
by Ovidiu Mățan
by Lucian Torje
by George Bara
