Self-Service BI Central

An excerpt from my research study on business intelligence systems

BUSINESS DATA AND BUSINESS INTELLIGENCE (BI)

According to the economist, “the world’s most valuable resource is no longer oil, but data” (Parkins 2017). This is because business generate data from different operational and transactional systems including customer relations, sales, marketing, finance, production, warehousing and supplier systems. Successful businesses see this an asset and therefore direct efforts at developing information systems on this wealth of data with the help of BI.

Business Intelligence utilises processes and technologies in developing information systems that supports strategic decision making for businesses including extraction, transformation and loading into data marts and data warehouses to harness value for management to make informed decisions. By preparing and presenting data, BI provides intelligence for which organisations stand to gain competitive advantage by analysing business data for trends and patterns. By extension, Vedder et. al (1999) reports that, organisations, in analysing external data, are able to predict the behaviour of “competitors, suppliers, customers, technologies, acquisitions, markets, products and services, and the general business environment” (Jourdan et al. 2008). This is achieved by getting BI experts who may be part of an organisation’s IT department or external to the organisation to provide such service to business users.

THE NEED FOR SELF-SERVICE BUSINESS INTELLIGENCE (SSBI)

Traditional business intelligence tools and technology require skilled personnel to provide this important function. However, self-service business intelligence (SSBI) tools is gaining popularity for its simplicity and affordability (Iyengar 2016) in achieving same objectives as traditional business tools. SSBI, an evolution of the traditional BI, involves the provision of information systems using new technologies that focuses on providing business users a platform to create drag and drop easy-to-use tools that requires less intervention of BI experts or IT support in creating business information for decision making. The business user, with SSBI, is empowered to ‘design and deploy their own reports and analysis’ in a supported environment (Gartner 2017d)

Apart from the simplicity and affordability tags on SSBI, changing data formats and the attendant need for revising the design and implementation of business intelligence systems is another reason for SSBI gaining popularity. Until recently, enterprise databases serving structured data has been used for designing business intelligence systems. However, McAfee and Brynjolfsson (2012) observes that current trends in the use of mobile devices and social media systems which businesses rely on presents unstructured data in a variety of formats, volumes and velocity (Alpar & Schulz 2016) requiring a new approach in the design of business intelligence systems; and this SSBI embraces. In addition, Böhringer et al. (2010) reports an extension of scope of business intelligence and analysis from strategic queries to include operational queries, a fundamental change that requires the development of more reports and analysis. In this regard, SSBI provides improved user experience with intuitive and easy to use interfaces that does not only empower business users who better understand the nuances of line-of business data and the problems at hand (Harvard Business Review 2016) to create their reports and analysis but also makes the work of BI specialist fast and easy.

BUSINESS INTELLIGENCE (BI) MARKET TRENDS

The BI landscape has changed since the concept of self-service solutions were introduced in 2004 (Gartner 2017a). Until then vendors like SAP, IBM, Oracle, SAS dominated the market with enterprise BI platforms which are scalable and required skilled personnel to develop BI systems. However, new vendors like Tableau, Microsoft, QlikView concentrated on providing drag and drop, easy to use interfaces for business users and this has gained in popularity. As a result, the traditional vendors have started offering self-service drag and drop functionalities with new vendors also expanding into enterprise offerings. This is pushing the frontiers of BI to go beyond data visualization, enterprise BI and reporting on structured historical data to include streaming data. With businesses relying on streaming operational data, a new offering: operational or real-time (Vo et al. 2017) or cloud BI- beckons with exciting prospects as cleansed streaming data holds the potential of being utilised for predictive analysis leading to actionable and suggestive BI (Bennett et al. 2017). In this regard, Artificial Intelligence, Machine Learning and Natural Language Processing is seen as technologies that would be utilised

VENDOR’S SHARE OF THE MARKET

As at 2012, traditional vendors such as SAP, Oracle, IBM and SAS had 61% of the market share between them; the following year the share of the market increased to 69% leaving the competition behind. By 2015, the gap has been closed to 34% with exciting new vendors like Microsoft, Qlik and Tableau with a host of exciting new vendors promoting the self-service concept.

Traditional and new vendors have aimed efforts at each other for some time now with new product offerings; whiles the enterprise BI market had been dominated by IBM Cognos and SAP Business Objects, self-service business intelligence market has seen new vendors Qlik and Tableau dominate.

However, the tide is gradually turning towards enterprise-cloud-self-service BI where both traditional and new vendors previously had as niche markets, but would now have to share the spoils (Woodie 2017) per Forrester’s 2017 wave for both enterprise and cloud BI products.

VENDOR PROFILES

With a competitive vendor market, various research analysis of the industry provide a good basis to streamline them from over 70 vendors (Bennett et al. 2017) to 10. To streamline them, two respected and notable research houses – Gartner and Forrester- were relied on in this study. Gartner’s annual report is presented with the ‘magic quadrant’, an illustration thatplaces SSBI technologies or platforms in challengers, leaders, niche players and visionaries quadrants, measured on completeness of vision and ability to execute. Even though previous Gartner’s magic quadrants were considered, the 2017 Magic quadrant formed a basis for the streamlining; reporting that currently Tableau and Microsoft are market leaders as illustrated below.

Forrester’s wave on the other hand uses challengers, contenders, strong performers and leaders as waves and the technologies or platforms positioned against current offering and strategy. It reports for 2017 Q3 MicroStrategy, TIBCO, IBM, Qlik, Oracle as leaders with on-premises and cloud BI; at the same time, Microsoft, SAS, Tableau and SAP recognised strong performers.

Based on Gartner and Forrester research analysis of the BI market, the following 10 vendors are profiled alphabetically

Vendor	Headline Vendor Claims	Defining Features	Research Endorsement
Alteryx	1. “Leader in Self-Service Data  Analytics 2. Deliver deeper insight in hours, not weeks, with a repeatable workflow for analytics” (Alteryx 2017b)	A drag and drop visual workflow that allows ETL and Predictive Analytics with no coding needed, thereby leading self-service data analytics for line-of -business analysts	Challenger in 2017 Magic Quadrant for Data Science Platform
IBM	1. “Modern. Able to span. Indispensable. … we offer an analytics vision, unmatched analytics services, data expertise and  global reach to help with virtually any implementation or use case…. offer both great software products and decades of experience…having one of the most functionally rich and capable BI portfolios in the industry” (Wakerell 2017). 2. “Smarter self-service capabilities, integrated solution, guided experience, consistent web-based experience and proven governed platform…Analytics you can trust”(IBM 2017a)	A well-rounded scalable and secure enterprise offering analytics and planning with automated alerts and contextualized smart search to key findings, proven governance and  integrated data modelling  for information consumers, data explorers and power users.	Named a market leader in the BARC Score Business Intelligence (Seidler et al. 2017)
Microsoft	“Business intelligence like never before: go from data to insights in minutes, any data, any way , anywhere. And all in one view” (Microsoft 2017c)	A dynamic, easy-to-use interactive data visualisation  BI platform built on the proven Microsoft cloud and enterprise framework with a wide range of data connectors and visuals that can be consumed across mobile devices with the flexibility of customisation  for developers, IT, business users and analysts.	Microsoft is recognized as a leader in the Gartner Magic Quadrant for Business Intelligence and Analytics for 2017 (Gartner 2017a) and Strong performer for the 2017 Forrester wave
Oracle	1. “There is a business analytics revolution happening…Oracle business analytics are changing the world 2. “from the agility of visual analytics and self -service data discovery, to the power of an enterprise platform, including operational analysis at scale, security, reliability, extreme performance, and centralized management. Only Oracle combines this agility and power in a single platform”	A modern analytics strategy with voice and touch enabled  data querying integrated into a single platform that powers advanced analytics, in-memory enhancements and a self-service capability that requires no modelling  for faster discovery of insights	Oracle is recognised  a leader in Forrester 2017 Q3 report(Oracle 2015)
Pentaho	1. “A unified data integration and analytics platform with real time  data processing to fast track  digital insights” 2. “Pentaho is the only vendor to allow users to visually explore data in-line at every step of the data pipeline, with a single platform” (Pentaho 2017c)	With visual ETL designer that eliminates coding, data from any source is prepared and integrated, allowing agile view of data in the preparation pipeline and presenting data for analysis without the need for staging	Top rated big data vendor (Dresner 2016) Leader for Data Integration By Gartner(Gartner 2017b)
Qlik	1. “Putting an end to analytics blind spots. That’s the associative difference…Powerful insights you miss with other tools” 2. “Get total flexibility with a cloud-ready data analytics platform that supports the full spectrum of BI use cases-ideal for analysts, team or global enterprise”	With associative  engine, any number of data sources is combined and explored, going beyond the limits of SQL-based queries in a secured and governed framework enterprise-class analytics solution that is cloud-ready	Qlik achieves top rankings in BARC’s BI Survey (BARC 2017)
SAP	1. “SAP BusinessObjects is the de facto standard for Big Data analytics in organisations  around the world (SAP 2017b)” 2. “One strategy for enterprise BI…one suite for all insight… one place for all information…one standard for enterprise BI (Rose & Kuruvilla)”	Consistent and Ubiquitous BI experience offering scalable , secure and integrated environment with an online and offline capabilities  that provides  dashboards in shock wave  files	SAP recognised as a strong performer in the 2017 Forrester Wave and a visionary in the Gartner 2017 magic quadrant
SAS	1. “Put the world’s most powerful analytics in everyone’s hands…share insights and performance metrics based on foresight not hindsight…backed by more than 40 years of expertise… to give you the power to know”  2. “Interactive reporting. Visual data discovery. Self-service analytics. Scalability and governance. All from a single, power in-memory environment  (SAS 2017c)	Translates its rich experience in analytics to the self-service market with strong algorithms in an automated and powerful analysis including forecasting goal seeking, scenario analysis, decision trees and extends this to text analytics	For the 12th year, SAS named a leader in Gartner’s October 2017 Magic Quadrant for Data Quality Tools SAS named a leader in the Forrester Wave for Predictive Analytics and Machine Learning Solutions, Q1 2017 (SAS 2017b)
Tableau	1. “Answer questions at the speed of thought…analytics that works the way you think” 2. “Harnesses people’s natural ability to spot visual patterns quickly, revealing everyday opportunities and eureka moments” 3. “We invest more in R&D than anyone else in the industry” (Tableau 2017d)	Benefit from R&D in computer graphics, analysis such as trend analysis, regression, correlation as well as databases either as big data, live or in-memory to provide powerful and fast analytics	Tableau recognised as a leader in 2017 Gartner Magic Quadrant for business  Intelligence and Analytics platforms for a fifth consecutive year (Tableau 2017e)
Tibco	1. “Global leader in integration and analytics 2. “TIBCO Spotfire is a smart, secure, governed, enterprise-class analytics platform with built-in data wrangling that delivers AI-driven, visual, geo, and streaming analytics” 3. “Spotfire is the only platform that empowers business users with an intuitive, easy-to-use interface to leverage the full spectrum of big data analytics technology, without requiring any data science or IT expertise.” (TIBCO Spotfire 2017)	Built on streaming capacity, augmented intelligence is gained through smart visual analytics, immersive data wrangling and predictive analytics on top of proven enterprise scalability enterprise	TIBCO Spotfire recognised  a leader in the Forrester Wave for Enterprise BI Platforms With majority on-premises deployment Q3 2017, a leader in the Forrester wave for streaming analytics and a market leader for predictive and advanced analytics

---

REFERENCE

Alpar, P. & Schulz, M., 2016. Self-Service Business Intelligence. Business and Information Systems Engineering, 58(2), pp.151–155.

Gartner, 2017a. Magic quadrant for business intelligence and analytics platforms. Gartner, (February 2014), pp.1–126.

Harvard Business Review, 2016. THE UNTAPPED POWER OF SELF – SERVICE DATA. A HARVARD BU S I N E S S R E V I E W A N A LY T I C SERVICES REPORT, pp.1–12.

Jourdan, Z., Rainer, R.K. & Marshall, T.E., 2008. Business intelligence: An analysis of the literature. Information Systems Management, 25(2), pp.121–131.

DEFINITION

Visualization involves “computer-based visualization systems [that] provide visual representations of datasets designed to help people carry out tasks more effectively” (Munzner 2014). Visuals or graphics such as chart, figures and tables are thought to be the simplest and yet powerful (Tufte 2013) as they use human perception to aid understanding of the underlying dataset. Therefore, to aid understanding and identify patterns, trends and correlation, data is represented with visuals in a process known as data or information visualization. Information Visualization is “the use of computer-supported, interactive, visual representations of abstract data to amplify cognition” Card, Mackinlay and Shneiderman (1999) cited in (Dykes, Jason; Slingsby 2017b).

EVOLUTION OF DATA VISUALIZATION

INTRODUCTION

Data visualization predates the 19^thcentury when statistics was increasingly being used for planning in trade and commerce taking different forms as it developed; including map, graphs and later extending into graphics (Friendly 2006). 

DEVELOPMENT OF VISUALIZATION ACROSS TIME

As illustrated in Figure 5 above, the earliest development started as ‘graph-like’ tables with point placements representing positions of celestial bodies in an effort at developing maps. These efforts were further developed into measuring physical quantities such as time, distance and space resulting in a table showing distance between cities which was extended into maps and chart. By this time, geometric shapes have been developed and included on the maps (graphs) – marking the beginning of modern graphics which is attributed to William Playfair (Tufte 2013) and coincided with statistical efforts to collect social data such as “wealth, population, agricultural land taxes…as well as insurance and annuities based on life tables”. Around this time, Florence Nightingale launched a bid for improved conditions as she puts it “more deaths occurred from disease and consequence of wounds than from the hands of the enemy”. Her efforts were later developed by Dr. John Snow in producing his famous dot map to highlight the correlation of cholera deaths to the Broad Street pump. There was a dormancy in developing data visualisations, but this dormancy was broken with John W. Turkey’s “Exploratory Data Analysis (EDA)” when he launched a campaign for data analysis to be recognised as a legitimate branch of statistics with effective graphic displays which included stem-leaf plots, boxplots, hanging rootogram, two-way table displays. His efforts saw computer processing of statistical data- shift from its hand-drawn counterparts in early data graphics developments. Close collaborative research work in computer science and statistics started in earnest and would soon see 2D and 3D interactive evolving and blossoming into a wide range of dynamic visualizations for desktop computers (Friendly 2006). 

VISUALIZATION STAGES

As visualization aims to aid cognition, it is vital to note the role vision and interaction play. Vision is important because “we acquire more information through vision than through all of the other senses combined” and the acquired information can be translated into cognition as “most cognition is done as a kind of interaction with cognitive tools… and increasingly computer-based intellectual supports and information systems” (Ware 2004). As a result, four basic stages of visualization are identified with Figure 6 illustrating their interrelation:

In the above, data collection (first stage of visualization) from the physical environment with social environment providing context, is the core to which transformations and pre-processing (second stage) are applied. The resultant transformation and pre-processing use graphics engines in rendering or displaying (third stage) visualization for interaction (fourth stage) which can take the form of selecting parameters or changing the perspective or position of 3D visualization all aimed at communicating insights.

VISUAL ENCODING

INTRODUCTION

Cognition, as already established, would only be deemed successful if the intended recipient of a visualization can understand the ‘message’ conveyed by a visualization. For this to occur, the design of the visualization must be considered to convey information from given datasets. One way of achieving this, is to present the information in a manner that can be understood by the recipient and as a result, various design principles and perceptions are proposed. A linear process of encoding and decoding based on human visual perception is therefore pursued going forward.

VISUAL COGNITION

It is reported, after a number of studies have been conducted that humans process and perceive visualization in one of two ways: pre-attentive and attentive processing (Nazemi 2016)

PRE-ATTENTIVE PROCESSING

Pre-attentive processing is the type of cognition that allows understanding visuals ‘at a glance’ without a conscious effort (Dykes, Jason; Slingsby 2017b). Visualizations that use visual properties: “length, width, size, curvature, number, terminators, intersection, closure, hue, intensity, flicker, direction of motion, binocular luster, stereoscopic depth, 3D depth cues and lighting direction” as well as visual variables: “luminance and brightness, colour, shape and texture” serve as visual stimuli that tend to gain human attention in less than 250 milliseconds (Ward, Matthew; Grinstein, Georges; Keim 2015). This, Ward et al. regards as pre-attentive processing, also known as ‘pop-out effect’ is associated with performing tasks such as target detection, boundary detection, region tracking as well as counting and estimation (Nazemi 2016). Treisman’s Feature Integration Theory (Treisman, Anne; Gelade 1980)is widely used to test detection of target objects from a sea of detractors. It is evidenced, that “the more an object differs from the detractors, the better it can be processed” and that “a green square, for example, in a sea of red circles can be better recognised pre-attentively than a red square” which was further expanded to include more than one unique visual feature (Nazemi 2016).

ATTENTIVE PROCESSING

Attentive processing which on the other hand requires conscious effort to understand visuals (Dykes, Jason; Slingsby 2017b), is based on Wolfe’s Guided Search model in which he proposes that information from the pre-attentive stage guides the attentive stage (Wolfe 2007). This theory is further developed by Ware who proposes a three-stage model of “pattern recognition, sequential goal-directed processing and pre-attentive processing” (Ware 2004). Visual features “orientation, colour, texture and movement patterns” are associated, with the visual stimuli passed from the pre-attentive stage to pattern recognition where regions and localization are identified leading into sequential goal-directed process which is active attention in nature (Nazemi 2016). 

ENCODING

For data visualization to successfully communicate insight, data elements and visual variables need to be carefully chosen and combined in a manner that would take advantage of human processing and perception of visualization. For this reason, a linear process of encoding information for visual transmission and subsequent decoding by the reader is advocated (Iliinsky, Noah; Steele 2011). 

VISUAL CHANNELS

MacEachren (1995) provides a number of visual elements or channels as choice for encoding with suggested usage regards characteristics of variables such as selective, associative, order, quantitative and length in Figure 7.

DATA ELEMENTS

To start, the designer needs to be accustomed with the data, knowing the data elements to be visualized in order to decide which visual elements to use. Data lends itself to being classified as categories or measures with each class further grouped into nominal, ordinal, interval and ratio illustrated in Figure 8 with suggested colour encodings (Dykes, Jason; Slingsby 2017b)

In much the same way, data attributes come grouped as categorical and ordered with ordering direction illustrated in Figure 9 below (Munzner 2014):

MAPPING VISUAL TO DATA ELEMENTS

It is important to consider, of the visual element, the natural ordering and the number of distinct values readers can identify, in choosing visual elements to represent data values. In respect to human perception and processing, natural ordering refers how the human brain without any prejudice considers visual elements like positions, length, line thickness or weight to be naturally ordered and that given a lot of colours (distinct values), it would be difficult to tell them apart. Figure 10 provides properties of common visuals to consider as a guide when deciding encoding visual properties to data elements (Iliinsky, Noah; Steele 2011)

As visual properties that can be encoded overlap in what data type it can represent, Figure 11 provides a grouping that can also serve as a guide (Iliinsky, Noah; Steele 2011)

The designer is encouraged to use the suggested encoding of ordered and categorical channels and their attributes by (Munzner 2014) as illustrated in Figure 12 below for effectiveness

The colour brewer (Brewer 2017), provides a guide for encoding sequential, diverging and categorical visual elements

VISUALIZATION PITFALLS TO AVOID

In the design of visualization, the focus is to have readers understand pre-attentively, therefore it behoves on designers to avoid common errors that often lead to attentive processing of visuals. These include:

• Design for recognition instead of recall (Ware 2004). Any design that requires the reader to remember something for understanding falls under this category. So is, design that causes mental calculations to understand 
• Colour is not naturally ordered, when had pressed to use colour in elevation and heat map, etc. varying luminance (brightness) along one or two axes is advised (Iliinsky, Noah; Steele 2011)
• Arranging the data poorly and adding unnecessary design or decorative features also referred to as chart junk or graphical paraphernalia (Few 2006)
• Maximise the core of the graphic supported by data (data ink) and remove non-data or redundant data
• Maximise data density which is the number of data entries per area of graphics (Dykes, Jason; Slingsby 2017a)
• Increase graphical integrity by reducing the lie factor – ratio of the size of effect shown in graphic to the size of the effect in data (Tufte 2013)

“Thirteen common mistakes in dashboard design” has been identified (Few 2006) and presented below:

• Exceeding the boundaries of a single screen
• Supplying inadequate context for the data
• Displaying excessive detail or precision
• Choosing a deficient measure
• Choosing inappropriate display media Introducing meaningless variety
• Using poorly designed display media Encoding quantitative data inaccurately
• Arranging the data poorly
• Highlighting important data ineffectively or not at all
• Cluttering the display with useless decoration
• Misusing or overusing colour
• Designing an unattractive visual display 

To efficiently utilize visualization to communicate insight clearly and precisely, “graphical displays should: 

• show the data
• induce the viewer to think about the substance rather than about methodology, graphic design, the technology of graphic production, or something else
• avoid distorting what the data have to say
• present many numbers in a small space
• make large data sets coherent
• encourage the eye to compare different pieces of data
• reveal the data at several levels of detail, from a broad overview to the fine structure
• serve a reasonably clear purpose: description, exploration, tabulation, or decoration
• be closely integrated with the statistical and verbal descriptions of a data set” (Tufte 2013).

BUSINESS DASHBOARD

“A dashboard is a visual display of the most important information needed to achieve one or more objectives; consolidated and arranged on a single screen so the information can be monitored at a glance” (Few 2006). It is likened to the dashboards on cars showing indicators for important processes taking place at various components of the car; such as brake fluid, speed, seat belt, distance covered and whole lot more. 

In business, dashboards serve many a function including strategic, analytical or operational purposes (Few 2006). Figure 14indicates the types of business dashboards, type of control and decision as well as the structure it takes.

KEY PERFORMANCE INDICATORS (KPI)

For business dashboards to play strategic, operational or analytical roles business data is used from which performance is monitored using key performance indicators (KPI) metrics. “KPIs are quantifiable metrics which reflect the performance of an organization in achieving its goals and objectives. KPIs reflect strategic value drivers rather than just measuring non-critical business activities and processes. KPIs align all levels of an organization (business units, departments and individuals) with clearly defined and cascaded targets and benchmarks to accountability and track progress” (Bauer, 2004, p63). When properly designed, KPIs provide indication of deviations or adherence to targets.

However, a number of factors account for not meeting the lofty ideals of KPI including creation of KPIs in isolation without their inter-relationships, gap in tacit knowledge of business analyst against KPIs as well as inconsistencies between business strategy and the KPIs (Maté et al. 2017). To avert this, the strategic alignment pyramid (Bauer 2004a) is used to translate business vision into KPIs and key action initiative as illustrated in Figure 15 below:

SCORECARDS

Scorecard integrated dashboards presents another way of meeting the ideals of KPIs, of monitoring the health of business in alignment with its strategic mission. This is because the use of scorecard combine KPIs with contextual information that leads analyst to the root cause of deviations in meeting set targets (Maté et al. 2017). A scorecard provides “fast but comprehensive view of the business” and a “starting point for improved managerial performance” (Kaplan and Norton, 1992, p. 71) cited in (Perkins et al. 2014). Often referred to, as Enterprise Dashboards, Balanced Scorecards, Performance Dashboard or KPI Summary, different businesses approach its design as varied as different technology platforms and strategy permit; a typical scorecard shapes up as below:

---

REFERENCES

Dykes, Jason; Slingsby, A., 2017a. A Summary of Tufte’s Theory of Data Graphics.

Dykes, Jason; Slingsby, A., 2017b. Information Visualization.

Few, S., 2006. Information Dashboard Design: The Effective Visual Communication of DataC. Wheeler, ed., Sebastopol: O’Reilly.

Iliinsky, Noah; Steele, J., 2011. Designing Data Visualizations, Sebastopol: O’Reilly.

Munzner, T., 2014. Visualization Analysis and DesignTamara Munzner, ed., Boca Raton: CRC Press (Taylor and Francis Group).

Nazemi, K., 2016. Information Visualization, Switzerland: Springer International Publishing Switzerland.

Tufte, E.R., 2013. The Visual Display.

Ward, Matthew; Grinstein, Georges; Keim, D., 2015. Human Perception and Information Processing. In Interactive Data Visualization : Foundations, Techniques, and Applications. pp. 81–138.