In his 2008 research paper (“The Rise of Data Discovery Tools”), Kurt Schlegel of Gartner identified a new and growing category of business intelligence (BI) tools, and coined the term “Data Discovery.” Since then, tools like QlikView, Tableau, and Spotfire (and to some extent, knock-offs from the big box vendors), have completely changed how we go about visualizing data, and have become a multi-billion dollar niche within the larger field of BI.
I remember when, back in 2003, Business Objects (now part of SAP) introduced their first dashboard tool called Dashboard Manager. We worked on a number of Dashboard Manager engagements, despite the level of effort (and costs) required to develop even the simplest of dashboards in the tools of the day. Usually, a large portion of the projects involved simply deciding what was to be shown on the dashboard screen. We had detailed requirements gathering sessions to determine every data element that was to be displayed, followed by detailed storyboards or “wireframes” like commonly used in website development projects. And when the data and visualization requirements were complete, they were typically etched in concrete since it was so difficult to change anything mid-stream. Ah, the good old days….
And that was just the front end. Most of our early dashboard projects assumed that the required data was in a format that was easily consumed by the dashboard tools, but it was very seldom the case. Most of these projects required extensive data management design and implementation, and many dashboard projects were shelved mid-stream as these costs were not usually factored in until after the fact.
So we had this hard-to-use dashboard tool, sitting on top of an even more change-resistant data environment. Well, as we all know, you only get the real user feedback after the user has seen the dashboard in production. And with that feedback, weeks or months passed while changes were made to each layer in the dashboard architecture. Good work if you can get it, but the cycle time required to keep up with end user changes usually kept real dashboard development on the fringe of IT.
Fast forward to the present day, with Data Discovery tools, this is no longer the case. BI developers can get together with business users, and in an hour or two, come up with a dashboard prototype that is closer to being production-ready than virtually any dashboard projects in the past. This is the essence of agility, and is why our friends at Gartner say that Data Discovery has become a multi-billion dollar subset of the overall business intelligence market.
But all that agility has a price, and is the subject of a future blog….
About WIT
WIT is an analytics and automation consulting firm with over 25 years of experience. We help our clients build solutions to maximize the value of their data and analytics assets, and automate key business processes. In partnership with elite software companies, WIT’s capabilities span a broad spectrum of business functions and industry verticals. Learn more at www.witinc.com