Last Mile of Dashboard Development
In a world where “Big data is the new natural resource”, it is a must to create applications taking advantage of this new knowledge-full resource. Brave people started venturing into this unknown world, They are using new and exciting tools like Notebooks or Spark engines. However, developers are still banging their heads against a wall when time comes to build dashboards. It can be a hard and very tedious task. Depending on your target platform, you will probably end up developing a web application that happens to generate dashboards. While there is nothing wrong with this approach, we believe there must be a better way!
The last analytic mile
We have identified two distinct activities that play a very important role in the analytic process: “Data processing and transformation” and “Insight exposure”. These two jobs take place in different places and at different times within the pipeline. We even go as far as to say that the set of skills required for them are completely different. We believe that they should be separated and tackled in different ways. A different set of skills implies a different set of tools.
We see lots of new thriving technologies in the realm of data processing: from full analytic engines to frameworks that allow to build analytic engines from scratch. The rise to power of the Jupyter notebook is a perfect example of the necessity to develop transformation processes to better navigate today’s Big Data. Notebooks make it possible to instantly see the results of any transformation process. Data scientists can fetch, transform and visualize data in a simple notebook interface. They can later convert this process into repeatable units using more powerful analytic engines like Apache Spark. It now only takes them hours to expose the information to the world. While these processes are second nature to data scientists, line of business users will find them daunting at the very least.
Once they have completed the data transformation and extracted proper insight, they need to get the findings into the hands of decision makers in a highly consumable way. This is what we are calling “The last analytic mile”. This mile is currently difficult and slow to walk. The process to create dashboards that take those findings and convert them into enticing visualizations is still quite complicated and lengthy.
A new discipline
Developing dashboards from the ground up is a discipline on its own. Within IBM Emerging Technologies, we created a development framework that enables developers to easily build dashboards, and focus on the dashboard itself, not the application. It changes the way we look at dashboard applications and the way we go about building them. This is an evolution of the paradigm usually followed when developers create web applications: from an imperative to a declarative view. They can better focus on what the dashboard should do and look like, rather than how it should be built.
Dashboard as data
We are changing the way we see dashboards, from executable to a process-able units, by converting a dashboard into data. Dashboards now inherit all the benefits that data possesses. They are:
- Easy to transport
- Easy to store
- Easy to generate
- Independent from a processing unit
We are introducing the concept of “dashboard specifications”, a set of data that determines what the dashboard is, its parts, its layout and its expected behavior. This specification is then transformed into a suitable representation that can be exposed to line of business users.
Building blocks are a developer’s best friend
There is only so much a development framework can do. The core of our process and framework are all about reducing time to value. While it’s true we can make developing dashboards easy, it is also true that a growing “dashboard development community” makes the task even easier.
We built on the concept of reusable parts that have proven to reduce software development complexity. It allows developers to share their work and re-use previously developed blocks in their own dashboards.
That sounds amazing! What are you doing?
We are still in early development and adoption stages of this framework. We are adding new exciting features all the time. We have conducted early successful PoCs with early adopters at IBM. However, we are now looking to get feedback from the worldwide dashboard developer community. If your dashboard development process needs to be improved and optimized, contact Thierry Mayeur, Senior Program Director, IBM Emerging Technologies at firstname.lastname@example.org.