Dynamic Dashboards from Jupyter Notebooks
A Jupyter notebook is an excellent means of capturing code, text, widgets, graphics, and other rich media in a computational narrative that distills data into insights. Today, notebook authors can share their notebooks for others to view and run in the Jupyter Notebook web application. Authors can also transform their notebooks into a variety of static formats for ease-of-viewing in common tools like web browsers or PDF readers.
In our consulting engagements, we’ve often needed to do more than deliver a static report or notebook document to our clients. Rather, clients have asked us to build dynamic dashboards which allow users to interact with insights in ways not possible in a static document. For example, one client asked us to mine unstructured text to identify critical events, create a visualization showing the time, magnitude, and location of those events, and provide them with a way to filter which events appear in the visualization. While we did most of the data exploration and visualization prototyping in notebooks, we had to step outside to build a traditional web application which our client could use. Unfortunately, this involved much copy/pasting and rewriting of notebook code.
We’ve recently started researching ways of transforming notebooks directly into dashboards We’ve built a proof-of-concept extension for Jupyter Notebook that builds upon existing community work like Thebe and the recent refactoring of the Jupyter Notebook in order to demonstrate the concept of producing dynamic dashboards from notebooks.
Here’s a brief walkthrough of creating a simple scotch recommendation app using notebooks, our extension, and a public whiskey data set.
After computing similarity metrics, creating visualizations, and wiring up the interactivity in the notebook, we can flip to a new Dashboard Mode to drag/drop and resize notebook cells in a grid layout.
Download / Deploy
We can next transform the notebook into a web app that retains our layout and can be deployed outside our notebook authoring environment. One option allows us to download a zip file containing the frontend assets for our dashboard, a Dockerfile for building it into a container, and a manifest for pushing it to Cloud Foundry. And, of course, the deployment can be further automated for specific cloud providers like IBM Bluemix.
Under the covers, our extension uses Jupyter’s nbconvert to turn the notebook into a simple HTML page that loads the Thebe JS library.
When a user visits the deployed dashboard frontend, Thebe contacts a separate Jupyter server to execute the dashboard code on a kernel. The Jupyter server can reside elsewhere in the cloud, such as on a Spark cluster close to the data it needs.
We hope that by open sourcing this prototype, we can help water the seeds already planted for a robust dashboarding capability in Jupyter Notebook. Much work lies ahead in maturing the security, execution model, scalability, and widget ecosystem of this approach, and we’re excited to start the discussion on how to proceed with the community.