Gain Analytics Agility on Apache Spark for IBM z Systems

  When people discuss real-time big data domains, rarely do they understand the realities of where daily transactional data occurs around the world. To give you an idea of the volumes we’re looking at, consider that for every second of the day worldwide, there are: 250 iTunes downloads 3600 Instagram photos taken 7000 Tweets sent 30000 Facebook likes 46000 YouTube […]

Read more

Last Mile of Dashboard Development

In a world where “Big data is the new natural resource”, it is a must to create applications taking advantage of this new knowledge-full resource. Brave people started venturing into this unknown world, They are using new and exciting tools like Notebooks or Spark engines. However, developers are still banging their heads against a wall when time comes to build […]

Read more

Jupyter DeclarativeWidgets make data exploration easy!

Over the past year, we have built many demonstrations using declarativewidgets to create dashboards and good-enough applications in the Jupyter Notebook. We’ve discovered several patterns that are in great need of simplification. One of these patterns is data exploration. By data exploration, I mean the ability for the user to query and visualize a data set. In declarativewidgets, we make it simple to bind visual elements to […]

Read more

Visualizing Big Data with Spark and Scala

While exploring data analytics with Apache Spark, the team came to the realization that there are many Python examples, but resources for Scala are somewhat lacking. In particular, there are few data visualization examples in Scala. Python’s predominant visualization module is Matplotlib, but we struggled to find a Scala library that offered the same breadth of functionality and granularity of control. Brunel In our […]

Read more

Load Your Data Into a Jupyter Notebook

You’ve heard all the flashy statistics about big data, like how every day more than 2.5 quintillion bytes of data is created and that more data has been created in the last two years alone than every previous year combined (IBM). Here’s another one to add to the list: 99.5% of newly created data is never analyzed (MIT). Only half a […]

Read more

Using Remote Kernels with Jupyter Notebook Server

Jupyter Notebook uses kernels to execute code interactively. The Jupyter Notebook server runs kernels as separate processes on the same host by default. However, there are scenarios where it would be necessary or beneficial to have the Notebook server use kernels that run remotely. A good example is when you want to use notebooks to explore and analyze large data […]

Read more

Spark on z/OS and Jupyter: fast, flexible analysis of mainframe data

Many enterprises are faced with the need to expand data processing access to users without impacting mission-critical transactional application environments. The trending approach to this problem is to move the data from these systems of record to a data warehouse. Moving data-at-rest to a mirrored data repository for analytics can yield costly side-effects such as expensive migration workloads, data concurrency and […]

Read more

jStart Spark Data Analysis Projects for Clients

  During the IBM InterConnect 2016 conference, Scott Laningham asked me about the mission of the IBM jStart team and about our team’s Spark data analytics projects with various clients such as SolutionInc and USA Cycling Women’s Team Pursuit. Why Spark?  Spark provides data analysts, data scientists, and even line of business users the ability to find new patterns in data, […]

Read more

Case Study: Delivering Transportation Insights using Jupyter Notebooks, Interactive Dashboards, and Apache Spark

Our IBM Cloud Emerging Technologies team recently worked with Executive Transportation Group (ETG) to analyze executive car service trips in New York City. ETG wanted to simulate potential changes to its driver dispatch algorithm, and assess the impact of those changes on its operations. The goal was to identify changes that might increase efficiency and have a positive impact on […]

Read more

Unleashing Exploration on Enterprise Data

Enterprise customers have huge investments in transactional data systems, yet they struggle to provide their users with flexible and timely exploratory access to this data. One solution to this problem is to empower these users with the ability to use Jupyter Notebooks and Apache Spark running natively on z/OS to federate analytics across business critical data as well as external […]

Read more
1 2