8 Step Approach to Developing Clinical Dashboards

Topic: Supply Chain Dashboard. An example of using dashboards to assist with the complexities of a logistics system.

Dashboard Spy post: 8 Step Approach to Developing BI Dashboards. The CTO of a healthcare company shares his 8 step dashboard methodology in the Computerworld article Developing Clinical Dashboards.

Hospitals, medical centers and other health systems are increasingly using digital dashboard technology to provide relevant, up-to-the-minute information to clinicians in a visually rich format to improve the quality of patient care.

Designing and using clinical dashboards requires substantial physician involvement and a well-defined process. The University of Pennsylvania Health System, a.k.a. Penn Medicine, is currently in the midst of developing the Penn Data Store, a data warehouse and series of individual dashboards, to serve our clinicians and researchers. The Penn story may be of use to you and your organization as you move forward in the world of dashboards.

So starts the article that details how an organization used Oracle’s Business Intelligence Suite to create a single data storehouse with all of their patient, admin, financial and supply chain data.

Here are the 8 steps:

  1. Meet with users to determine data needs
  2. Design the presentation layer
  3. Design the semantic layer
  4. Design the physical layer
  5. Develop and test all 3 layers
  6. Perform QA
  7. Conduct a Pilot
  8. Begin general rollout

Here are some particulars. See the article for the full story.

1. Meet with users to determine major data needs

Many users and potential users of dashboards aren’t familiar with the systems’ power and capabilities. They typically use noninteractive spreadsheets and graphs that present data in fixed rows and columns and lack the flexibility of Web-based tools.

The first step in this phase was to inform clinicians of the many additional possibilities that a more powerful tool offers. We did this by first building sample dashboards to demonstrate the tool’s capabilities. We then asked the clinicians to identify several key measures, dimensions and filtering criteria for the dashboards they were interested in.

2. Design the presentation layer

For dashboards to deliver the most benefit, users must agree on presentation standards before the design phase begins. It’s crucial to achieve agreement on items such as color schemes, graphical objects and navigation standards so future dashboards will look, feel and behave consistently. This will improve user satisfaction and reduce training demands for each new dashboard.

Designers must think carefully about the hierarchy of the data they want depicted in the dashboards and what levels of the hierarchy will be visible. In our example, the hierarchy is Practice Physician Patient. Depending on a user’s security status, he may or may not be able to see the patient data.

Once the hierarchy and associated measures such as year, month and practice name were grouped into a dashboard page, we selected the graphical elements. We chose histograms, line graphs, pie charts and simple tables to present the data in an intuitive fashion that met users’ needs and simplified navigation through the dashboard pages.

3. Design the semantic layer

The semantic layer maps the presentation layer to the physical layer. Developers prove their worth in the design of this layer. Defining patient populations by disease categories, grouping drugs hierarchically by therapeutic area and organizing physical locations are examples of the challenges that the semantic layer’s designer faces. Users play a key role in formulating those definitions. In-depth knowledge of both the presentation layer design and the physical data models is essential.

4. Design the physical layer

As mentioned, try not to change the design of the physical layer. Whenever possible, avoid creating an entire new physical data structure, because doing so generates the need for additional extract, transform and load (ETL) steps each time the clinical data warehouse is updated. Redundantly storing data produces additional storage, backup and maintenance costs and opens up the risk that duplicate copies of data won’t be updated with the same frequency as the original.

5. Develop and test all three layers

Users who are new to the dashboard development process will likely need to see how the systems operate with real data. We have found it useful to introduce clinicians to a working prototype to gain early feedback on the design. Regular weekly demo and review sessions then help developers refine and test the design. When you’re engaged in this phase of the project, take care to manage scope creep, since participants might be tempted to request new capabilities or data that weren’t in the original design. Put off responding to such requests until a subsequent release of the dashboard.

6. Perform quality assurance

Here are several quality-assurance requirements we developed at Penn Medicine:

  • Use data from the actual data warehouse, rather than simulated or test data.
  • Include monthly and quarterly data warehouse updates during the QA process, especially when the data being used in dashboards will cross calendar months, quarters and years. A year-end data update would be ideal, but most projects can’t wait that long.
  • Compare dashboard data to the original source systems, to ensure that no translation or presentation errors were introduced during development.
  • Test system performance and response times with large amounts of data to ensure that the dashboard responds effectively to users’ needs.

7. Conduct pilot tests

Before making a dashboard available, we ask a small group of users to pilot-test it for a few weeks, as a short extension of the QA phase. This provides additional information on usability, performance and quality. If the pilot group recommends moving forward, we proceed to the general rollout. If the pilot group identifies problems, the development team resolves them.

8. Begin general rollout

This phase involves these major components:

  • User orientation: New users may need a brief training session. Those who have used dashboards in the past should be able to use the new dashboard with no assistance — if the presentation layer was designed well.
  • Support documentation: Gather design and operational specifications and post them on support sites.
  • Help-desk hand-off: Write scripts and give them to the help desk for guidance in responding to support calls.

2008 BI Magic Quadrant Diagram

The recently released 2008 Gartner BI Magic Quadrant analysis focuses on the consolidation that is underway in the BI reporting and dashboarding space. The diagram below seems straight-forward, but you must realize that with the speed of acquisitions in our space, it was already obsolete when released. The Gartner analysis missed the late breaking news of the SAP/Business Objects acquisition.

Dashboards By Example Update: Readers are advised to visit the Gartner link posted above as they have updated their report. As they mention regarding the changing vendor landscape:

“This document is an updated version of the document published on 1 February 2008.”

Market Overview – One “macro trend” defined the BI platforms arena in 2007 — market consolidation — making it the most turbulent year, so far, in business intelligence.

As anticipated in last year’s Magic Quadrant and other Gartner research (see, for example, “Market for Business Intelligence Platforms: Round Two of Consolidation Begins”), large application and software infrastructure vendors completed or initiated significant strategic acquisitions in the BI platform market in 2007:

In July, Oracle completed its purchase of Hyperion. An example of straight market consolidation, this move brought two competing BI platforms, Hyperion System 9 and Oracle Business Intelligence Enterprise Edition, both Leaders on the 2007 Magic Quadrant, under Oracle ownership and expanded Oracle’s BI resources and staffing. (See “Hyperion Purchase Will Strengthen Oracle in BI Platform and CPM Suites Markets.”)

In October, SAP announced its acquisition of Business Objects, which will expand its presence into the “business user” market, which SAP defines as being made up of business roles involved in analytical and information-intensive activities. This acquisition, which was completed in January 2008, fills a significant gap in SAP’s query and reporting tools portfolio, but represents a major strategic shift away from “slot-in” technology buys and organic software development. (See “SAP’s Planned Business Objects Buy Signals Strategic Shift.”)

As the year closed, Cognos completed its acquisition of Applix, and its in-memory online analytical processing (OLAP) engine. It also agreed to be bought by IBM. Though not strictly a consolidating move, this acquisition is significant, as it will end IBM’s abstinence from the BI platform and applications market. IBM has repeatedly stated that it will focus on the infrastructure and the middleware layer, and that it will only “enable” applications. While a BI platform includes many infrastructure components, the Cognos BI and performance management applications will fill a big void in IBM’s stack. (See “IBM Aims for the Business Intelligence Endgame With Cognos.”)

Megavendors are beginning to dominate the BI market — in less than one year, Microsoft, Oracle, SAP and IBM will have gone from accounting for a quarter of the market to owning over two-thirds of it. As such, the “Magic Quadrant for Business Intelligence Platforms, 2008″ reflects the tipping point at which the market moves away from being led by independent BI vendors like Business Objects and Cognos, to one where the megavendors rule. Future BI investment decisions will be tethered much more closely to strategic sourcing and stack-led factors, and will be more influenced by organizational relationships with application and infrastructure vendors.

During the same period, “flattening” factors — including the maturing of Microsoft’s BI portfolio, the adoption of Web 2.0 techniques, the growth of open-source BI and the continued emergence of software as a service (SaaS) offerings — have made BI capabilities more accessible and affordable than they have ever been. As a result, this Magic Quadrant includes commentary on some emerging vendors which, while not yet meeting the inclusion criteria for the Magic Quadrant itself, offer a viable alternative for some BI use-cases.

Here is the Magic Quadrant Diagram:

Gartner BI Magic Quadrant 2008 Diagram

Here are some snippets from the report itself. You can read more by using the link at the top of this post:

Continue reading

Supply Chain Management Dashboard Screenshot – Key Performance Indicators & Metrics for Forecast, Inventory, Orders

Thanks goes to the Dashboard Spy who sent me these enterprise dashboard screenshots for a supply chain management system. Lots of interesting KPIs to study here. The application is divided into the following tabs: Home, Alerts, Forecast, Inventory, Reports and KPIs. The first dashboard screenshot shows the home page of the system. The left side acts as both navigation and summary by including the number of each item. For example, in the Tasks portlet, we see that there are 4 Orders to Ship. I presume that clicking on that link will bring up those orders. Likewise, the KPIs section shows the status (red/green/yellow), the KPI name (which is hyperlink to the KPI detail), and the actual value. On the right side are 2 KPI charts. The first is a graph of the Perfect Order Rate (more on how to calculate this KPI later below in this post). The second is the Back Order Rate.

Enterprise Dashboard Supply Chain

Here is the orders screen:

Orders Dashboard

This is the sales order drill-down:

Sales Order Detail

And here is the KPI Dashboard. Forecast KPIs include: Forecast Accuracy, Supply Plan Variation, Ship to Commit Percent. Inventory KPIs include Inventory Turns, Stock Out Rate, Inventory Aging, Average Days on Hand, Inventory Dollars, Target Inventory Rate, Excess Inventory Average. The Order KPIs include: Perfect Order Rate, Delayed Shipment Rate, Open Returns, Order Fill Rate, Backorder Rate, Closed Returns and Exceptions per Item.

KPI Dashboard

Homework: Do you know what is meant by “Perfect Order”? This supply chain metric calculates the error-free rate of each stage of an order. Here is an explaination from http://www.supplychainmetric.com/perfect.htm

“This measure should capture every step in the life of an order. It measures the errors per order line. An example: your warehouse picks and ships the wrong item. Once the customer receives the order and notices the error, they contact the manufacturer and notify them of the mistake. The manufacturer then enters a credit for the item not shipped and an invoice for the item shipped in its place. For almost all errors that occur, a corrective credit is issued. It is through an analysis of these credits that you derive your metric. Most systems require a “reason code” to be used when entering a credit. Tracking these reason codes and assigning them to a category allow you to group them for the Perfect Order Measure. Sample calculation:

Order Entry Accuracy: 99.95% Correct (5 errors per 1000 order lines)
Warehouse Pick Accuracy: 99.2%
Delivered on Time: 96%
Shipped without Damage: 99%
Invoiced Correctly: 99.8%

Therefore, the Perfect Order Measure is 99.95% * 99.2% * 96% * 99% * 99.8% =   94.04%”

If you want to learn more about the calculations behind supply chain management, start with these books on supply chain metrics.

Also, take a look at this novel approach to presenting supply chain management KPIs and metrics via the desktop. Klipfolio offers a way to use the PC Desktop as a dashboarding platform. To resolve the complexity and confusion associated with a complex logistics system, a supply chain dashboard becomes more necessity than luxury. And the desktop approach makes it especially compelling. Check it out at Klipfolio.com

So what or who is The Dashboard Spy? As his about page states, The Dashboard Spy is just a guy interested in the design of enterprise dashboards. He could not find any executive dashboard design source books (or even screenshots of real business dashboards) and so set about creating his own. Finally convinced to post his extensive collection of dashboard screenshots online, he was amazed to find how popular it has become. If you have a nice screenshot of a digital dashboard, balanced scorecard, or any business intelligence graphic to share, please send an email to info _at_ dashboardspy.com. Also check out The Dashboard Spy’s favorite books on business dashboards.