For this post, I'm going to “nerd out.” I love using data. And I love monitoring, improving and discussing quality and safety. That was one of the fun things I did as hospital CMO. I also enjoy playing around with ways to display data, especially for lay people with a limited understanding of statistics. So today we will make our way to the “popular” Bubble Graph (well – it's popular with me at least), as I demonstrate how to prepare quality reports.

The CEO and the Board of Trustees (or Directors) definitely want to see meaningful reports about hospital performance. They truly want to see that performance is improving. Along with positive financial results, nothing will make the CEO and board happier than seeing that quality can be measured, is at an acceptable level, and is improving.

There are many ways to collect, analyze and present such data. I am going to show you how to prepare quality reports that my board found useful and interesting. Several of my board members told me that they really looked forward to seeing these reports.

First, Some Basics

Anyone who works in the quality arena will take the following caveats as a “given.” But for those who are just starting, here are some things to keep in mind.

It is not easy to reliably measure quality. And for hospitals, we generally do not measure “chart-level” outcomes. Our quality measurements are limited by the fact that we use billing data to collect mortality, complications, length of stay and other quality outcome measures.

UB-04 Billing Form

However, there is more than two decades of work by universities and quality vendors validating the use of billing information to measure outcomes. These researchers have produced an approach to quantify outcomes and apply statistical methods to the process. This enables us to compare the performance of facilities across the country. These methodologies are the same as those used by CMS and private quality reporting organizations such as Truven Health Analytics and HealthGrades.

You should verify that the following requirements are in place before you try to create reports like the ones I describe:

1. Your organization has a quality measurement tool that is interfaced to all of the necessary hospital systems.
2. The tool accurately draws data such as date of admission, date of discharge or death, source of the patient, destination on discharge, length of stay, etc.
3. The system(s) have the ability to distinguish between medical conditions that were present on admission (POA) from those that arose during a hospital stay.
4. Your hospital has robust medical records, coding and billing departments that accurately assign CPT codes, ICD-10 codes, and DRGs. You must be confident that those codes comply with the definitions from CMS and other quality agencies.
5. You have staff in the quality department that is knowledgable and can pull the outcome reports you need.

Other Considerations

The systems that I’m describing perform risk-adjustment of the data. It is therefore valid to compare outcomes to other organizations. In addition to reporting risk-adjusted rates, the outcomes are also expressed in one of two ways:

1. As an index, defined as the ratio of the observed rate over expected rate. If the observed and expected rates are the same, then this ratio is 1.0. A higher ratio indicates a rate worse than expected. A lower ratio than 1.0 indicates a better than expected rate.
2. As the difference between the actual and expected rates. If the difference is a positive number, the actual rate is worse than expected. If it is less that zero (a negative number) the rate is lower/better than expected.

Finally, most systems will also indicate whether the difference or ratio is statistically significant, and at what level of significance. When considering quality comparisons, we want to know differences at the 75th and 90th percentile. In research studies a significance level of 95% is desired. But we want to err on the side of identifying opportunities for improving quality. And we ignore opportunities with a significance of less than the 75th percentile, because they are more likely a result of chance alone.

Bringing It All Together to Prepare Quality Reports

As complicated as all of that sounds, the majority of hospitals have systems that meet those requirements. They can produce risk-adjusted outcomes and deviations from expected. And they can apply statistical testing for any of the high volume diagnoses. This allows us to prepare quality reports for our medical staff, our executive team and our board.

Using these systems, the quality department can create a report for the high volume diagnoses that lists the mortality, morbidity (complications), readmissions and length of stay.

I usually break these reports into two main categories. I produce one report that focuses on the mortality and length of stay for serious medical conditions. In this way, I can show the CEO and board a report that addresses pure quality (mortality) and a combination of quality and utilization (LOS).

For the high volume surgical or procedural admissions, since the mortality rates are very low, I find it more useful to present the morbidity (complication) rates and LOS data.

Start With a Spreadsheet

I use Microsoft Excel to create the data table and the graphs. But most spreadsheet software should work as well. I copy them to Microsoft PowerPoint for my presentation. I keep the slides as simple as possible. Usually, I include a minimum of labels so the information is clear. Yet I try to be parsimonious: showing everything that is needed, but nothing that is not needed.

Creating the bar graphs is simple: just highlight two columns (the list of diagnoses and the outcomes) and select the type of bar graph from the drop down menu. I generally use 2-D graphs for simplicity. Then, I save the charts on a separate page so that I can easily copy them to my slides.

The “bubble graph” is created by highlighting the three columns to be included (LOS, mortality and volume columns in this case), then choosing the prefered style from the “Scatter or Bubble Chart” option.

An example of a table that includes all of the information needed for these graphs follows. Note that these are completely fictional data that I generated for demonstration purposes only. But the information I present will look very much like this table, but for a larger number of conditions (20 or 25).

Using the information from the table, I create a bar graph to show the spread of outcomes from best to worst (in this case, lowest to highest mortality and LOS indexes). To do so requires sorting the data (lowest to highest) before creating each chart. Here is how those graphs would look:

Preparing the Bubble Graph

Finally, in order to bring it all together when I prepare quality reports, I combine all of the information into a single graph. This graph displays a grid showing the mortality index, the LOS index and the volume of cases. That graph is shown below.

I usually add small labels for each bubble indicating which diagnosis it represents. [Those must be added manually using individual text boxes. I did not do it for this post to save time. – VPE] I should also note that most of this work was delegated to a capable quality improvement nurse once I had created the first few iterations of these reports.

When I was presenting these data regularly for our board (twice a year), I would show the previous results and the current results. And I would skip through the bar graphs quickly, since all of that information was also incorporated into the bubble graph. But I wanted the board to understand where the bubble graph information came from.

Then I would just leave that bubble graph up on the screen while I discussed the great results of conditions sitting in the bottom left quadrant. And I would point out that it is possible to have great outcomes and a short length of stay (contrary to the opinions of some of my medical staff colleagues).

I would then describe the challenges of the conditions in the top right quadrant. And I would outline the procedures we had instituted to address the excess mortality and/or LOS. I might also comment on how we would prioritize working on the largest bubbles in the top right quadrant because they represented more cases.

Other Steps

It is best to preview these presentations with your CEO and senior executive team. That way, you can better anticipate questions the board may ask. And it gives you an opportunity to engage other team members in your quality improvement efforts.

You can combine any two variables (plus the volumes) to create similar slides. In addition to mortality and LOS, I would present a bubble graph of morbidity and LOS as noted above, or incorporate readmission rates. You may want to apply this method of analysis to metrics from patient safety or infection control. Even the finance team can find a use for these types of graphics.

Next Steps

Get creative and come up with some other combinations. Once you create some charts, why don’t you include a picture in the COMMENTS below, or just describe what you have created.

If you like this, Subscribe here.

Here is a short survey about your interests:  Survey Page

Email me directly at john.jurica.md@gmail.com with any questions.

If you're a nerd like me and like this post, please SHARE – SHARE – SHARE on social media.

Thanks so much and see you soon!