Managed Long Term Care Consumer Guide Methodology

The consumer guides use domains to represent the quality of care for a group of people or condition. The domains may include several measures of care relevant to the group or condition. Plan scoring on a domain is determined using a stepwise process.

A. Domains with only one measure.

  1. For each sample-based measure, the statewide rate is compared to a 95% confidence interval (CI) about the plan rate to determine whether there is a significant difference between the statewide and plan rates. A Z statistic is used.

    Upper CI = plan rate + 1.96 * SQRT(plan rate * (1 - plan rate) / plan denominator)
    Lower CI = plan rate - 1.96 * SQRT(plan rate * (1 - plan rate) / plan denominator)

  2. For each population-based measure, the plan rate is compared to decision limits (DL) about the statewide rate to determine whether there is a significant difference between the statewide and plan rates. Nelson's H statistic and Analysis of Proportions (ANOP) methodology are used.

    Halpha = the quantile from the t distribution based on a probability = 1 - (1 - (1 - 0.05) (1/number of plans)) / 2), and degrees of freedom = statewide denominator - plan denominator

    Upper DL = statewide rate + halpha * SQRT(statewide rate * (1 - statewide rate)) * SQRT((statewide denominator - plan denominator) / (statewide denominator * plan denominator))

    Lower DL = statewide rate - halpha * SQRT(statewide rate * (1 - statewide rate)) * SQRT((statewide denominator - plan denominator) / (statewide denominator * plan denominator))

B. Domains with multiple measures.

  1. Measures with denominators smaller than 30 are excluded from the respective domain.
  2. There are no domains with multiple measures that contain sample-based measures so methods that would be used for that situation are not described in this document.
  3. Determine the plan's measure standardized difference.
    1. For population-based measures:

      Standardized Difference = (plan rate - statewide rate) / (SQRT (statewide rate * (1-statewide rate)) * SQRT ((statewide denominator - plan denominator) / (statewide denominator * plan denominator)))

  4. Determine the plan's measure limits.
    1. For population-based measures, the measure limits are positive and negative halpha values, calculated as described in section A above.
  5. Trim the measure standardized differences.
    1. If the standardized difference is greater than the upper value or less than the lower value of the measure limit, the standardized difference is trimmed to the respective upper or lower value of the measure limit multiplied by the number of measures in the domain.
  6. Determine the plan's domain average standardized difference.

    Average Standardized Difference = (sum of all the domain's measures' standardized differences after trimming) / (number of measures in the domain)

  7. Determine the plan's domain limits.

    Domain limits = (sum of all measure limits) / (number of measures in the domain)

  8. Compare the plan's domain average standardized difference to the plan's domain limits to determine whether the difference between the plan and statewide rate is significant.

C. Plan performance and number of stars.

  1. Domains with only one measure.
    1. The significance test described in section A above is used to determine the number of stars.
      1. 3 stars (above average) = plan rate is significantly higher than the statewide rate
      2. 2 stars (average) = plan rate is not significantly different from the statewide rate
      3. 1 star (below average) = plan rate is significantly lower than the statewide rate
  2. Domains with multiple measures.
    1. The significance test described in section B above is used to determine the number of stars.
      1. 3 stars (above average) = plan average standardized difference is equal to or above the upper domain limit
      2. 2 stars (average) = plan average standardized difference is between the upper and lower domain limits
      3. 1 star (below average) = plan average standardized difference is equal to or below the lower domain limit