Regional Consumer Guide Methodology

The consumer guides use domains to represent the quality of care for a group of people or condition. The domains often include several measures of care relevant to the group or condition. Plan scoring on a domain is determined using a four step process:

  1. Calculate the standardized difference between the plan and overall statewide rates for each measure.

    1. For sample-based measures - A standard z-score is used based on statewide variation:
      Standardized Difference = (plan rate - statewide rate)/SQRT[(statewide rate * (1 - statewide rate))/plan denominator]
    2. For population-based measures - The Nelson's H statistic is used, which is derived from Analysis of Proportions (ANOP) methodology:
      Standardized Difference = (plan rate - statewide rate)/(SQRT[statewide rate * (1 - statewide rate)] * SQRT[(statewide denominator - plan denominator)/(statewide denominator * plan denominator)])
    3. For CAHPS measures in the satisfaction domains - The significance of the plan's rate compared to the statewide rate is used for each measure to achieve points:
      • 3 points are achieved for plan measure rates significantly higher than the statewide measure rate,
      • 2 points for rates not significantly different than the statewide rate,
      • 1 point for rates significantly lower than the statewide rate
    4. For single measure domains - Domains with only one measure will use the significance of the plan's rate compared to the overall statewide rate rather than a z-scoe to determine plan performance.

  2. Determine the plan's average standardized difference for measures in the domain.

    1. A measure with a standardized difference greater than the +/- values of the measure limits (defined below) is trimmed to the respective +/- value of the measure limit multiplied by number of measures in the domain. Measures deemed to have "Failed Audit" are assigned the negative measure limit value. Measures with denominators smaller than 30 are not included in the respective domain.
      1. The measure limits correspond to the +/- 95% confidence limits for each measure
        1. For sample-based measures, the measure limits are +/- 1.96 (based on the Normal distribution of z-scores);
        2. For population-based measures, the measure limits are +/- 95% confidence limit of the corresponding t distribution (using Nelson's H statistic).
    2. The average standardized difference for a domain is calculated as follows:
      Average Standardized Difference = Sum of all the domain's measures' (trimmed) standardized differences / The number of measures in the domain

  3. Determine the domain limits for measures in the domain.

    1. The domain limits are determined by summing the measure limits (from step 2) and then dividing by the number of measures in the domain.

  4. Determine plan performance for domain (i.e., calculate the number of stars).

    1. For domains with more than one measure, the plan's average standardized difference for the domain (step 2) is compared to the domain limits (step 3) to determine the number of stars for the domain as follows:
      • Average Standardized Difference below the LOWER domain limit = 1 Star (below average)
      • Average Standardized Difference less than the UPPER domain limit and more than the LOWER domain limit = 2 Stars (average)
      • Average Standardized Difference more than the UPPER limit = 3 Stars (above average)
    2. For domains composed of a single measure, the number of stars is determined using the significance of the plan rate to the statewide rate. The domain earns 3 stars for plan rates significantly higher than statewide rate, 2 stars for rates not significantly different than the statewide rate, and 1 star for rates significantly lower than the statewide rate.
    3. Plan performance for the satisfaction domains is determined using the sum total of points achieved across measures within the domain, with stars determined based on the following:
      • Satisfaction Domain Total Points 3 to 4 points = 1 Star (below average)
      • Satisfaction Domain Total Points 5 to 7 points = 2 Stars (average)
      • Satisfaction Domain Total Points 8 to 9 points = 3 Stars (above average)

Once a star rating is calculated for each domain, an overall rating is calculated by taking the total number of stars received across all ranked domains and dividing that sum by the total number of possible stars. This rate is multiplied by 100 to create a percentage between 0 and 100.