New York State Community Health Indicator Reports (CHIRS) - Methodology and Limitations

Index

Return to top of page

Types of Estimates

1. Percentage/age-adjusted percentage:
Percentages are calculated per 100 population (e.g., the percentage of infants exclusively breastfed in the hospital represents the number of infants that were fed exclusively with breast milk among 100 live infants born in the hospital).
In some instances, the percentages were age-adjusted to the U.S. 2000 standard population using appropriate age distributions. 1 Age-adjustment is a process that is performed to allow communities with different age structures to be compared.2
2. Weighted percentage/age-adjusted weighted percentage:
Weighted percentages were generated for survey data (e.g., Expanded Behavioral Risk Factor Surveillance System, Oral Health Survey of 3rd Grade Children; US Census Bureau's Small Area Estimates) which ensures that the data are as representative of New York's population as possible. Weighted estimates are shown as a percentage (%) and corresponding 90% or 95% confidence intervals (CI) are presented when available
The weighted percentages were age-adjusted to the U.S. 2000 standard population using appropriate age distributions. 1 Age-adjustment is a process that is performed to allow communities with different age structures to be compared.2
3. Rate/age-adjusted rate:
A rate is a measure of the frequency with which an event occurs in a defined population over a specified period of time. Rates used for the CHIR indicators are per 1,000, 10,000 or 100,000 population.
The rates were age-adjusted to the U.S. 2000 standard population using appropriate age distributions. 1 Age-adjustment is a process that is performed to allow communities with different age structures to be compared.2

Return to top of page

Unstable Estimates

Multiple years of data were combined to generate more stable estimates when the number of events for an indicator was small (i.e., rare conditions).
The relative standard error (RSE) is a tool for assessing reliability of an estimate. A large RSE is produced when estimates are calculated based on a small number of cases.2 Estimates with large RSEs are considered less reliable than estimates with small RSEs. The National Center for Health Statistics recommends that estimates with RSEs greater than 30% should be considered unreliable/unstable.3

The RSE is calculated by dividing the standard error of the estimate by the estimate itself, then multiplying that result by 100. The RSE is expressed as a percent of the estimate.

For notation purposes, an asterisk (*) symbol is used to indicate that a percentage, rate, or ratio is unreliable/unstable. This usually occurs when there are less than 10 events in the numerator (RSE is greater than 30%).

Return to top of page

Direction of Indicator Estimates

CHIR indicators fall into two categories with regard to the direction of their estimates. Sometimes lower estimates are better (e.g., the percentage of premature deaths before age 75 years, or cardiovascular hospitalizations) and in other cases higher estimates are better (e.g., the percentage of the population with health insurance, or the percentage of infants exclusively breastfed in the hospital).

This direction of the CHIR indicator is important to note because the county bar chart and map use color categories that are based on the direction of the indicator.

Return to top of page

Grouping County Estimates into Three Categories for the County Maps

Color Categories Defined

For each CHIRS indicator, county estimates are grouped into three categories: yellow, green, and blue. The three colors represent the quartile distribution of estimates for the counties ordered from those doing the best to those doing the worst.

For CHIRS indicators where lower estimate are better (e.g., percentage of premature deaths before age 75 years or the age-adjusted rate of cardiovascular disease hospitalizations):

  • The YELLOW category includes counties who are performing the best (i.e., 50% of counties with the lowest estimates; those in quartile 1 and quartile 2) and is the most favorable category for a county's estimate to be in.
  • The BLUE category includes counties who are performing the worst (i.e., 25% of counties with the highest estimates; those in quartile 4) and is the least favorable category for a county's estimate to be in.
  • The GREEN category includes counties who are performing in the middle (i.e., 25% of counties or those in quartile 3).

For CHIRS indicators where higher estimates are better (e.g., the percentage of the population with health insurance or the percentage of infants exclusively breastfed in the hospital):

  • The YELLOW category includes counties who are performing the best (i.e., 50% of counties with the highest estimates; those in quartile 3 and quartile 4) and is the most favorable category for a county's estimate to be in.
  • The BLUE category includes c counties who are performing the worst (i.e., 25% of counties with the lowest estimates; those in quartile 1) and is the least favorable category for a county's estimate to be in.
  • The GREEN category includes counties who are performing in the middle (i.e., 25% of counties or those with estimates in quartile 2).

Return to top of page

Data Suppression Rules for Confidentiality

Results are not shown (i.e., suppressed) when issues of confidentiality exist. Suppression rules vary depending on the data source and the indicator. An 's' notation indicates that the data did not meet reporting criteria.

Table 1. Summary of Data Suppression Rules

Data Sources Suppression Criteria
Bureau of Dental Health (BDH) Margin of error>20% or Denominator <50
Behavioral Risk Factor Surveillance System (BRFSS) and Expanded BRFSS Denominator <50 or Numerator < 10
Vital Statistics - Death Records Denominator population <30
Statewide Perinatal Data System (SPDS) - birth records Denominator population or total Births <30
AIDS/HIV Numerator 1-2 cases
Statewide Planning and Research Cooperative System (SPARCS) - ED and hospital records Numerator 1-5 cases
Office of Quality and Patient Safety (QARR and eQARR) Denominator <30 and Numerator >0 cases
Cancer Registry Numerator 1 - 5 cases

Return to top of page

References

  1. Klein RJ, Schoenborn CA. Age adjustment using the 2000 projected U.S. population. Healthy People Statistical Notes, no. 20. Hyattsville, Maryland: National Center for Health Statistics. January 2001. (see: www.cdc.gov/nchs/data/statnt/statnt20.pdf)
  2. About Age Adjusted Rates, 95% Confidence Intervals and Unstable Rates (see: www.health.ny.gov/statistics/cancer/registry/age.htm)
  3. Klein RJ, Proctor SE, Boudreault MA, Turczyn KM. Healthy People 2010 criteria for data suppression. Statistical Notes, no 24. Hyattsville, Maryland: National Center for Health Statistics. June 2002. (see: www.cdc.gov/nchs/data/statnt/statnt24.pdf
  4. Statistical Significance (see: www.health.ny.gov/statistics/chac/chai/docs/statistical_significance.pdf)
  5. One-sided 95% confidence interval (see: http://www.graphpad.com/guides/prism/6/statistics/index.htm?one_sided_confidence_intervals.htm
  6. Guidelines for using confidence intervals for public health assessment, Washington State Department of Health, (see: http://www.doh.wa.gov/Portals/1/Documents/1500/ConfIntGuide.pdf)