New York State Community Health Indicator Reports (CHIRS) - Methodology and Limitations

Index

Return to top of page

Types of Estimates

1. Percentage/age-adjusted percentage:
Percentages are calculated per 100 population (e.g., the percentage of infants exclusively breastfed in the hospital represents the number of infants that were fed exclusively with breast milk among 100 live infants born in the hospital).
In some instances, the percentages were age-adjusted to the U.S. 2000 standard population using appropriate age distributions. 1 Age-adjustment is a process that is performed to allow communities with different age structures to be compared.2
2. Weighted percentage/age-adjusted weighted percentage:
Weighted percentages were generated for survey data (e.g., Expanded Behavioral Risk Factor Surveillance System, Oral Health Survey of 3rd Grade Children; US Census Bureau's Small Area Estimates) which ensures that the data are as representative of New York's population as possible. Weighted estimates are shown as a percentage (%) and corresponding 90% or 95% confidence intervals (CI) are presented when available
The weighted percentages were age-adjusted to the U.S. 2000 standard population using appropriate age distributions. 1 Age-adjustment is a process that is performed to allow communities with different age structures to be compared.2
3. Rate/age-adjusted rate:
A rate is a measure of the frequency with which an event occurs in a defined population over a specified period of time. Rates used for the CHIR indicators are per 1,000, 10,000 or 100,000 population.
The rates were age-adjusted to the U.S. 2000 standard population using appropriate age distributions. 1 Age-adjustment is a process that is performed to allow communities with different age structures to be compared.2

Return to top of page

Unstable Estimates

Multiple years of data were combined to generate more stable estimates when the number of events for an indicator was small (i.e., rare conditions).
The relative standard error (RSE) is a tool for assessing reliability of an estimate. A large RSE is produced when estimates are calculated based on a small number of cases.2 Estimates with large RSEs are considered less reliable than estimates with small RSEs. The National Center for Health Statistics recommends that estimates with RSEs greater than 30% should be considered unreliable/unstable.3

The RSE is calculated by dividing the standard error of the estimate by the estimate itself, then multiplying that result by 100. The RSE is expressed as a percent of the estimate.

For notation purposes, an asterisk (*) symbol is used to indicate that a percentage, rate, or ratio is unreliable/unstable. This usually occurs when there are less than 10 events in the numerator (RSE is greater than 30%).

Return to top of page

Direction of Indicator Estimates

CHIR indicators fall into three categories with regard to the direction of their estimates. Higher estimates in some indicators mean poorer health or greater risk of poorer health (e.g., the percentage of premature deaths before age 75 years, or cardiovascular hospitalizations). Lower estimates in some other indicators mean poorer health or greater risk of poorer health (e.g., the percentage of the population with health insurance, or the percentage of infants exclusively breastfed in the hospital). A few indicators do not have a direction (e.g., total population, percent of births which were first birth), and the higher or lower estimates have no meaning in terms of health or heath risk.

Return to top of page

Grouping County Estimates into Three Categories for the County Maps

Color Categories Defined

For each CHIRS indicator, county estimates are grouped into three categories: yellow, green, and blue. The three colors represent the quartile distribution of estimates for the counties, ordered from counties with the lowest percent of population with poorer health or at risk of poorer health, to counties with the highest percent of population with poorer health or at risk of poorer health.

For CHIRS indicators where higher estimates mean poorer health or greater risk of poorer health (e.g., percentage of premature deaths before age 75 years or the age-adjusted rate of cardiovascular disease hospitalizations):

  • The YELLOW category includes counties which have a lower percent/rate of population with poorer health or risk of poorer health (i.e., 50% of counties with the lowest estimates; those in quartile 1 and quartile 2)
  • The BLUE category includes counties which have a higher percent/rate of population with poorer health or risk of poorer health. (i.e., 25% of counties with the highest estimates; those in quartile 4)
  • The GREEN category includes counties which have a percent/rate of population in the mid-range. (i.e., 25% of counties or those in quartile 3)

For CHIRS indicators where lower estimates mean poorer health or greater risk of poorer health (e.g., the percentage of the population with health insurance or the percentage of infants exclusively breastfed in the hospital):

  • The YELLOW category includes counties which have a lower percent/rate of population with poorer health or risk of poorer health. (i.e., 50% of counties with the highest estimates; those in quartile 3 and quartile 4)
  • The BLUE category includes counties which have a higher percent/rate of population with poorer health or risk of poorer health. (i.e., 25% of counties with the lowest estimates; those in quartile 1)
  • The GREEN category includes counties which have a percent/rate of population in the mid-range. (i.e., 25% of counties or those with estimates in quartile 2)

Some indicators do not fall into the two types of indicators listed above, such as population, and percent of births which were first birth. The county dial is only a visual representation of where the county is in relation to other counties, e.g. larger or smaller, higher or lower.

Return to top of page

Data Suppression Rules for Confidentiality

Results are not shown (i.e., suppressed) when issues of confidentiality exist. Suppression rules vary depending on the data source and the indicator. An 's' notation indicates that the data did not meet reporting criteria.

Table 1. Summary of Data Suppression Rules

Data Sources Suppression Criteria
Bureau of Dental Health (BDH) Margin of error>20% or Denominator <50
Behavioral Risk Factor Surveillance System (BRFSS) and Expanded BRFSS Denominator <50 or Numerator < 10
Vital Statistics - Death Records Denominator population <30
Statewide Perinatal Data System (SPDS) - birth records Denominator population or total Births <30
AIDS/HIV Numerator 1-2 cases
Statewide Planning and Research Cooperative System (SPARCS) - ED and hospital records Numerator 1-5 cases
Office of Quality and Patient Safety (QARR and eQARR) Denominator <30 and Numerator >0 cases
Cancer Registry Numerator 1 - 5 cases

Return to top of page

References

  1. Klein RJ, Schoenborn CA. Age adjustment using the 2000 projected U.S. population. Healthy People Statistical Notes, no. 20. Hyattsville, Maryland: National Center for Health Statistics. January 2001. (see: www.cdc.gov/nchs/data/statnt/statnt20.pdf)
  2. About Age Adjusted Rates, 95% Confidence Intervals and Unstable Rates (see: www.health.ny.gov/statistics/cancer/registry/age.htm)
  3. Klein RJ, Proctor SE, Boudreault MA, Turczyn KM. Healthy People 2010 criteria for data suppression. Statistical Notes, no 24. Hyattsville, Maryland: National Center for Health Statistics. June 2002. (see: www.cdc.gov/nchs/data/statnt/statnt24.pdf
  4. Statistical Significance (see: www.health.ny.gov/statistics/chac/chai/docs/statistical_significance.pdf)
  5. One-sided 95% confidence interval (see: http://www.graphpad.com/guides/prism/6/statistics/index.htm?one_sided_confidence_intervals.htm
  6. Guidelines for using confidence intervals for public health assessment, Washington State Department of Health, (see: http://www.doh.wa.gov/Portals/1/Documents/1500/ConfIntGuide.pdf)