New York State Department of Health - NYPORTS
The New York Patient Occurrence
Reporting and Tracking System
Annual Report 2000/2001
Completeness of Reporting in NYPORTS
As noted in the 1999 NYPORTS Annual Report, completeness of reporting is an important concern when using NYPORTS for quality improvement and adverse event reduction purposes. If data are not reported completely and accurately, the occurrence frequency, or the occurrence rate (number of occurrences per number of discharges or number of occurrences per number of procedures of a given type) for hospitals and regions cannot be accurately computed. Incomplete reporting exacerbates the task of determining which occurrence codes should be scrutinized and makes it difficult to compare adverse event rates among regions and among hospitals.
In addition, it was noted that it is very difficult to assess the completeness of reporting in NYPORTS because, for nearly all of the adverse event codes, a "gold standard" database that includes cases that should be reported, does not exist.
Matching NYPORTS 605 Occurrences with SPARCS Data
The 1999 NYPORTS Annual Report examined the completeness of reporting for one NYPORTS code (code 605) that matches well to data contained in the Statewide Planning and Research Cooperative System (SPARCS). SPARCS is a database containing information on all inpatient stays in New York State acute care hospitals. This database contains up to 15 procedure codes for any inpatient stay, as well as the dates of these procedures and the date of discharge (which is the date of death for inpatients that died in the hospital). The completeness of reporting for code 605 was again examined as part of this report.
It should be noted that the definition of code 605 was modified as of June 1, 2000. There are two components to this change. One occurred in order to define the time frame of the event occurrence. Formerly, a report to NYPORTS for a death within 48 hours of a procedure was required. Since SPARCS collects data by date, the 48-hour time frame could not be precisely determined. The new time frame for code 605 was redefined as "occurring the same day as, or the first or second day after" the date of the procedure. If the adverse event occurs on the day of the procedure or during the next two calendar days, it is reportable in NYPORTS.
The other change relates to the kind of procedure performed. Initially, a 605 report was required for a death that occurred within 48 hours of any procedure performed in an operating room. Procedures done at the bedside, in specialty suites, in radiology or in other such sites were excluded. This resulted in inconsistent reporting since some hospitals do most, if not all, procedures in the operating room, while others have specialty areas where some surgical procedures can be performed. In the 1999 NYPORTS Annual Report, a common set of procedures that are customarily performed in an operating room were identified and chosen as "valid operating room procedures". These "valid operating room procedures" are consistent with those recognized by volume 3 of The International Classification of Diseases, 9th Edition, Clinical Modification (ICD-9-CM), a nationally accepted standardized classification system, which is used to classify procedures in SPARCS. Reporting is now required for ten specific categories of procedures (appendectomy, non-cardiac anteriography, cholecystectomy, endarterectomy, resection of large intestine, hysterectomy, colonoscopy, prostatectomy, replacement of joint of lower extremity and spinal fusion) regardless of the location where the procedure was performed.
During the analysis of NYPORTS occurrence code 605 during 2000 and 2001, it was found that non-cardiac anteriography and colonoscopy were commonly performed on an outpatient basis, and therefore, were not captured within the SPARCS system. Consequently, the assessment of reporting completeness for the NYPORTS code 605 in year 2000 and 2001 was confined to the other eight procedures (appendectomy, cholecystectomy, endarterectomy, resection of the large intestine, hysterectomy, prostatectomy, replacement of joint of lower extremity and spinal fusion).
Process for Measuring Reporting of 605 Occurrences
- Use SPARCS data submitted as of December 31st of the following year to identify all patients undergoing any of the following procedures, as either a principal procedure or a secondary procedure: appendectomy, cholecystectomy, endarterectomy, resection of the large intestine, hysterectomy, prostatectomy, replacement of joint of lower extremity and spinal fusion.
- Use SPARCS to determine which patients, undergoing any of the above 8 procedures, died on the day of the procedure or on either of the 2 days following the day of the procedure.
- Match all of the patients identified in SPARCS, with the 8 corresponding procedure codes, with patients who were reported for the year 2000 or 2001 in NYPORTS.
- The estimated completeness of reporting (percentage of cases that were reported) is the total of matched cases (SPARCS and NYPORTS) divided by the total number of identified cases for the 8 procedures in SPARCS.
Results of Process
Using the methods described above, 161 SPARCS cases were identified as potentially reportable under NYPORTS occurrence code 605, from June 1, 2000 to December 31, 2000. Of these patients, a total of 128 cases (80%) were reported by hospitals to NYPORTS as of December 31, 2001.
For 2001, there were 363 SPARCS cases identified as potentially reportable under NYPORTS occurrence code 605 for the entire year. Of these patients, a total of 265 cases (73%) were reported by hospitals to NYPORTS as of December 31, 2002.
In 1999, a total of 1,030 cases were judged to be reportable 605 occurrences based on SPARCS (using the 1999 definition of a 605 occurrence). Of these patients, a total of 167 (16.2%) were reported by hospitals to NYPORTS as of September 9, 2000. Thus, between 1999 and the second half of 2000, the completeness of reporting of code 605 increased from 16.2% to 80%.
When SPARCS was used to identify the 1999 occurrences of code 605 using the definition introduced in June 2000 (limited to the eight procedures mentioned above), there were a total of 320 cases identified in SPARCS. A total of 65 of these cases (20.3%) were reported in the 1999 NYPORTS database as of September 15, 2000. Thus, when the same definition of 605 cases was used in both years, the reporting percentage rose from 20.3% to 79%. When 605 data for 2001 was examined, there continued to be a high level of reporting from hospitals with 73% of the records identified in SPARCS being reported.
This increase in reporting percentages is a direct result of the efforts taken by the Department of Health to encourage reporting and hospital compliance with reporting responsibilities. As an example of these efforts, the Department contacted facilities that were identified as missing SPARCS validated 605 records in 2000. These facilities were provided information to locate and report these occurrences. A total of 49 occurrences among 38 facilities were identified by SPARCS validation. In the process of locating and re-evaluating these occurrences, the facilities were able to assess and make improvements to their own internal identification processes. 92% of these cases were reported into NYPORTS, while the remaining 8% of cases were determined not to match code 605 reporting criteria. A similar effort is being undertaken for 2001 SPARCS validated 605 records. Facilities that exhibit repeated non-compliance with reporting will be closely monitored and fines will be assessed in instances of repetitive non-compliance. Through this, and similar efforts, the Department is confident that reporting rates will continue to increase.
Examination of Regional Variation in Reporting NYPORTS Data
Another strategy for assessing the completeness of NYPORTS reporting is to examine differences in reporting frequency among large groups of hospitals within certain geographical regions of the state. In order to accomplish this goal, the number of inpatient discharges was compared with the number of NYPORTS cases per region. The result is the number of NYPORTS cases per 100,000 discharges. The table below reflects the results of data collection that was entered into the NYPORTS system as of December 31st of the following year. The regions are defined as Western New York, Finger Lakes, Central New York, Northeastern New York, Hudson Valley, Long Island, and New York City. The counties comprising these regions are listed in Appendix A.
NYPORTS Cases Submitted/100,000 Discharges by Region: 1999, 2000 and 2001
|Region||NYPORTS Cases||Acute Care Discharges||NYPORTS Cases per 100,000 Discharges|
|Year 1999||Year 2000||Year 2001||Year 1999||Year 2000||Year 2001||Year 1999||Year 2000||Year 2001|
|Central New York||2,235||2,760||2,695||198,910||200,449||202,668||1,124||1,377||1,330|
|New York City||4,849||8,267||10,814||1,077,136||1,124,189||1,148,175||450||735||942|
|Northeastern New York||2,051||2,701||3,042||165,706||155,525||167,780||1,238||1,737||1,813|
|Western New York||1,547||2,434||2,654||192,688||189,804||190,295||803||1,282||1,395|
For the year 1999, there were 16,939 NYPORTS cases reported for all of the various occurrence codes and 2,365,357 SPARCS cases submitted by December 31, 2000. The number of NYPORTS cases submitted per 100,000 discharges for 1999 in New York State was 716. This differs from the rate reported in the 1999 NYPORTS Annual Report (625 NYPORTS reports per 100,000 discharges) due to the use of updated data. The 1999 Annual Report used cases that were submitted by the hospitals as of September 15, 2000, while this report uses year 1999 data that was submitted by the hospitals as of December 31, 2000.
As indicated in the table above, a total of 24,368 NYPORTS cases for 2000 were submitted by December 31, 2001 for all of the various occurrence codes in NYPORTS, and a total of 2,427,392 patients were discharged from New York State acute care hospitals in 2000, based on data submitted by December 31, 2001. The number of NYPORTS cases submitted per 100,000 discharges for 2000 in New York State was 1,004.
Also indicated in the table above, a total of 28,689 NYPORTS cases for 2001 were submitted by December 31, 2002 for all of the various occurrence codes in NYPORTS, and a total of 2,475,480 patients were discharged from New York State acute care hospitals in 2001, based on data submitted by December 31, 2002. The number of NYPORTS cases submitted per 100,000 discharges for 2001 in New York State was 1,159.
The following bar chart compares the reporting for year 1999 (reported as of December 31, 2000), year 2000 (reported as of December 31, 2001) and year 2001 (reported as of December 31, 2002) by region, and for the entire state.
Increased Reporting Statewide
The statewide number of NYPORTS cases reported per 100,000 discharges in 1999 was 716. Consequently, the NYPORTS reporting rate per 100,000 discharges has risen by 61.9% between 1999 and 2001. It should be noted that definitions for several of the NYPORTS codes changed between 1999 and 2001. On inspection of the definitional changes, the codes subject to reporting changes based on definitional modifications are Code 401 (new acute pulmonary embolism), Code 402 (new documented deep vein thrombosis), both of which included for the first time re-admissions as of June 1, 2000, and the aforementioned Code 605. In addition, codes 911 and 912 changed in order to be more in line with Joint Commission on Accreditation of Heathcare Organizations (JCAHO) sentinel event definition.
These analyses of the number of NYPORTS events reported each year, which are based on the total number of records reported after similar periods of time had elapsed following the close of each year, demonstrate the impact of the Department of Health's efforts to improve reporting. Among these efforts were a press release on February 12, 2001 and a letter from the Commissioner, sent to all hospitals on February 22, 2001, advising all facilities to increase their reporting efforts. An indication of the effects of these efforts is that the total number of NYPORTS records for 1999 increased from 15,127 cases to 19,551 cases by November 7, 2001.
Increased Reporting by Region
The regional percentage increase in NYPORTS cases reported per 100,000 discharges between 1999 (reported as of December 31, 2000) and 2001 (reported as of December 31, 2002) ranged from 18.4% (from 1,124 to 1,330) in the Central New York region to 109.2% (from 450 to 942) in New York City.
For the year 2000, the number of NYPORTS cases submitted per 100,000 discharges per region varied by a factor of 2.4, whereas in 1999 the number of NYPORTS cases submitted per 100,000 discharges varied by a factor of 2.75. This regional reporting gap continues to decrease in 2001 where the number of NYPORTS cases submitted per 100,000 discharges per region varied by a factor of 1.9.
For the year 2000, three regions (Western New York, Finger Lakes, and Central New York) had similar reporting rates (ranging from 1,282 reported occurrences per 100,000 discharges in Western New York to 1,381 reported occurrences per 100,000 discharges in the Finger Lakes). Two other regions, Hudson Valley and Long Island had very similar reporting rates (990 and 979 occurrences per 100,000 discharges respectively). Northeastern New York had the highest reporting rate (1,737 occurrences per 100,000 discharges). New York City again reported the fewest occurrences per 100,000 discharges (735).
For the year 2001, Central New York and Western New York continued to have similar reporting rates (1,330 and 1,395 reported occurrences per 100,000 discharges respectively). Hudson Valley and Long Island also had very similar reporting rates (1,098 and 1,076 occurrences per 100,000 discharges respectively). Finger Lakes and Northeastern New York had the highest reporting rates (1,774 and 1,813 occurrences per 100,000 discharges respectively). New York City again reported the fewest occurrences per 100,000 discharges (942).
All regions except for New York City, Hudson Valley and Long Island Regions, are above the statewide average for reporting for years 2000 and 2001. As mentioned in the 1999 report, these variations in reporting frequencies could be a result of a variety of factors, including quality of care, types of hospital admissions, procedures performed, and accuracy and completeness of reporting. The precise contribution of each of these factors could not be estimated without a thorough medical record audit in each region. It is the Department of Health's view, however, that differences in types of patients and treatments performed should have a minor impact on the variations, because the size of the regions used to calculate rates were large enough to compensate for major differences in types of patients and treatments. Furthermore, it seems unlikely that there would be large differences among regions in the overall quality of care provided. It is likely that accuracy and completeness of reporting is the reason for most of the differences in the table above. In addition, since it is doubtful that there is widespread over-reporting of occurrences, under-reporting in the regions with the lowest reporting rates is the likely cause of variation.
In addition to the issuance of the 1999 NYPORTS Annual Report, the Department has taken several other steps to promote complete reporting. Ten days after the annual report was issued, Commissioner Novello called for all hospitals to conduct an internal review of records for 1999 and 2000 for the purposes of identifying occurrences that had been missed. Prior to completing this report, the Department has already been in contact with the lowest reporting hospitals, detailing their 2000 rates and regional comparisons. The Department strongly encouraged a re-evaluation of their internal NYPORTS identification process. Additionally, the Department encourages identification of NYPORTS occurrences by using an outside review agent to conduct medical record review. NYPORTS reportability will be evaluated and shared with the hospital with the intent that knowledge of unreported occurrences will lead to improvement in the facility's occurrence identification process. Finally, the Department, in its role as regulator of hospitals, will continue to identify non-compliance with the regulations and statutes regarding NYPORTS reporting, and will assess fines and mandate corrective measures where warranted.