Uses and Limitations of Routine Hospital Admission/Separation Records for Perinatal Surveillance
Shi Wu Wen, Shiliang Liu, Sylvie Marcoux and Dawn Fowler

Abstract

This study examined the quality of data for delivering mothers and their newborns (April 1, 1984 to March 31, 1995) recorded by the Canadian Institute for Health Information (CIHI). The number of illogical and out-of-range values in the CIHI data were quite few; the occurrence of maternal and infant diseases estimated from CIHI data was quite similar to that in the literature; and major medical/obstetric complications recorded in CIHI were, in general, good predictors of adverse pregnancy outcomes. The authors conclude that CIHI data contain some of the information pertinent to perinatal surveillance that may be used to monitor maternal and infant health and to assess intrapartum care and hospital resource utilization. To adequately monitor and analyze patterns of health determinants and outcomes in all pregnant women and their infants in Canada, additional data collection mechanisms are needed to cover all recognized pregnancies and to collect antenatal and postpartum information and more detailed information on intrapartum care.

Key words: Canada; data collection; epidemiologic methods; infant, newborn; mothers; pregnancy outcome

Background and Introduction

The Bureau of Reproductive and Child Health in the Laboratory Centre for Disease Control, Health Canada, is leading the development of the Canadian Perinatal Surveillance System (CPSS). This new surveillance system will monitor and analyze the patterns of health determinants and outcomes in all pregnant women and their infants in Canada, in order to improve the effectiveness and efficiency of clinical perinatal care, public health practice and health policy making. The Bureau has investigated and evaluated existing databases to see if they could serve the needs of a national perinatal surveillance system.

In this evaluation, hospital admission/separation records collected by the Canadian Institute for Health Information (CIHI) [formerly Hospital Medical Records Institute (HMRI)] seemed an attractive source of data for perinatal surveillance because, at present, most of the births in Canada take place in hospitals. However, like other routine data sets, CIHI data are not designed and collected specifically for perinatal surveillance purposes. As a result, it would be natural for people to be sceptical about the quality and appropriateness of CIHI data for perinatal surveillance.

A special concern regarding CIHI data for perinatal surveillance purposes is that so far only some Canadian hospitals (about 78%) participate in the CIHI project. If the delivering mothers and their infants from these participating hospitals differ from those of non-participating hospitals, the results and conclusions based on analyses of data captured by CIHI may be biased.

Thus, we decided to undertake a study to assess the quality and appropriateness of CIHI data for perinatal surveillance. We abstracted the relevant information available on all delivering mothers and their newborns from the CIHI database for the period from April 1, 1984, to March 31, 1995, and systematically assessed the appropriateness of use and the limitation of these data for perinatal surveillance.

Subjects

In order to assess the appropriateness of using CIHI data to monitor temporal trends, 11 years of CIHI data (fiscal years 1984/85 to 1994/95) were utilized (data presented here exclude every second year). Using the entire CIHI data set, we abstracted data on newborns by a field of "age unit" with a code of "NB" (indicating live births) or "SB" (indicating still births), and we abstracted data on delivering mothers with appropriate codes for "case mix group." Table 1 lists the case mix group codes used by CIHI to define obstetric delivering over the 11-year period.

Methods

The variables chosen for assessment were those available from CIHI, and they are important indicators or explanatory variables for the CPSS. Demographic and administrative variables, including infant sex, maternal age (derived by birth date and admission date), case mix group, neonatal in-hospital death, maternal and neonatal length of in-hospital stay, were coded by the CIHI manual. Variables derived from diagnoses were coded according to the ninth revision of the International Classification of Diseases (ICD-9),¹ and we re-grouped some of the diagnosis codes to meet perinatal surveillance purposes. Procedures were coded according to the Canadian Classification of Diagnostic, Therapeutic and Surgical Procedures (CCP).²

Two aspects of the study variables were assessed.

• Whether and to what extent the sample from CIHI formed a representative sample for the whole country
• Whether the needed variables were available and valid in CIHI data
  

To accomplish the first part of the assessment, figures obtained from CIHI were compared with figures reported by Statistics Canada (based on birth certificates) for the same years. To assess the consistency of CIHI data over time, this comparison was carried out for the 11-year period.

To accomplish the second part of the assessment, several techniques were applied. First, we performed internal logic checks on certain variables. For example, we looked for obvious errors such as the sex of a delivering mother not recorded as female, the length of in-hospital stay for a still-birth newborn not equalling 0, a case mix group code for the delivering mother indicating cesarean section while the procedure code indicated vaginal birth or a case mix group code indicating a delivering mother had a cesarean section in a previous pregnancy but with no corresponding diagnosis code.

Secondly, we evaluated the range and extreme values for continuous variables to assess whether the ranges were reasonable. Both illogical and out-of-range values would have been possible because of human errors. What was important to know was how frequently these errors occurred in the CIHI data, and then to judge whether the measurements for these variables were reasonably valid.

Thirdly, we compared the frequency of relevant diagnoses with the literature. There will always be variations in disease incidence across populations, so we would not expect rates estimated from CIHI data to be exactly the same as those reported in the literature. However, if the difference was too large, the incidence of the particular disease might be invalid, and caution would have to be applied in the uses and interpretations of the CIHI data in perinatal surveillance.

Finally, we calculated the relative risk of major medical/obstetric complications with certain adverse outcomes. If the measurements of these medical/obstetric complications were valid, they could be used to predict the risk of certain adverse outcomes for which the causal relationships have been well established. We used cesarean delivery as the adverse outcome to assess the validity of measurement on maternal complications, and neonatal in-hospital death and prolonged length of in-hospital stay as the adverse outcomes to assess the validity of measurement on birth weight. Not only were these adverse outcomes relatively "hard" measures and their causal relationships with maternal and neonatal complications established,^3-5 but their validity had been assessed by our logical check and agreement analysis as well.

Results

The variables studied, their definitions and the assessment techniques used for each study variable are listed in Table 2.

About 72% of all deliveries in Canada were recorded by CIHI in fiscal year 1994/95. The CIHI coverage was lower in earlier years, and substantial interprovincial variation in participation was noted (Table 3).

Key statistics obtained from CIHI, namely maternal age, infant sex ratio, still birth ratio, frequency of multifetal pregnancy, cesarean delivery rate and low birth weight rate, were quite comparable with corresponding figures reported by Statistics Canada, indicating the CIHI cases form a reasonable representative sample of the national figures (Table 4). We have based our assessment on the clinical importance and consistency of the differences, rather than statistical significance, since trivial differences may be statistically significant because of the large sample size.

We found that illogical or unlikely outlier values (all irregular codes, such as a blank or a character for a numerical code) did occur in the CIHI data. However, the frequency of these values was extremely low (Table 5).

For most of the adverse pregnancy conditions and outcomes, especially those with less controversial definitions or diagnoses, we found the prevalence to be within a reasonable range of that reported in the literature (Table 6).

Our calculations of relative risk suggested that major maternal complications and indications coded by CIHI were strong predictors of cesarean delivery and that low birth weight recorded by CIHI was a strong predictor of neonatal in-hospital death and prolonged length of in-hospital stay (Tables 7A and 7B).

Discussion

Whether and to what extent a routine data set can be used for disease surveillance depends on the availability of study subjects and of variables pertinent to the purpose of a particular surveillance project, as well as the validity of measurement of the variables recorded in that data set. In the following sections, we explore the potential uses and appropriateness of the uses of CIHI data for perinatal surveillance by discussing our assessment of the performance of CIHI data with respect to the three aspects.

Availability of Study Subjects for Perinatal Surveillance

Many surveillance projects have difficulty in clearly defining their study subjects; however, the situation is better for perinatal surveillance. If we focus our surveillance on births, we can define study subjects without much difficulty. Routine hospital records such as those of CIHI then become an attractive data source for perinatal surveillance because, up to now, most births in Canada occur in hospitals. Although CIHI recorded only about 72% of all deliveries and participation rates varied across provinces and over time, we feel this sample may still be quite representative of all births in Canada, based on our comparison of key statistics from CIHI with Statistics Canada's figures (which are based on all births in Canada).

TABLE 4 Comparison of key statistics obtained from CIHI and those reported by Statistics Canadaa							+
Statistic	Source of data	1984b	1986b	1988b	1990b	1992b	1994b
Maternal age (% <18 yrs)	CIHI	2.1	1.9	1.9	2.2	2.3	2.4
	Statistics Canada	2.1	2.0	1.8	1.9	2.1	2.1
Maternal age (% <20 yrs)	CIHI	6.6	5.7	5.9	6.4	6.4	6.8
	Statistics Canada	6.4	5.9	5.8	5.9	6.1	6.2
Maternal age (% >35 yrs)	CIHI	4.6	5.7	6.3	6.7	5.2	8.5
	Statistics Canada	4.1	4.7	5.5	6.1	6.9	8.1
Sex ratio (% male)	CIHI	51.4	51.1	51.4	51.2	51.1	51.4
	Statistics Canada	51.4	51.2	51.2	51.4	51.3	51.5
Multifetal pregnancy (%)	CIHI	0.8	0.9	0.9	0.9	1.0	1.0
	Statistics Canada	1.0	1.0	1.0	1.0	1.0	1.1
Still birth (%)	CIHI	0.6	0.6	0.6	0.6	0.6	0.7
	Statistics Canada	0.6	0.6	0.6	0.6	0.6	0.6
Low birth weight (%)	CIHI	5.4	5.4	5.3	5.6	5.5	5.8
	Statistics Canada	5.7	5.7	5.7	5.5	5.5	6.0
Cesarean delivery (%)	CIHI	19.9	20.4	20.3	19.3	18.3	17.7
	Statistics Canada	18.9	19.2	19.6	19.1	17.7	NA
a Sources: For cesarean delivery: Millar WJ, Nair C, Wadhera S. Declining cesarean section rates: a continuing trend? Health Reports 1996;8:17-24. For others: Health Statistics Division. Births and deaths, 1982-1994. Ottawa: Statistics Canada. b CIHI data are based on fiscal years, i.e. 1984/85 to 1994/95. NA: Not available

On the other hand, correctly identifying delivering mothers and newborns from CIHI data can pose problems. Because the CIHI system is designed to collect data on all hospital admissions/separations and because there is no single specific field reserved for births, caution should be applied in choosing the appropriate variable(s) to define delivering mothers and newborns.

Diagnostic and procedure codes have been used frequently to select study subjects from CIHI data for epidemiologic studies,^7-9 since seeking treatment (either medical or surgical) for a particular disease is usually the sole reason for hospital admission. However, diagnostic and procedure codes are not appropriate for subject selection in perinatal surveillance. Because pregnancy is normally a physiological rather than pathological process, most of the delivering mothers and their newborns have no disease and need no treatment. As a result, they are not assigned any diagnostic or procedure code and therefore cannot be picked up by the ICD-9 or CCP codes.

TABLE 6 Comparison of prevalence of major adverse pregnancy conditions and outcomes estimated from CIHI (1994/95 data) with that reported in the literature
Adverse condition/outcome	% estimated from CIHI	% reported in literature	Reference
Maternal age <18 years	2.4	2.1	Statistics Canadaa
Maternal age >35 years	8.5	8.1	Statistics Canadaa
Previous cesarean section	10.1	7.2b	Reference 6
Breech presentation	3.9	3.0-4.0	Reference 3
Dystocia	17.5	12.0	Reference 3
Fetal distress	10.6	Openc	Reference 3
Preeclampsia and eclampsia	0.6	0.1-5.0d	Reference 3
Gestational diabetes	2.9	1.0-3.0	Reference 3
Placenta previa	0.4	0.3-0.5	Reference 3
Abruptio placenta	1.2	0.7	Reference 3
Polyhydramnios	0.4	0.1-1.7	Reference 3
Multifetal pregnancy	1.0	1.0	Statistics Canadaa
Fetal death	0.7	0.6	Statistics Canadaa
Birth weight <2500 g	5.8	6.0	Statistics Canadaa
Birth weight  >4000 g	12.7	12.2	Statistics Canadaa
Respiratory distress	1.2	1.2	Reference 3
a Health Statistics Division. Births and deaths, 1994. Ottawa: Statistics Canada. b Figure for 1982, which is expected to be lower than the current one. c Reference 3 described the various difficulties in diagnosing this condition, but did not give a reference percentage. d Percentages obtained from CIHI were restricted to severe preeclampsia and eclampsia (ICD-9: 6425, 6426, 6427), whereas no such restriction was applied to those cited from Reference 3.

It seems that the codes we used to select study subjects, namely age unit codes of "NB" and "SB" for newborns and case mix group codes for delivering mothers, are appropriate here. For example, the selection of a newborn would be incorrect if the admission date of the newborn was prior to his/her birth date, and the selection of a delivering woman would be incorrect if her gender was not recorded as female or her age was too old or too young. Neither case occurred according to our selection criteria.

TABLE 7B Predictability of birth weight on neonatal in-hospital death and length of in-hospital stay (LOS), based on 1994/95 CIHI data
Birth weight	Neonatal in-hospital death (%)a	Mean LOS in daysb (SD)	LOSb <2 days (%)	LOSb >4 days (SD)
500- <1500 g	15.17	18.0 (5.5)	4.7	90.5
1500- <2500 g	0.71	7.8 (6.3)	6.1	55.6
2500-4000 g	0.05	2.6 (1.6)	19.0	8.6
>4000 g	0.05	2.8 (1.6)	14.9	11.0
a Only live births were included. b Only live newborns who were discharged alive from the birth hospitals without transferring to other institutions were included. SD: Standard deviation

Availability of Variables for Perinatal Surveillance

The CPSS Steering Committee has developed a preliminary list of indicators and explanatory variables that should be collected and analyzed for the Canadian Perinatal Surveillance System.¹⁰ The CIHI data contain some of these indicators and explanatory variables, with concentration on administrative variables and diagnoses. (It should be pointed out that the list in Table 2 is not exhaustive.)

Some variables are not explicitly recorded by CIHI, but can be derived by various techniques. For example, by matching delivering mothers with information on the same individuals contained in other CIHI data (CIHI data after discharge of the mothers), rates of maternal re-admission could be calculated and analyzed by time period, region or reason for re-admission. The ability to ascertain re-admission is a unique feature of CIHI data. Once delivering mothers are discharged from hospitals, they are no longer obstetric patients and can be re-admitted to hospitals as any other type of patient, such as medical, surgical or psychiatric patients. Any system with no full coverage of various hospital services or with no complete individual follow-up would miss a large proportion of re-admission cases.

It should be noted that there are large gaps between the CPSS requirements and CIHI data in terms of availability of variables for perinatal surveillance. First, antenatal information, including antenatal care and maternal antenatal exposure information, is missing from CIHI data. Second, no information is collected by CIHI on patients after their discharge. Third, many important operative or anesthesia procedures and medication, such as forceps, induction, epidural anesthesia, etc., are not available in the CIHI data (at least for the copy that Health Canada receives). Fourth, maternal and infant data are recorded separately by CIHI, which makes an analysis of the relationship between maternal characteristics and infant's outcomes difficult.

Some of these problems with CIHI data can be remedied by more intensive data collection effort. For example, length of gestation, anesthesia procedures, forceps usage and inductions are routinely recorded (and parity will soon be recorded) by Med-Echo, a similar hospital discharge database in Quebec. Data linkage can provide some remedies for CIHI data as well. For example, a matching between delivering mothers and their own infants may allow an analysis of the relationship between maternal characteristics and infant's outcomes, although there is no guarantee of perfect matching here. However, some of the problems cannot be fixed without additional data collection and information gathering.

Validity of Variables Recorded by CIHI

If the measurement of a study variable is not valid, the study result and conclusion can be seriously compromised. As a result, our assessment of the quality of CIHI data focuses on the validity of measurement of variables recorded by CIHI.

For some of the "soft" diagnostic variables, large variability is expected. For example, what amount of fluid should be used as the criterion to diagnose "oligohydramnios" or "polyhydramnios," or what criterion should be used to distinguish "severe" from "mild" preeclampsia has been and will continue to be controversial.³ If caused by random errors, the great variability of these "soft" diagnostic variables may lead to non-differential misclassification.

Systematic bias may also be introduced when comparing rates based on these diagnoses for different groups of pregnant women (e.g. across time periods and geographic areas). For example, if the blood glucose screening for pregnant women is more intensive and the diagnostic criteria for gestational diabetes are more lax in recent years than in previous years, or if the screening is more intensive and the diagnostic criteria are more lax in certain provinces, the apparent temporal trends or interprovincial variations in diagnosed gestational diabetes may reflect differences in screening aspects and diagnostic criteria rather than true incidence. As a result, caution should be applied in the interpretation of temporal trends and geographic area variations of rates and distributions for these variables, with particular attention being paid to variability in criteria and intensity of diagnosis.

For most of the demographic and administrative variables and some of the "hard" diagnostic variables (including maternal age, infant's sex, birth weight, length of in-hospital stay, case mix group, cesarean delivery and previous cesarean delivery), it seems that the records in CIHI data are quite valid. As shown in Table 6, illogical and unlikely outlier values for these variables are very few. Logical and within-range values can be invalid as well. For example, it would be "logical" (i.e. the sort of response that would fit the data field) but invalid if a female infant was recorded as male, a length of in-hospital stay of 1 day recorded as 10 days or a birth weight of 500 grams recorded as 5000 grams. However, it is a general tendency that illogical and unreasonable values are less frequently observed in data sets with good quality.

The validity of the variables studied in the CIHI data could also be judged by the similarity of their rates and distributions with those in the literature and by their strong predictability for adverse outcomes. Tables 7A and 7B showed that history of previous cesarean section and breech presentation were strong predictors of cesarean delivery while dystocia and fetal distress were weak predictors of cesarean delivery, and very low birth weight and low birth weight were strong predictors of neonatal mortality and morbidity (as measured by prolonged stay in hospital). These findings are very consistent with the related literature.^3-6

Conclusion

CIHI data contain some of the information pertinent for perinatal surveillance, with concentration on administrative variables and diagnoses. Meticulous analysis of these data may generate results that could be applied to improve intrapartum care, hospital resource utilization, and maternal and infant health. To meet the goal of the Canadian Perinatal Surveillance System to monitor and analyze patterns of health determinants and outcomes in all pregnant women and their infants in Canada, additional data collection mechanisms are needed to cover all recognized pregnancies and to collect antenatal and postpartum information and more detailed information on intrapartum care. Development of valid data linkage mechanisms is also needed.

Acknowledgements

This study was carried out under the auspices of the Canadian Perinatal Surveillance System. Dr Sylvie Marcoux holds a National Research Scholarship from Health Canada.

References

1. Commission on Professional and Hospital Activities. International classification of diseases, 9th revision, clinical modification. Ann Arbor (MI): Commission on Professional and Hospital Activities, 1992.

2. Statistics Canada. Canadian classification of diagnostic, therapeutic, and surgical procedures. Ottawa, 1986.

3. Cunningham FG, MacDonald PC, Leveno KJ, Gant NF, Gilstrap LC III, editors. Williams obstetrics. 19th ed. Norwalk: Appleton & Lange, 1993.

4. Enkin M, Keirse M, Renfrew M, Neilson J, editors. The Cochrane collaboration: pregnancy and childbirth database, 1994, Disk Issue I.

5. Kramer MS. Determinants of low birth weight: methodological assessment and meta-analysis. Bull World Health Organ 1987;65:663-737.

6. Anderson GM, Lomas J. Determinants of the increasing cesarean birth rate: Ontario data 1979 to 1982. N Engl J Med 1984;311:887-92.

7. Wen SW, Naylor CD. Diagnostic accuracy and short-term surgical outcomes in cases of suspected acute appendicitis. Can Med Assoc J 1995;152:1617-26.

8. Wen SW, Hernandez R, Naylor CD. Pitfalls in nonrandomized outcomes studies: the case of incidental appendectomy with open cholecystectomy. JAMA 1995;274:1687-91.

9. Wen SW, Simunovic M, Williams JI, Johnston KW, Naylor CD. Hospital volume, calendar age, and short term outcomes in patients undergoing repair of abdominal aortic aneurysms: the Ontario experience, 1988-1992. J Epidemiol Community Health 1996;50:207-13.

10. Health Canada (Laboratory Centre for Disease Control, Bureau of Reproductive and Child Health). Progress report: Canadian perinatal surveillance system. Ottawa, 1995.  

Author References

Shi Wu Wen, Shiliang Liu and Dawn Fowler, Bureau of Reproductive and Child Health, Laboratory Centre for Disease Control, Health Canada, Tunney's Pasture, Address Locator: 0601E2, Ottawa, Ontario K1A 0L2
Sylvie Marcoux, Epidemiology Unit, Laval University, Quebec City, Quebec

This paper was presented in part at the Annual Clinical Meeting of the Society of Obstetricians and Gynaecologists of Canada in Halifax, Nova Scotia, June 23-25, 1997.