Reliability of interrater occupation coding and potential impact on occupational exposure assessment

Abstract

OBJECTIVES: Coding of job descriptions to occupation codes is a key step in exposure assessment in occupational epidemiology studies. We assessed agreement between occupation codes assigned by coder pairs in a hospital-based case-control study in East Asia and evaluated agreement between job-exposure matrix (JEM) estimates assigned to these codes. METHODS: Each participant-reported job was classified following the 1988 version of the International Standard Classification of Occupations (ISCO-88) independently by 2 research centers. We calculated percent agreement between the ISCO-88 codes from different research center pairings (center 1-Chengdu versus 2-Tianjin; 3-Hong Kong versus 4-Taiwan). Stratified analyses for job coding agreement were conducted for different occupations, industries, and study phases. The different sets of job codes were linked to a population-based JEM and the exposure estimates were compared using weighted Cohen's kappa (κ). RESULTS: Our study included 34,353 reported jobs from 12,590 subjects. Centers 1 and 2 coded 21,774 jobs from Mainland China; centers 3 and 4 coded 12,579 jobs from Hong Kong and Taiwan. Job coding percentage agreement ranged from 51.0% to 77.1% depending on job code granularity and different center pairing. JEM-based exposure estimates comparisons resulted in κs ranging from 0.72 to 0.95 for job codes from centers 1 and 2 (median: 0.86) and 0.62 to 0.95 for job codes from centers 3 and 4 (median: 0.75). Agreement in both measures varied depending on occupation and industry. CONCLUSION: Job coding agreement was similar with agreement reported in other occupational studies. Agreement between JEM estimates was higher than between assigned job codes, as not all job coding disagreements led to different exposure assignment.