Quality control, data cleaning, imputation

Publication date

2023-11-05

Authors

Liu, Dawei
Oberman, Hanne I.
Munoz Avila, Johanna
Hoogland, Jeroen
Debray, ThomasORCID 0000-0002-1790-2719ISNI 0000000390283878

Editors

Asselbergs, Folkert W.
Denaxas, Spiros
Oberski, Daniel L.
Moore, Jason H.

Advisors

Supervisors

Document Type

Part of book

Collections

Open Access logo

License

taverne

Abstract

This chapter addresses important steps during the quality assurance and control of RWD, with particular emphasis on the identification and handling of missing values. A gentle introduction is provided on common statistical and machine learning methods for imputation. We discuss the main strengths and weaknesses of each method, and compare their performance in a literature review. We motivate why the imputation of RWD may require additional efforts to avoid bias, and highlight recent advances that account for informative missingness and repeated observations. Finally, we introduce alternative methods to address incomplete data without the need for imputation.

Keywords

Conditional modelling imputation, Heckman selection model, Imputation, Informative missingness, Joint modelling imputation, Machine learning imputation, Matrix completion, Missing at random, Missing completely at random, Missing data, Missing indicator, Missing not at random, Nearest neighbor, Neural networks, Pattern submodels, Rubin's rules, Sporadically missing, Support vector machines, Surrogate splits, Systematically missing, Tree-based ensembles, Taverne, General Medicine, General Health Professions, General Nursing, General Biochemistry,Genetics and Molecular Biology, General Agricultural and Biological Sciences, General Computer Science

Citation

Liu, D, Oberman, H I, Muñoz, J, Hoogland, J & Debray, T P A 2023, Quality control, data cleaning, imputation. in F W Asselbergs, S Denaxas, D L Oberski & J H Moore (eds), Clinical Applications of Artificial Intelligence in Real-World Data. 1 edn, Springer, pp. 7-36. https://doi.org/10.1007/978-3-031-36678-9_2