Filling in the Blanks: Multiple Imputation with R’s MICE Package

Introduction

Missing data is a common challenge in statistical analysis and can lead to biased results if not handled properly. Multiple imputation is a powerful technique used to address this issue by creating multiple complete data sets from the incomplete data. Each data set is analyzed separately, and the results are combined to produce estimates that account for the uncertainty due to missing data (Figure 1).

Figure 1

In this article, we will walk through a step-by-step guide on how to perform multiple imputation using the mice (Multivariate Imputation by Chained Equations) package in R (Buuren and Groothuis-Oudshoorn 2011). We will use a simulated data set involving a streaming service survey, which includes variables such as age, satisfaction, and service reliability.

Step-by-Step Guide to Multiple Imputation

Step 1: Install and Load the mice Package

Load the mice package for multiple imputation, the tidyverse for data manipulation and visualization, and the naniar package for visualizing missing data:

if (!require("pacman")) install.packages("pacman", quiet = T)
pacman::p_load(tidyverse, mice, naniar)

Step 2: Explore the Data

This article examines a simulated data set that captures user feedback for a TV streaming service, featuring seven key variables that provide insights into customer experiences:

1. Age: User’s age in years
2. Satisfaction: Overall satisfaction score (0-100)
3. Service Reliability: Score for service dependability (0-100)
4. Value for Money: Score reflecting perceived value (0-100)
5. Program Variety: Score reflecting perceived program variety (0-100)
6. Premium Subscriber: Yes/No - Is the user a premium subscriber?
7. Premium Duration: Months as a premium subscriber (if applicable)

Our primary objective is to identify the factors that most significantly predict customer satisfaction with the streaming service. To achieve this, we will employ linear regression analysis, allowing us to quantify the relationship between the various independent variables and the overall satisfaction score.

data |> 
  flextable::flextable() |> 
  flextable::autofit() |> 
  flextable::set_table_properties(
    opts_html = list(
    scroll = list(
      height = "300px",
      freeze_first_column = TRUE
    )
  )
)

id	age	sat	ser_rel	vfm	pro_var	prm_sub	prm_dur
1	23.62		65.65	12.23	17.72	No
2		57	67.48	59.81	53.72	No
3	36.83	51	59.78	57.71	19.92	No
4	26.25	64	49.77	62.50		No
5	48.69		27.18	62.29	33.99	No
6	36.56	71	45.55		61.73	Yes	52
7		77	27.69	24.77	21.44	Yes	36
8	29.00	94	14.16		66.78	Yes	45
9	27.30	52	7.60	31.96	14.82	No
10	39.87	58	63.83	40.83	52.92	No
11	29.60	66	27.28	49.69	14.40	No
12	23.94		12.04	7.46	23.86	Yes	88
13	52.62	64	84.16	62.21	72.05	Yes	12
14	48.99		45.53		47.85	Yes	28
15	45.72	32	60.84	48.88	46.74	No
16	24.87	96	24.28	42.93	62.49	Yes	104
17	51.58	63	35.02	46.56	76.16	Yes	69
18	52.40	51	69.86	90.55	123.17	No
19	87.09	28	80.75	78.64	86.77	No
20	39.37	50	25.35	31.07	26.38	No
21	47.04	59	75.16	40.24	50.22	No
22	60.91	47	75.40	81.00	75.34	No
23	50.46	45	70.49	53.52	31.55	No
24	26.97	94	61.90	10.23	28.72	Yes	106
25	19.46	69	17.28	17.59	32.81	No
26	51.54	52	60.81	39.50	67.12	No
27	45.39	39	41.94	41.63	78.14	No
28	81.25	20	80.08	51.50	83.75	Yes	3
29	34.50	69	29.34	42.57	59.86	Yes	4
30	42.49	49	46.91	69.10	68.30	No
31	28.41	77	50.94	44.76	42.76	Yes	21
32	30.55	77		44.57	35.52	Yes	36
33	23.03		42.07	43.38		No
34	59.98	72	56.77		52.27	Yes	81
35	39.88	85	34.83	38.70	52.15	Yes	90
36	63.11	82			81.26	Yes	107
37	54.94	76	83.66	63.90	37.48	Yes	88
38	45.58	45	64.15		78.18	No
39	92.99	23	58.33	75.90	104.19	No
40	50.31	37	35.30	37.12		No
41	47.85	47	29.39	66.03	39.22	No
42	49.01	68	41.85	65.74	56.69	Yes	111
43		70	21.23	51.30	42.33	No
44	56.46	27	21.18		72.43	No
45	64.72	41		71.26	58.89	No
46		65		43.29	31.02	No
47	49.20	53		75.62	54.36	Yes	12
48	38.15	43	16.12	36.39	24.06	No
49	34.69	65				Yes	72
50	53.57	32	41.33	31.04	35.29	No
51	39.66		48.95	37.15	50.41	Yes	102
52	55.52	48		47.98	68.96	Yes	23
53		89	28.32			Yes	85
54	29.18	81	52.17	37.54	38.47	Yes	0
55	39.01	81	56.15	21.18	72.36	Yes	111
56		49	49.73		33.45	Yes	23
57	55.68	52	41.67	23.46		No
58	34.65	47		47.36	32.06	No
59	81.54	36	75.74	89.91	113.25	No
60	37.03	63	68.82	30.64	42.59	No
61	48.85	91	50.14			Yes	84
62	44.99		45.52	35.61		No
63		82	44.52	29.52	54.19	Yes	108
64	29.80	67	23.41	21.60	40.95	No
65	65.71	22	19.40	104.93	44.28	No
66	18.38	87	30.54		36.55	Yes	46
67	36.40	71	47.04	28.92		Yes	22
68	32.97	35	10.23	47.40	43.65	No
69	37.13	53	43.89	38.69		No
70	35.61	57	31.37		41.36	No
71	60.78	44	83.93	81.15	28.94	No
72	43.77	90	55.45	62.40	37.46	Yes	82
73	37.72	49	29.68	46.50	44.59	No
74	54.73	63	65.82		72.55	Yes	85
75	59.11		58.58		56.64	Yes	106
76	44.57	52	52.76	26.33	40.38	No
77	34.65	36	37.16	50.83	28.22	No
78	48.88	42	11.18	37.61	70.16	No
79	54.83	50	43.52	58.83		No
80	61.53	70	51.57	64.37	77.89	Yes	97
81	28.71	86	34.33	25.62	21.83	Yes	67
82	35.03	68	44.13		58.91	No
83	49.21	32	38.03	57.97	44.38	No
84		71	45.01	71.49	52.25	No
85			71.52	93.62	44.80	No
86	41.85	75	27.71	43.02	47.26	Yes	20
87	57.69	47	43.90	78.47	27.81	No
88	31.34		43.56	62.93	62.40	No
89	42.73	78	14.11	37.85	47.52	Yes	24
90	22.83		55.81	25.49	42.79	No
91	20.91	65			50.57	Yes	53
92		57	61.41	53.01	66.49	No
93	34.15	39	33.46	53.91	35.07	No
94	27.14	68		26.37	45.89	No
95	61.12	51	67.93	63.35		Yes	47
96	50.55	58	68.42		40.99	No
97	58.27	41	83.62	59.72	57.24	No
98	64.26	67	74.02	88.64	49.57	Yes	91
99	33.92	87	49.69	63.52	40.44	Yes	44
100	32.17	64	79.58	30.73	17.16	No
101	42.06	37	20.23	39.57	26.71	No
102	3.44	100		37.75	58.06	Yes	110
103	64.17	58	53.70	67.32	65.88	Yes	52
104		57	19.74	30.67	10.14	Yes	1
105	64.90	66	63.91	59.79	45.90	Yes	53
106	39.41	75	55.53	52.79	40.19	No
107	56.65			53.10	57.22	No
108	46.66	47	39.02	51.30		No
109	36.89	78	65.08	45.53	36.31	No
110	48.55	51	59.69	44.38	36.94	No
111	47.07	59	39.09	48.81	60.16	Yes	18
112	39.74	80	59.28	54.09	35.04	Yes	83
113			64.40	57.07	53.55	No
114	48.41	75		52.63	49.51	Yes	78
115	68.32	31	73.79	81.28	64.51	No
116	46.98	62	39.38	70.93		Yes	67
117	38.22	87	72.27	58.73	52.00	Yes	92
118	14.43	56		23.15	3.83	No
119	57.09	72	77.41	68.84	66.76	Yes	56
120	50.94	65	69.52	48.70	20.75	Yes	17
121	49.09		54.20		89.16	No
122	45.02	57	97.86	71.05	60.77	No
123	39.14	60		31.25		No
124	49.34	56	58.21	65.46	74.80	Yes	25
125	50.06	37	44.57	55.80	44.58	No
126	31.10	53	51.58		61.66	No
127	40.80		53.52	58.95	72.31	No
128	67.17	64	86.68	68.83	84.94	Yes	62
129	50.77	47	56.45	71.11	81.56	No
130	39.95	51		45.44	52.63	No
131	26.09	61	43.47	64.69	25.67	No
132	42.07		57.83	58.69		No
133	35.64	90	55.16	66.41	35.34	Yes	119
134	16.07	92	61.82	25.31	26.98	Yes	31
135	30.21	66	22.53	37.71		No
136	51.17	57	55.58	74.40	72.09	No
137	40.94	78	64.37	60.56		Yes	65
138	40.49	92		28.50	24.90	Yes	101
139	55.92	59	57.96	37.29	70.58	Yes	10
140	38.72	57	61.02		68.34	No
141	51.89	45	28.81	86.19	31.67	No
142	51.89	88	78.40	75.12	38.88	Yes	114
143	45.64	46	63.80		39.04	No
144	46.75	72	63.35	57.53		Yes	92
145	49.63	61	46.92	37.24	82.79	No
146	28.25	68	8.86	34.92	57.17	Yes	61
147	46.89	91	50.57	62.36	59.96	Yes	106
148	51.98	33		51.07	72.57	No
149	47.02	65		-2.79	46.75	No
150	52.40	59	59.98	62.70	76.66	Yes	7
151	57.74	95	43.42	29.29	68.19	Yes	112
152	55.47	75	88.92	80.64	55.50	Yes	68
153	49.38	66	36.85	35.35	11.70	Yes	75
154	55.43	53		35.94	59.33	No
155	48.53	75	94.65	30.11	49.64	Yes	51
156	39.01	63	75.58	62.92	81.99	No
157	42.12	57	78.09	32.68	43.32	No
158	50.42		65.69			No
159	62.35	47	32.76	54.30	75.11	No
160	30.34		-0.36	24.69	65.18	No
161	29.77	95	66.75	38.82	65.99	Yes	110
162	58.23	88	50.05	87.51		Yes	111
163	33.38			44.22	54.62	No
164	34.11	93	84.90	47.10	47.37	Yes	63
165		84	44.09	35.64	37.55	Yes	80
166	32.47	56	24.33		26.96	No
167	50.23	53	37.81		65.21	No
168	51.25	59	28.32	41.67	48.93	Yes	45
169	39.59	53	41.59	38.57	68.18	No
170	18.47	100	51.96	17.76	53.61	Yes	61
171	55.58	79	57.43		53.32	Yes	108
172	27.92	69	40.47	46.90	30.77	No
173	65.96	59	76.42	81.18	68.41	Yes	57
174	15.32	74		35.30	21.90	Yes	92
175	21.77		42.09	33.14	46.36	No
176	50.11		54.68	51.98	46.82	Yes	85
177	53.12		79.02	39.06	102.69	Yes	84
178	33.07	65	68.35	31.98	26.10	No
179	44.02	48	25.56	47.02	50.27	No
180		59		48.18		No
181	34.14	66	60.01	52.32	38.24	No
182	63.65	36	32.01	56.58	75.15	No
183	39.36	72	40.98	52.12	40.56	Yes	96
184	24.37	61	41.80	53.71	53.77	No
185	45.99	80	40.87	63.80	57.63	Yes	97
186	58.75	61	46.78		49.60	No
187	28.12	61	35.49	37.46	51.72	No
188	30.74	54	38.10	44.47	47.92	No
189	53.18		57.18	74.28	55.58	No
190	68.68	39	40.75		101.17	No
191		65	55.75		31.99	Yes	4
192	55.37	52	63.29	67.93	28.97	No
193	24.44	65	62.90	52.61	37.56	No
194	78.36	45	73.22	72.64	46.70	No
195	62.61	44	63.98			No
196	26.78	50	57.36	24.06	25.94	No
197		63	32.33	54.19	46.09	Yes	35
198	24.58	78	28.05		52.48	Yes	41
199	14.31	70	32.16	32.86	12.02	No
200	45.64	70	53.58	42.36	46.34	No
201	47.73	58	73.15		92.16	No
202	46.38	67	43.96	43.94	42.36	Yes	16
203	38.27	42	35.90		28.15	No
204	21.25	83	26.37	28.17	3.89	Yes	49
205	53.68	54	83.73	56.36	76.85	Yes	24
206	64.25	54	77.36	75.80	43.25	Yes	34
207	48.23		26.16	53.33	102.15	Yes	90
208	74.41	70	83.98	32.20	67.78	Yes	74
209	56.67	59	63.98	49.77	61.16	Yes	35
210	62.90	34	39.27	67.22	37.60	No
211	51.47	46	52.72	72.50	81.83	No
212	31.54	70	31.17	28.62	-7.82	No
213	52.01	25	50.38	46.85		No
214	51.87	47	64.56		51.09	No
215	54.54	63	37.24	29.64	48.05	Yes	3
216	54.26	43	42.71	48.02	48.75	No
217	57.77	65	80.31	36.73	31.51	Yes	97
218		72			41.35	Yes	39
219	40.26	61	60.56	69.69	34.82	No
220	65.82	45	71.32	69.28	61.03	Yes	20
221	51.61	44	47.23	18.40	58.11	No
222	31.29	96	68.05		36.76	Yes	107
223	61.45	52	49.83	39.04		Yes	31
224		52	84.34		66.41	No
225	23.09	76	13.63	67.67	59.78	Yes	43
226	44.37	50	41.36	33.86	82.89	No
227	21.94	75	46.01	47.43	46.54	Yes	16
228	39.73	94	57.75	33.46	74.96	Yes	93
229	59.96	52	62.49	54.93	55.48	No
230	30.64		41.39	20.50	32.62	No
231		39		67.75	55.17	No
232	35.42	41	36.15	45.17	24.83	No
233	42.75		78.94	90.41	46.58	No
234	40.02	65	78.43	57.43	43.80	No
235	48.55	42	51.69	46.51	62.44	No
236	46.08	50	79.18	58.02	33.87	No
237	36.29	61	24.71		16.77	No
238		39	49.19	55.15	45.53	No
239	30.99	73	52.00	46.03	49.52	No
240	49.54	71	57.58	74.35	63.51	Yes	63
241	62.66		67.35	40.48	67.04	Yes	83
242	22.29	52	19.83	43.12	49.53	No
243	15.29	91	36.67	34.90	34.11	Yes	85
244	34.06	64	23.42	45.62	21.46	No
245	61.44	51	60.97		63.91	No
246	13.55	72	39.98	7.97	18.06	No
247	32.56	60	50.82	31.20	23.92	No
248	47.18	83	72.46		51.23	Yes	114
249	28.38	54	38.67	60.85	25.06	No
250	58.83	35	62.93	59.93	67.27	No
251	49.07	56	45.34	84.51	73.01	No
252	39.89	75	44.82	36.68	68.74	Yes	55
253	66.57	50		49.63	58.99	No
254	38.74	71	42.30	61.87		No
255	12.21	81	20.25		27.91	Yes	7
256	53.75	59		63.41	68.44	Yes	12
257	38.06	66	82.65	59.90		No
258	37.15	66	28.61	62.23	60.39	Yes	34
259	49.84		62.38	70.38	66.75	No
260		63	52.93	38.09	52.44	Yes	33
261	40.07	74	64.34	33.94	10.39	Yes	78
262		74	26.59	0.54	76.58	Yes	86
263	42.71			30.13	36.57	Yes	11
264	60.48	54	71.14		76.26	No
265	18.90	70	43.80	65.63	62.46	No
266	47.25	55	14.07	31.57	43.45	No
267	29.45	97	37.04	33.48	47.37	Yes	116
268	38.24	68	54.78	74.46	35.58	Yes	7
269	69.62	75	84.78		71.78	Yes	112
270	54.19	46	60.94	40.39	38.42	No
271	44.33	66	42.58	29.16	47.49	Yes	55
272	55.61	74	44.13	45.16	48.81	Yes	97
273	44.20	48	38.53	39.87	62.34	No
274	36.69	59	26.06	48.20	57.04	No
275	44.58	41	28.61	65.46	74.48	No
276	14.36	58	39.20	13.95	50.76	No
277	40.90	64	43.08	53.76	30.49	No
278	56.72	46	87.47	68.86		No
279	24.47	71	35.98	23.78	32.29	No
280	40.31	84	41.51	21.15	26.72	Yes	71
281	45.57	41	35.69	58.45		No
282	85.55		54.64	69.15		Yes	7
283	55.26	62	60.68		41.41	Yes	58
284	35.95	39	46.22	51.73	24.88	No
285	34.33	59	40.44		75.95	No
286	39.57	49	44.85		57.73	No
287	32.82	56	39.13	3.41	-12.98	Yes	14
288	50.42	74	46.13		88.88	Yes	116
289	86.01	41	95.29		62.08	No
290	38.51	80	43.62	36.67	59.20	Yes	35
291	55.37	71	79.86	34.67	64.83	Yes	17
292	60.98	33	24.89	28.73		No
293		80	60.29	48.91	76.40	Yes	52
294	4.80			20.98	14.68	No
295		59	52.04	60.18	48.20	No
296	40.41	54	68.91	41.40	68.87	No
297	46.64	73	49.71	54.23	52.66	Yes	12
298	29.64	65	56.66	37.50	22.55	No
299	46.51	69	49.98	57.72	35.26	Yes	112
300			77.77	51.80	80.32	No
301	33.94	53	36.12	52.13	68.74	No
302	25.12	55	33.00	29.15	61.56	No
303	42.15	75	65.54	49.33	35.76	Yes	74
304	43.99	40	44.45	30.54	36.23	No
305	49.08	68		64.18	40.37	Yes	69
306	45.55	40	48.98	50.43	52.32	No
307	53.57	56	49.48	43.20	24.66	No
308	29.63	73	33.80	32.90	41.42	Yes	32
309	75.86	16	63.15		63.63	No
310	28.91	79		49.41	81.09	Yes	101
311	49.04	68	75.19	34.81	41.76	Yes	50
312	62.00	20	68.79	58.56		No
313	67.47		72.89	76.36	38.23	No
314	36.24				70.95	Yes	97
315	31.28	77	43.32	49.15	36.31	Yes	11
316	30.62	62	55.50	50.96	76.11	No
317	49.78				76.83	Yes	31
318				50.15	37.27	Yes	85
319	68.81	36		55.45	60.44	No
320		69	26.48	35.57	38.51	Yes	77
321	46.75	51		55.96	63.17	No
322		50	67.63	35.55	29.03	No
323	40.58	50		47.87	73.83	No
324	56.87	47	56.77	27.53	56.22	No
325	28.61	64	18.53	36.12	10.48	No
326	59.73	57	57.98	50.81	72.63	Yes	18
327	38.38	43	22.89	42.98	26.62	No
328	38.87	71		41.02	45.88	No
329	39.73	83	49.72	62.61	84.54	Yes	81
330	53.18	43	62.23	37.17	48.00	No
331	24.43	92	63.12		32.01	Yes	102
332	39.15	84	35.34	68.69	70.79	Yes	97
333	50.04	65	45.47	45.21	94.52	Yes	41
334	47.97	56	54.61	26.61	58.08	Yes	13
335	37.16		21.05	40.10	40.33	No
336	27.23	60	46.47	63.59	56.56	No
337	62.17	38	62.32	53.25	93.60	No
338	42.64	39		75.18	77.15	No
339	74.53		64.23	47.18	56.01	Yes	35
340	40.92		51.50		78.60	Yes	80
341	59.96	46	46.78	52.54	39.10	No
342	33.64	57	52.03	40.38	30.84	Yes	2
343	53.48	56	32.58	69.85	61.48	No
344	49.04	64	58.04	71.51	75.26	Yes	64
345	38.72	48		61.16	8.21	No
346	46.06	50	39.14	51.60	42.44	No
347	42.26	83		38.61	54.10	Yes	114
348	57.20	64		79.30	86.13	Yes	43
349	29.21	56	64.72	42.86	50.55	No
350	16.34	69	28.64	52.70		Yes	43
351	36.29	77		39.17	43.92	Yes	110
352	56.06	47	58.79	64.22	61.71	No
353	33.48	60	50.73		34.88	No
354	54.27	61	55.06	56.63	76.19	Yes	15
355	46.88	51	63.89	38.44	70.17	No
356	26.64	77	42.76	31.88		Yes	35
357	67.31	37	33.32	62.72	49.63	No
358	35.00	79	37.36	40.43	35.00	No
359	58.28	58	43.00	46.78		Yes	30
360	87.18	20		66.20	63.61	No
361	40.34		44.30	80.76	71.61	No
362	56.91	59	87.22	56.79	55.64	No
363	50.88	57	75.71	33.28	62.94	No
364		41	39.38	68.15	47.98	No
365	42.16	39	43.62	61.06	54.25	No
366		64	39.69	70.73	47.60	Yes	6
367	23.30	98	21.25	-2.57	46.48	Yes	96
368	63.32	32	61.15	78.67	68.30	No
369		61	59.17	47.05	66.90	Yes	33
370	62.07	31	11.43		52.04	No
371	14.42	78	32.82	28.88	25.04	Yes	46
372	40.10	63	49.87	53.23	45.31	Yes	1
373	31.84	38	9.03	50.09	32.48	No
374	67.59	63	75.54	57.25	86.64	Yes	56
375	16.21	93	33.97	23.63		Yes	106
376	50.52	92	56.78	62.01	62.69	Yes	114
377	34.27	69	57.62	69.48	45.50	Yes	51
378	65.90	55	68.25	67.33	56.67	No
379	34.22		55.63	37.29	58.17	No
380	12.18	67	22.28	39.49	17.71	No
381	16.50		32.55	11.84	-0.68	Yes	83
382	44.99	55		60.82		Yes	31
383	40.13	53	40.73	38.93	47.28	Yes	16
384	23.37	62	45.58	30.43	17.28	No
385	41.37	55	68.48	29.51		No
386		68	47.67	74.81	48.72	No
387		84		51.58	82.60	Yes	59
388	46.62	58	28.60		52.16	No
389	42.82	60	9.01	55.87	9.91	No
390	45.95		11.78	45.40	52.42	Yes	94
391	39.76	83	50.16		52.00	Yes	59
392	48.78	45		63.88	46.54	No
393	42.00	39	51.12	37.38	37.15	No
394	47.34	48	58.57	76.87	41.30	No
395	52.22		58.25	60.22	40.34	No
396	38.35	42	25.41	33.43	36.43	No
397	46.72	79	27.64	44.74	50.68	Yes	116
398	52.67		54.32	39.16	37.54	No
399		64	65.47	58.11	41.14	Yes	24
400	34.59	97	11.67	40.53	53.54	Yes	98
401	28.11	51	44.10	9.55	38.63	No
402	40.70	69	40.74		71.64	Yes	83
403	34.89	45	51.16	50.95	52.19	No
404	40.44	55	44.56	58.43	46.80	No
405	53.16	72		59.88	58.11	Yes	90
406		41	75.87	52.99	56.51	No
407	50.51	53	30.02	57.85	55.61	No
408	24.49	100	40.68	48.68	40.91	Yes	113
409	34.50	53	63.75	14.01	3.69	No
410	56.70	83		90.66	91.40	Yes	70
411	60.04	62	48.39	65.01	60.62	Yes	57
412	38.79	52	56.45		51.22	No
413	52.24	39	79.46	72.47	80.73	No
414	53.10	64	52.69	60.37	69.21	Yes	80
415		51	59.36	63.56	67.29	No
416	63.34	63	72.31	63.10	38.97	Yes	47
417		38	80.86	82.29	91.11	No
418	46.50	79	49.12	39.20	48.53	Yes	53
419		53	61.20	57.84	51.64	No
420	23.23	46	35.59	19.01	31.14	No
421	39.27		44.59	19.72	47.42	Yes	86
422	55.64	53	72.66	69.35	76.53	No
423		65	41.86	47.60	64.37	No
424	29.29	89	64.05	50.70	78.80	Yes	79
425	24.10	78	26.69	27.15	63.49	Yes	42
426	47.12	75	49.46	52.91	42.38	Yes	80
427	50.92	68		65.09	56.15	Yes	33
428	28.95		45.23	50.23	43.02	Yes	15
429		78	32.61	43.33	67.46	No
430	23.29	58		53.62	66.37	No
431	26.76	75	4.26	14.61		Yes	73
432	41.80	59		59.47	66.07	No
433	48.85		64.33	56.95	50.63	Yes	94
434	66.21	41	73.72	86.69	54.79	No
435	37.41		65.63		27.75	No
436	47.56	66	40.34		51.87	Yes	44
437		49	79.51	79.75	79.84	No
438	51.82	45	66.33	39.84	52.07	No
439	66.48	39	80.93	50.21	65.42	No
440	34.32	62	7.62	19.84	51.53	No
441	33.88	65	42.15	15.20	29.08	No
442	62.73	47	49.60	33.87	34.45	Yes	1
443	22.45	72	45.11	15.23	31.51	Yes	12
444	67.85	60	44.83	61.91	69.52	Yes	1
445	22.44	100	74.56	29.18	15.40	Yes	97
446	43.83	59	23.01	67.25	88.25	No
447	35.80	50	46.99	44.41	48.01	No
448	39.30	69	58.37	56.16	55.40	Yes	76
449	48.14	56	51.46	65.07	44.89	No
450	44.27	78	51.11	22.56	47.72	Yes	78
451	51.84	76	72.20		66.45	Yes	36
452	49.58	37	41.50	59.37	57.19	No
453	49.13	82		36.62	36.00	Yes	39
454	49.39	78	51.19	58.50	57.25	Yes	110
455	35.90	56	59.42	40.00	52.18	No
456	36.94	58		52.04	71.80	No
457	16.12	76		28.91	40.66	Yes	38
458		86	30.06		26.05	Yes	119
459	24.68	74	-11.18	34.74	-18.04	Yes	75
460	48.27	52	70.27	63.74	55.74	No
461	46.66	38	12.63		42.95	No
462	8.83	80	9.97	29.05	25.78	Yes	47
463		62	47.82	61.84	31.33	No
464	54.48	95	41.10	85.71	70.82	Yes	114
465	51.40	45	58.09	89.56	49.71	No
466	45.79	45			81.12	No
467	37.55	61	48.40	13.12	25.09	No
468	44.64	65	32.97	38.09	37.95	Yes	42
469	75.74	40	70.20		63.40	No
470		43	54.65	71.50	65.66	No
471	39.04	87	58.10	39.45	57.43	Yes	92
472	57.59	63	66.77		83.54	No
473	41.77	52	30.72	36.66	46.61	No
474	58.18	58	84.72	77.23	108.12	No
475	36.99	71	64.94	61.95	22.27	Yes	23
476	56.18	64	75.65	55.72	31.57	Yes	2
477		40	45.06	59.82	59.40	No
478	50.05	65		51.83	55.11	Yes	54
479	55.62	46	36.28		37.75	No
480	18.64	97	77.50	23.75	50.00	Yes	73
481	49.32	85	60.81	71.21	26.46	Yes	82
482	64.40	48	51.86	102.51	80.29	No
483	51.25		66.73	59.44	94.58	Yes	23
484	54.03	43	39.71	79.39	98.94	No
485	37.19	51	37.41	55.21	21.96	No
486	58.75		59.87	37.04	55.08	No
487	42.36	39	32.93	47.46	55.47	No
488	67.22	66	51.76	95.38	66.22	Yes	50
489	42.82	82	77.78	47.70	55.57	Yes	85
490	32.40			39.35	58.92	No
491	46.55	84	65.38	57.47	35.67	Yes	89
492	46.87	67	54.21		55.64	Yes	39
493	50.55	42	61.92	57.38	30.30	No
494	28.94	53	47.82	19.49	31.09	No
495	59.36	40	72.44	60.48	38.95	No
496	39.61	56	57.27		26.61	No
497	21.13	52	45.51	48.95	26.13	No
498	61.47		67.94	58.31	73.59	Yes	11
499	25.88	82	39.69		57.06	Yes	23
500	37.15	62	34.90		65.38	Yes	21
501	34.05	80	26.39	31.15	44.67	Yes	96
502	21.04	100	39.06	40.18	40.08	Yes	91
503	42.08	48	36.44	58.06	56.06	No
504	42.29	75	60.10	38.09	46.85	Yes	78
505	50.62	52	49.16	60.14	73.04	No
506	38.45	94	60.05	52.88	52.51	Yes	110
507	62.45	65	74.88	53.61	62.78	Yes	26
508	60.95		44.75		79.80	Yes	59
509	55.66	65	82.50		69.78	No
510	36.23	62	7.45	36.47	65.16	No
511	23.34	51	40.10	54.24	12.81	No
512	50.43		76.98	42.06	49.62	No
513	64.82	25	71.37	59.79	63.18	No
514		38	63.42	52.90	36.10	No
515	18.44	63	10.63	27.87	11.37	No
516	32.68	58	22.02	8.31	19.80	No
517	58.28	50	91.63	74.01		Yes	4
518	54.20		34.03	65.54	59.84	Yes	52
519	28.98			13.92	25.03	Yes	105
520		55	32.75	49.39	74.44	No
521		100	20.96	6.82	52.10	Yes	110
522	46.80	58	92.67		46.39	No
523	43.49	70	46.91	13.91	57.78	Yes	57
524	45.09		44.44	32.36	60.22	Yes	39
525	-10.07	82	24.08	-4.87	2.44	No
526	48.23	41	29.10	31.19		No
527	38.99	58	81.58			No
528	57.16	70	51.60	70.99		Yes	68
529	30.58	55	13.78			No
530	76.69	48	80.06	70.89	75.11	Yes	47
531	68.44	42	66.76	105.46	54.19	Yes	12
532	69.51	39	53.59	71.52	63.81	No
533	46.45	55	55.40	51.84	59.43	No
534	8.12	96	58.57		-3.27	Yes	63
535	45.34	55		48.62	56.68	No
536	43.83	43	59.31	44.67	38.42	No
537	56.61	42	56.32	39.01	48.71	No
538	41.53	72		54.79	70.51	Yes	10
539		71	41.27	9.86	41.06	No
540	52.56	40	64.59	87.01		No
541	59.51	52	31.31		58.98	Yes	76
542	33.37	59	64.33		49.68	No
543	45.50	49	7.50		55.44	No
544	34.93		10.48	31.23	28.40	No
545	36.85	65	42.34	55.80	30.03	No
546	35.52	52	26.09	48.05	8.47	No
547	39.38	42	32.02	16.95	63.41	No
548	45.62		74.64	68.82	39.80	No
549	35.64		35.76	54.68	60.70	Yes	98
550	6.72	97	-0.24	63.92	39.63	Yes	101
551	77.72	59	78.23	48.06	81.10	Yes	53
552	52.24	39	42.45	61.16	22.44	No
553	57.52	63	47.05	59.91	40.45	Yes	1
554	33.18	77	32.33	52.82	26.11	Yes	38
555	28.27	55	17.44	51.01	52.11	No
556	24.28	67	37.07	61.16	50.41	No
557	21.58	91	38.02	-0.81	51.45	Yes	73
558	53.55	46	74.05	75.04	74.94	No
559	19.36	83	21.21		63.70	Yes	30
560		70	79.52		78.03	Yes	37
561	33.42	64	54.25	29.33	44.35	No
562	46.96	54	45.59	42.89	54.19	No
563	25.43	76	38.51	34.30		Yes	51
564	63.44		62.93		26.60	Yes	100
565	86.76	20	70.06	68.77	61.26	No
566	82.61	59	90.87	63.37	73.64	Yes	62
567		50	42.13		15.25	No
568	52.19	41	70.04	59.82	19.17	No
569	37.63	73	60.85	66.98		Yes	70
570	36.12	71	71.04	71.93	31.05	Yes	19
571	47.45	68	55.74	62.28	64.77	Yes	51
572	60.55	33	32.22	50.70	60.99	No
573	53.79		77.70	48.03	67.41	Yes	39
574	77.95	52		49.90	74.29	No
575	45.83	64	48.60	49.45	61.87	No
576	29.82	53	57.15	18.99	30.95	No
577		54	19.99	20.20	25.45	No
578	43.58	53	63.35	43.57	44.32	No
579	57.53	57	33.06	63.49	45.78	Yes	41
580	20.69	100	44.23	12.22	61.03	Yes	120
581	43.69	86	38.18	62.85	45.48	Yes	111
582	47.66		37.06	47.28	84.47	No
583	63.77	48	83.77	61.77		No
584	54.38	48		37.24	81.83	No
585		73	31.21	39.59	55.62	Yes	38
586	46.68	54	62.02	86.09	76.89	Yes	18
587	37.69	51	18.17	43.17	36.38	No
588	39.13	67	68.47	93.41	63.60	No
589	61.27	53	57.68	33.42	56.15	Yes	76
590	51.18	71	58.08	43.04	44.05	Yes	110
591	45.03	42	56.28	47.68	46.41	No
592	20.40	85	65.47	47.27	48.95	Yes	16
593		54	35.31	51.31	37.43	No
594	38.49	69		59.43	50.15	Yes	32
595	58.36	50	59.76	37.88	54.90	No
596	26.73	47	48.42	38.91	67.63	No
597	33.29	88	47.04	66.21	67.48	Yes	65
598	22.60		9.84	34.05	39.42	Yes	29
599	71.44		118.35	61.88	66.97	No
600	24.58	67	13.54	45.51	49.64	No
601	45.24	77	54.39	56.62	51.33	Yes	109
602	38.34		72.13	24.73	37.26	No
603	28.27	58	49.56	22.84	38.30	No
604	27.75	90	37.68	37.97	42.46	Yes	73
605	50.80	36	67.15	50.64		No
606	36.20	47	50.66	18.17	5.71	No
607	80.52	32	69.96	66.79	71.02	No
608	42.77	57	27.08	31.70	45.31	No
609	51.75	70	77.45	48.86	46.51	Yes	50
610	57.10	56	106.40	70.29	67.57	Yes	38
611	73.48	40	59.49	87.90	77.71	No
612	55.31	46	52.58	31.73		No
613	16.26	96	21.43			Yes	72
614	39.65	80	56.63	12.24	70.78	Yes	82
615	42.70	67		27.93	24.99	No
616	35.84	45	37.52	35.83	38.27	No
617	73.14	70			40.58	Yes	101
618	31.21	83	19.82	34.28	53.73	Yes	23
619	23.61	80	29.87	61.37	54.64	Yes	66
620		52	54.29		40.97	No
621	35.66	79	36.27	23.11	43.82	Yes	62
622	32.63	85		23.52		Yes	105
623	38.22	67	38.65	76.53		No
624	46.96	45	34.00	49.82	42.00	No
625	60.16	37	63.44	58.51	63.06	No
626	38.66	37	33.61	29.93	56.43	No
627	63.56	47	52.76	51.12	53.67	No
628	29.17		52.15	40.89	45.10	No
629	41.46	56	42.36	36.29	53.53	No
630		70	60.49		59.83	Yes	25
631	43.08	31		31.54	55.35	No
632	41.06	79	36.27	63.88		Yes	90
633	45.40	84	49.96	68.44	42.54	Yes	118
634	16.91	72	46.94	44.32	18.45	No
635	28.80	51		19.12	27.38	No
636	67.61	50	87.93	55.46	25.74	No
637	34.05		52.51	75.50	70.49	No
638	32.33	75	34.41	43.69	52.97	No
639	59.98	56	70.35	36.96	79.48	No
640	40.63	49	82.43	50.74	45.46	No
641	26.15	62			31.62	No
642	40.94	49	54.25	61.61	66.77	No
643	58.81	55	69.47	92.07	79.03	Yes	59
644	41.05	37	63.65		-7.51	No
645	82.76	54	88.37	42.47	75.58	Yes	54
646	65.26	53	70.29	71.47	63.94	Yes	55
647	73.54	44	118.22	93.64	77.93	No
648	29.06	65	39.91		41.95	No
649	30.25	68	38.06	61.07	47.55	No
650	31.85	56	44.90	34.95	55.11	No
651	27.72				33.32	No
652	56.19	65	24.60	48.67	67.42	Yes	20
653	62.79	76	61.14	68.94	63.83	Yes	105
654	21.08	77	41.12	59.03	37.60	Yes	41
655	36.77	95	36.46	16.98	33.13	Yes	118
656	73.03	22	59.75	46.58	100.07	No
657	49.43	77	65.89	67.27	45.52	Yes	35
658	60.51	82	57.77	52.44	42.78	Yes	101
659	62.08	41	63.03	83.40	57.78	No
660	41.75	53	37.92	20.90	55.78	No
661	50.22	64	92.81	42.10	44.88	Yes	82
662	43.96		50.26	42.61	-8.38	No
663	45.27	65	71.02		14.37	Yes	38
664	32.99	71		28.95	49.52	No
665	29.21			27.83	34.38	No
666	40.36	78	54.67		20.54	Yes	116
667	29.02		36.68	47.08	20.69	Yes	117
668	51.48	80	95.53	16.25	24.98	Yes	90
669	58.18	52		86.01	86.99	Yes	1
670	39.64	53	48.03	72.47		No
671	31.87	65	62.46	22.01	38.50	No
672	23.47	57	17.55	64.84	55.51	No
673	60.70	50	44.21	37.07	64.32	No
674	52.57	55	73.49	62.67	72.53	No
675	32.68	71	38.99	29.67	55.58	No
676	54.74	43	49.37	91.52	48.50	No
677	52.60	40	67.52	63.05	79.41	No
678	19.52	69	66.30	24.99		No
679	61.44			71.06	82.54	No
680	47.25		65.06	47.64	23.20	No
681	42.51	69	63.54	44.79	43.75	No
682	53.46	30	52.58	100.77	58.77	No
683	21.33	57	14.72	50.58	38.05	No
684	69.11	47	53.20		46.01	No
685	27.39	77	38.19		34.39	Yes	81
686	50.33	69	73.42		72.84	Yes	57
687	51.72	38	57.16	55.00	62.38	No
688	42.61	46	57.01	52.70	43.55	No
689	71.52	31	66.38	30.95	51.19	No
690	49.14	44		39.00	55.80	No
691	44.97	58		57.38	62.83	No
692	61.46	26	70.54	72.57		No
693	47.28	48	43.74	29.22		No
694	45.12	78	81.04	48.72	63.86	Yes	11
695	76.66	25	79.94		62.58	No
696	52.47	76	55.42	36.39	78.52	Yes	74
697	52.06	70	66.79	34.75	85.28	Yes	99
698	79.35	34	101.89	80.36	58.68	No
699	35.75	60	78.75		51.96	No
700	54.89	69	29.71	64.25	37.25	Yes	47
701	36.36	77	67.99	42.97	31.04	Yes	41
702	31.10	52	45.83	51.46	41.94	No
703	49.47	54	43.23	78.64	75.55	No
704	49.28	38	72.66	26.74	65.26	No
705	47.16	57	67.89	75.92	57.73	No
706		71	79.54	53.10	64.37	Yes	30
707	61.33	63		46.04	63.28	Yes	85
708	66.06	74	78.31	54.83	60.14	Yes	110
709	34.33	39	36.85	37.26		No
710	50.78	41	50.76	61.24	46.99	No
711		38	37.53	63.93	53.89	No
712	13.97	55	17.10	39.19	24.29	No
713	60.92	73	52.89	67.62	19.59	Yes	107
714	42.49	78	55.56	55.18	68.89	Yes	70
715	36.27	59	48.74	38.65	29.55	No
716	67.33	85		49.40	45.54	Yes	112
717	30.35	72		18.96	33.54	Yes	34
718	47.82	65		57.95		No
719	39.38	71		73.39	70.95	Yes	109
720	47.12	50	71.14	75.31	63.72	Yes	9
721		66	32.35	35.70	71.10	Yes	41
722	23.32	81	39.94	22.91	25.97	Yes	60
723	64.29		61.14	44.94	47.56	Yes	116
724	35.99	92	52.41	34.46	55.92	Yes	66
725	28.53	55	33.98	50.56	58.87	No
726	33.17	62	17.57	41.71	22.42	No
727	37.23	54	33.39	47.42	38.52	No
728	57.81	52	66.68	73.45	56.43	No
729	68.18	49	88.67	59.39	22.51	No
730	61.43	26	44.50		63.37	No
731	51.27	60	14.31	53.41	66.42	Yes	6
732	41.67	48	49.05	57.12	45.77	No
733	37.37	63	33.53	45.74	55.41	Yes	16
734	48.99	60	28.76		83.26	Yes	58
735		78	45.25	64.81	62.33	Yes	88
736	23.92	68	37.79	49.31	38.91	Yes	36
737		57	34.74	65.83		Yes	30
738	50.66	76	69.70	58.46	52.34	Yes	84
739	33.42	55	49.28	11.99	46.26	No
740	54.51	50	41.89	48.29	50.29	No
741	47.06	31	43.81	80.62		No
742	42.62	55	29.04	20.53	10.66	No
743	52.87		55.45	73.70	75.30	Yes	48
744	71.44	52	60.86	79.32	82.20	Yes	78
745	60.12	64	86.64	70.87	100.35	Yes	54
746	37.99	79	26.42	50.86	41.58	Yes	97
747	42.92	76		51.12	72.52	Yes	66
748	39.71	69	45.05	68.05		Yes	72
749	29.30	66	55.06	43.43	48.25	Yes	31
750		62	44.61	66.66	59.09	No
751	69.20	50	58.32	67.27	89.31	Yes	48
752	64.47	35	64.04	71.10	101.72	No
753	45.56	52	55.64	38.86	72.50	No
754	39.11	96	65.89			Yes	115
755	76.37	27	67.19	94.41	83.34	No
756	50.51	71	46.06	43.06	40.34	Yes	60
757	18.11	75	41.09	49.87	56.47	No
758		75	18.08	61.93	16.20	Yes	24
759	51.20	71	60.59	22.46	53.58	Yes	73
760	55.42	45	34.74	58.94	35.52	No
761	43.71	70		33.48	23.42	Yes	40
762	42.39	56	40.95	40.70	69.88	Yes	12
763	51.16	80	58.65	40.82	49.80	Yes	101
764		44		34.97	45.32	No
765	42.05	68	68.33	31.12	14.69	No
766		38	42.98	78.68	43.59	No
767	63.88	53	53.76	62.22	44.88	No
768	58.29	49	74.22	50.34		No
769	51.12	68	57.38	60.95	76.67	Yes	22
770	50.18	47	50.94	38.16	39.11	No
771	17.99	73	24.68		27.12	Yes	59
772	51.61	76	48.34	68.90	77.23	Yes	95
773	21.95	60	27.89	80.51	66.87	No
774	58.38	77	71.03	55.98	67.18	Yes	12
775			30.77	49.24	25.11	No
776	33.32	60	83.50	26.62	67.40	No
777	64.42	36	24.59		19.45	No
778	30.83	75	40.92	34.71	65.77	No
779	45.15	61	79.70	70.55	56.19	Yes	53
780		52	48.71	40.41	67.84	No
781	49.90	60	59.21	65.41	33.42	Yes	6
782	33.20	87	60.92	35.19	27.94	Yes	87
783	65.07	55	96.42	50.27	54.89	No
784	49.91	74	69.93		58.76	Yes	99
785	41.02	53	25.47		36.63	Yes	48
786	51.12	43	42.53	60.58		No
787	68.78	46	46.13	53.34	65.36	Yes	28
788	45.81	57	41.34	63.79	36.89	Yes	67
789	46.26	50	37.51	74.25	52.20	No
790	23.85	61	32.09	15.15	19.69	No
791	47.33	75	64.90		50.60	Yes	111
792	34.21	74	43.79		60.64	No
793	22.78	97	25.24		28.74	Yes	77
794	51.51	40	30.45	65.53		No
795	44.90	61	59.94	63.94	23.66	No
796	42.92	71	69.56	60.80	77.99	Yes	82
797	60.31	45	57.77	109.41	90.33	No
798		63	64.75	68.95	68.71	No
799	53.57	72	45.08	38.89	52.23	Yes	49
800	57.18	64	51.93	72.38	69.47	Yes	66
801	34.73	66	23.23	57.28	38.67	Yes	99
802	30.78	93	34.84	41.96	44.48	Yes	101
803	34.10		39.13		38.61	No
804	39.84	100	45.11	33.12	85.14	Yes	113
805	62.10	50	35.28	112.69	58.01	No
806	42.09	56	53.84	32.89	81.17	No
807	51.91		69.14		56.18	Yes	16
808	34.46	44	57.93	39.16	26.37	No
809	31.96	75	60.59	12.67	18.70	No
810	47.80		43.88	38.80	37.74	No
811	63.72	51	48.60	47.71	97.06	No
812	24.39	97	33.97	-2.75	45.63	Yes	114
813	23.91		54.26	21.11	37.91	No
814	48.41	46	71.85	35.50	29.28	No
815	71.94	37	91.73	75.78		No
816	47.13	64	86.13	65.60		No
817			54.30	56.20	35.23	No
818	42.01	46	18.93	48.56	41.89	No
819	50.68		56.23	58.53	92.07	No
820	32.79	72	24.35	-12.33	-22.81	Yes	114
821	45.58	67	104.32		93.99	Yes	22
822	52.16	91	42.04	42.48	70.80	Yes	111
823	41.96	81	45.02	55.21	64.50	Yes	89
824	52.76	55		60.09		No
825	38.16	55	59.50	63.38	49.06	No
826		63	36.85	39.74	42.40	No
827	47.75	54	50.62	81.87	60.25	No
828	74.20	47	94.67	80.26	63.71	No
829	47.15	68	47.66	60.12	34.69	Yes	47
830	51.47	39	58.64	48.01	38.31	No
831	47.51	38	43.69	48.21	55.55	No
832	35.50	61	43.42	40.02	37.82	No
833	44.19	48	65.02	54.04	32.66	No
834	52.75	38	43.46	40.03		No
835	76.38	42	88.82	86.64	57.26	No
836	55.07	48	40.11	75.13	84.20	No
837	53.96	51	43.40	37.06	43.07	Yes	27
838	52.16	33	50.93	71.09	40.11	No
839	31.38	43	27.98	58.84	21.99	No
840	45.85	63	58.96	72.12	70.59	No
841	40.21	57	62.57	31.85	21.76	No
842		42	63.90	69.18	42.73	No
843	51.03	54	65.25	57.87	80.66	No
844	39.63	55	70.83	51.78	50.33	No
845	30.60	75	45.38	56.26		Yes	91
846	50.47	57	71.50	62.85	76.34	No
847	64.21	50	77.98	44.56	53.77	No
848	62.69	31	61.32	73.60	66.42	No
849	22.73	72	27.22	54.69	29.83	No
850	20.34	63	36.69	33.52	59.43	No
851	29.40	54	57.43	97.94	60.45	No
852	28.99	86	48.46	26.91	57.13	No
853	51.73	60	56.82	56.44	44.59	Yes	21
854		51	80.93	40.97	86.44	Yes	3
855	43.50	82	68.67	32.85	28.37	Yes	96
856	65.12	72	69.39	47.31	35.22	Yes	82
857	12.48		26.26		50.71	No
858	46.98	86	81.22	25.11	53.97	Yes	103
859	21.11	61	38.20	53.19	13.54	No
860	58.52	38	48.93	61.51	54.76	No
861	43.14	58	73.09	30.75		No
862	43.31	76	49.89	39.04	42.28	Yes	100
863	43.91	83	70.01		34.81	Yes	67
864	43.72	80	45.93	50.26	43.18	Yes	112
865	46.55	31	38.94	66.93	41.95	No
866	34.29	77	26.31	35.72	98.41	Yes	25
867	40.84	49	43.27	34.63	2.83	No
868	46.69	47	20.53	28.07	49.62	No
869	50.56	62	71.19		41.46	No
870	26.00	86	32.79	50.42	6.62	Yes	94
871	25.66	85	59.86		50.48	Yes	50
872	54.21	50	76.29	27.52	73.73	No
873	20.68	100	47.18	49.06	62.85	Yes	107
874	43.55	48	46.26	73.94		No
875	28.81	73	49.55	60.71		Yes	50
876	48.94	60	80.76	67.52		Yes	21
877	51.70	30	55.17	78.54		No
878		40	27.09	18.16	40.67	No
879	46.41	60	43.88	-0.34	53.95	No
880	51.15	68	74.47	74.91	43.07	Yes	49
881	53.33	49	74.35	49.62	67.63	No
882	21.64	83			12.50	Yes	53
883	17.67	100		5.11	29.09	Yes	88
884		68	65.71	58.07	62.90	Yes	53
885	65.36	59	57.17		69.24	Yes	51
886		63		-13.78	1.78	No
887	54.16	52	16.11	31.45		No
888	41.51	61	59.64	26.19	57.13	No
889	48.84	39	52.10	47.32	43.01	No
890	22.09	84	32.44	21.71	38.26	Yes	54
891	31.73	66		72.54	48.57	Yes	7
892	31.41	52	24.57	23.84	33.27	No
893	51.63	78	77.48	29.61	50.65	Yes	81
894	52.77			34.33		No
895	54.20	33		75.17	62.10	No
896	25.32	54	40.10	57.88	56.89	No
897	32.07	72	25.51	40.98	28.24	Yes	38
898	31.80	79	55.38		61.39	Yes	64
899	17.34	92	16.20	45.91	39.11	Yes	60
900	8.67	92	29.49			Yes	76
901	28.84	80		43.08	29.22	Yes	81
902	35.48	70		35.21	58.59	No
903	53.47	54	59.65	35.65	43.04	No
904	53.21		75.10	56.61	35.72	No
905	8.66	83	3.48	25.62	31.12	Yes	104
906	71.49	43	101.70	98.65	86.64	No
907	31.35	94	47.25	52.95		Yes	31
908	45.92	46			68.29	No
909	49.11	69	36.09	60.25	37.45	Yes	117
910	58.31	54	75.98	70.68	75.50	Yes	31
911	50.98	53	45.43	59.35	28.15	Yes	41
912		50	18.42	55.66	69.16	No
913	32.36	64	43.78	40.36	8.78	No
914	48.33	61	41.74	57.11	19.83	Yes	10
915	22.85	69	22.98	35.49	47.70	No
916	55.31	54	47.47	79.95	63.51	No
917	49.77	48	40.50	76.97	33.78	No
918	36.40		26.21	73.43	44.41	No
919	42.30	65	47.74	32.82	56.39	Yes	55
920	57.21	52	79.69	47.35	51.20	No
921	37.76	84	28.79	21.34	25.90	Yes	106
922	60.19	58	63.33		63.29	Yes	53
923	57.99	42	52.66	83.25	87.60	No
924	43.52	49	20.99		18.20	No
925	49.77	64	23.73	42.31	42.74	No
926	27.72		48.82	47.54	16.99	No
927	67.48	30	46.67	77.29	56.84	No
928	34.72		65.21	70.93	35.24	No
929	51.11	63			77.66	Yes	51
930	35.68	84	43.27	9.27	59.35	Yes	68
931	23.18	52	35.35	50.46	61.12	No
932	35.37		23.01	20.80	63.13	Yes	3
933	17.00	86	46.11	48.99	43.90	Yes	41
934		83		68.20	56.42	Yes	71
935	31.45	49	24.99	53.11	70.33	No
936	75.58	44	81.46	86.70	59.26	No
937	26.14	82	18.64	72.79	43.78	Yes	22
938	44.25	83	27.00	67.74	2.71	Yes	101
939	46.83	87	65.25	89.10	40.33	Yes	86
940	65.35	75	70.77	92.66	82.99	Yes	112
941	52.96		53.74	83.14	85.87	No
942		46		70.88	63.22	No
943	17.00	66	34.95		14.23	No
944	49.98	51	34.83	67.21	41.92	Yes	0
945	66.38	39	45.32	84.36	67.07	No
946	32.90	82		40.80		Yes	33
947	57.14	76		86.11	93.03	Yes	69
948	53.22	40	45.00	64.21	46.09	No
949	23.20	88	57.28	43.41	8.04	Yes	108
950	38.64	86	39.14	67.82	64.18	Yes	91
951	22.63	75	39.16	48.73		Yes	61
952	13.85		21.80	35.21	20.60	No
953	73.19	57	62.85	66.54	59.80	Yes	59
954	62.64		54.93		83.91	Yes	38
955	58.62	53	73.07	47.72	67.34	Yes	76
956		66	20.28		57.23	No
957	44.05	72	37.58	36.28		Yes	79
958			36.43	39.26	28.57	No
959	27.72	58	47.41	17.05	30.24	Yes	25
960	41.37	54	28.76	48.03	17.48	No
961	24.74	98	46.57	38.45	30.69	Yes	117
962	42.78	40	53.25	31.23	58.20	No
963	46.74	67	55.41	78.52	60.70	Yes	27
964	48.87	39		79.15	45.70	No
965	38.59	62		44.88	44.90	Yes	27
966	23.46	66	42.90	14.13	20.59	No
967	33.10	75	26.92	1.69	39.08	Yes	57
968	72.72	64	79.11	85.59	65.70	Yes	93
969	63.50	38	92.31	50.44	38.90	No
970	35.53	38	33.74	10.19	37.36	No
971	69.18	41	66.13	63.45	57.58	No
972		55	79.29	66.86	73.53	Yes	109
973	44.40	60	61.63	23.69	34.00	No
974	21.33	64	40.42	3.96	32.14	No
975	20.48	63	28.20	8.70	28.79	No
976		70	32.46	45.35	44.45	Yes	70
977		56	45.98	51.59	23.35	Yes	32
978	34.07	60	45.27	42.31	40.57	No
979	45.09	77	23.60	54.10	70.31	Yes	82
980	96.50	22	70.82	93.91	121.89	No
981		89	18.07	46.72	61.29	Yes	73
982	45.51	82	76.71	63.99	30.63	Yes	65
983	36.22	58	43.59		31.03	No
984	52.21		24.69	87.20	62.42	No
985	50.53	84	83.32	79.97	69.51	Yes	95
986			35.68	44.50	20.44	Yes	44
987	31.24	74	47.04		70.31	No
988	1.80	73		4.45	0.42	No
989	44.26	51	15.66	21.44	62.22	No
990	32.43	54	47.58	47.36	9.19	No
991	32.53	54	31.85	51.49	34.32	Yes	10
992	71.86	33	50.15	46.11	63.53	No
993	43.27	82	65.14		47.14	Yes	116
994	33.07	40	42.48	8.33	51.83	No
995	28.13	81	30.73	44.68	30.40	Yes	33
996	28.25	84	36.84	70.62		Yes	81
997	61.45	54	65.66	62.73		No
998	18.28		11.45		29.48	No
999		52	70.91	68.09	65.06	No
1,000		42	59.62	65.34	47.25	No

Step 3: Visualize Missing Data

To better understand the pattern of missing data, we can use the vis_miss() function from the naniar package:

# Visualize missing data
naniar::vis_miss(data %>% 
                   arrange(prm_sub) %>% 
                   select(-id), sort_miss = T) +
  ggplot2::theme_minimal() +
  ggplot2::theme(
    axis.text.x.top =
      ggplot2::element_text(
        angle = 45,
        hjust = 0.1,
        vjust = 0.1
      )
  )

With approximately 15% of data missing, multiple imputation is warranted (Jakobsen et al. 2017). However the data exhibits two distinct patterns of missingness:

1. Structured missingness: The premium_duration variable contains values only for premium subscribers, clearly relating to subscription status.

2. Ambiguous missingness: Other variables like satisfaction and service_reliability show less clear patterns, potentially falling into one of three categories (Bhaskaran and Smeeth 2014):

   • MNAR (Missing Not at Random): Missingness is related to the unobserved values themselves.

   • MAR (Missing At Random): Missingness is related to other observed variables but not to the missing values themselves.

   • MCAR (Missing Completely At Random): Missingness is unrelated to any variables.

To determine the type of missingness for these variables, we can employ the mcar_test() function from the naniar package, which helps distinguish MCAR from MAR/MNAR:

naniar::mcar_test(data %>% 
                    select(-id, -prm_dur)) %>% 
  flextable::flextable() %>% 
  flextable::autofit()

statistic	df	p.value	missing.patterns
70.48884	89	0.9259705	24

The results of the mcar_test() (p > 0.05) fail to reject the null hypothesis that the data are MCAR. This finding is valuable as it supports the use of multiple imputation, which is valid under both MCAR and the less stringent MAR assumptions (Sterne et al. 2009). While MCAR is not a necessary condition for multiple imputation, its presence strengthens our confidence in the appropriateness of this technique for our data¹.

Step 4: Set Up the MICE Imputation

The mice package uses a predictor matrix to identify variables for imputing missing values. In our data set, premium_duration has structured missingness, with values only for premium subscribers. This characteristic poses significant challenges for imputation. Most critically, the algorithm might incorrectly fill missing values for non-subscribers using premium subscriber data, leading to biased results. Given these risks, we will exclude premium_duration from the imputation process, ensuring our model relies solely on universally applicable variables.

To implement this, we’ll use the quickpred() function² to create a predictor matrix that omits premium_duration, thereby enhancing the accuracy and relevance of our imputation:

# Create a predictor matrix excluding premium duration
pred_matrix <- mice::quickpred(data, mincor = 0.1, minpuc = 0, exclude = "prm_dur")

# View the predictor matrix
pred_matrix

        id age sat ser_rel vfm pro_var prm_sub prm_dur
id       0   0   0       0   0       0       0       0
age      0   0   1       1   1       1       0       0
sat      0   1   0       1   1       1       1       0
ser_rel  0   1   1       0   1       1       0       0
vfm      0   1   1       1   0       1       0       0
pro_var  0   1   1       1   1       0       0       0
prm_sub  0   0   0       0   0       0       0       0
prm_dur  0   0   1       0   1       0       1       0

Step 5: Specify Imputation Methods

Next, we need to specify the imputation methods for each variable. mice allows you to set different methods for different variables. For example, we can use predictive mean matching (pmm) for continuous variables and logistic regression for binary variables. By explicitly setting the method for premium_duration to an empty string, we ensure that this variable is excluded from the imputation process:

# Specify the imputation methods
method_list <- make.method(data)
method_list["prm_dur"] <- "" # Skip imputation for premium_duration

# View the method list
print(method_list)

     id     age     sat ser_rel     vfm pro_var prm_sub prm_dur 
     ""   "pmm"   "pmm"   "pmm"   "pmm"   "pmm"      ""      "" 

Step 6: Perform Multiple Imputation

Now we are ready to perform the multiple imputation. We can specify the number of imputed data sets we want to create (e.g., 5³):

# Perform multiple imputation
imputed_data <- mice(data,
  m = 5, method = method_list, predictorMatrix = pred_matrix, seed = 42,
  printFlag = F
)

# print multiple imputation summary
summary(imputed_data)

Class: mids
Number of multiple imputations:  5 
Imputation methods:
     id     age     sat ser_rel     vfm pro_var prm_sub prm_dur 
     ""   "pmm"   "pmm"   "pmm"   "pmm"   "pmm"      ""      "" 
PredictorMatrix:
        id age sat ser_rel vfm pro_var prm_sub prm_dur
id       0   0   0       0   0       0       0       0
age      0   0   1       1   1       1       0       0
sat      0   1   0       1   1       1       0       0
ser_rel  0   1   1       0   1       1       0       0
vfm      0   1   1       1   0       1       0       0
pro_var  0   1   1       1   1       0       0       0
Number of logged events:  1 
  it im dep     meth     out
1  0  0     constant prm_sub

Step 7: Check Imputed Values

It’s important to check that the imputed values are consistent with the original data distribution. The stripplot() function in the mice package provides a useful visualization for this purpose.

# Select numeric variables excluding 'id' in one step
vars <- data %>%
  select(where(is.numeric), -id) %>%
  names()

# Create stripplots using map for efficiency
stripplots <- purrr::map(vars, ~ {
  mice::stripplot(
    imputed_data,
    as.formula(paste(.x, "~ .imp")),
    pch = 19,
    xlab = "Imputation number",
    cex = 2,
    main = .x,
    cex.axis = 10,
    cex.lab = 3,     
    cex.main = 12    
  )
})

This visual assessment enhances confidence that the imputed data set is reliable for further analysis, as it demonstrates consistency across imputations, shows no implausible values, and maintains the overall structure and relationships present in the original data.

Step 8: Analyze the Imputed Data sets

Now that the imputation is complete, we can analyze the imputed data sets. First we leverage the with() function to apply a linear model to each of our imputed data sets. Then, mice provides a convenient function called pool() to pool the results of each model.

Model 1: Linear regression excluding premium_duration:

# Fit the first model
model <- with(imputed_data, lm(sat ~ age + ser_rel + vfm + prm_sub + pro_var))

# Pool the results
pooled_results <- pool(model)

# Summary of the pooled results
broom::tidy(pooled_results, conf.int = TRUE) %>% 
  filter(term != "(Intercept)") %>% 
  select(term, estimate, std.error, statistic, dfcom, conf.low, conf.high,p.value) %>% 
  mutate(p.value = format.pval(p.value, digits = 2)) %>%
  mutate(across(is.numeric, round,2))

        term estimate std.error statistic dfcom conf.low conf.high p.value
1        age    -0.62      0.03    -20.43   994    -0.68     -0.56 < 2e-16
2    ser_rel     0.14      0.03      5.09   994     0.08      0.19 5.3e-05
3        vfm    -0.03      0.02     -1.44   994    -0.07      0.01   0.151
4 prm_subYes    18.77      0.71     26.28   994    17.36     20.17 < 2e-16
5    pro_var     0.04      0.02      1.84   994     0.00      0.08   0.067

Model 1 examines all subscribers. It finds that age has a significant negative effect of -0.62 (p < 2e-16), indicating that satisfaction decreases as age increases. Service reliability (ser_rel) exhibits a significant positive effect of 0.14 (p = 4.1e-11), suggesting that greater reliability is linked to higher satisfaction. Furthermore, premium subscription (prm_subYes) demonstrates a positive effect of 18.66 (p < 2e-16), highlighting its strong impact on satisfaction.

Model 2: Linear regression including premium_duration variable:

# Fit the second model
model2 <- with(imputed_data, lm(sat ~ age + ser_rel + vfm + prm_dur + pro_var))

# Pool the results
pooled_results2 <- pool(model2)

# Summary of the pooled results
broom::tidy(pooled_results2, conf.int = TRUE) %>% 
    filter(term != "(Intercept)") %>% 
  select(term, estimate, std.error, statistic, dfcom, conf.low, conf.high, p.value) %>% 
  mutate(p.value = format.pval(p.value, digits = 2)) %>%
  mutate(across(is.numeric, round,2))

     term estimate std.error statistic dfcom conf.low conf.high p.value
1     age    -0.60      0.04    -13.54   428    -0.69     -0.51 < 2e-16
2 ser_rel     0.12      0.04      3.36   428     0.05      0.19  0.0026
3     vfm    -0.02      0.04     -0.59   428    -0.10      0.05  0.5625
4 prm_dur     0.18      0.02     11.27   428     0.15      0.22 2.3e-12
5 pro_var     0.03      0.03      1.03   428    -0.03      0.09  0.3067

Model 2 focuses exclusively on premium subscribers. In this model, age and service reliability remain significant factors influencing satisfaction. Furthermore, premium subscription duration (prm_dur) demonstrates a significant positive effect of 0.19 (p < 2e-16), suggesting that longer subscription periods are linked to higher levels of satisfaction.

Final thoughts

Multiple imputation using the mice package in R offers a robust solution for handling missing data. By customizing predictor matrices and imputation methods, we can address complex data structures while avoiding bias from irrelevant variables. This approach accounts for uncertainty in missing data, leading to more reliable statistical inferences and trustworthy conclusions. Happy analyzing!

References

Azur, Melissa J., Elizabeth A. Stuart, Constantine Frangakis, and Philip J. Leaf. 2011. “Multiple Imputation by Chained Equations: What Is It and How Does It Work?” International Journal of Methods in Psychiatric Research 20 (1): 40–49. https://doi.org/10.1002/mpr.329.

Bhaskaran, Krishnan, and Liam Smeeth. 2014. “What Is the Difference Between Missing Completely at Random and Missing at Random?” International Journal of Epidemiology 43 (4): 1336–39. https://doi.org/10.1093/ije/dyu080.

Buuren, Stef van, and Karin Groothuis-Oudshoorn. 2011. “Mice: Multivariate Imputation by Chained Equations in r” 45: 1–67. https://doi.org/10.18637/jss.v045.i03.

Jakobsen, Janus Christian, Christian Gluud, Jørn Wetterslev, and Per Winkel. 2017. “When and How Should Multiple Imputation Be Used for Handling Missing Data in Randomised Clinical Trials a Practical Guide with Flowcharts.” BMC Medical Research Methodology 17 (1). https://doi.org/10.1186/s12874-017-0442-1.

Sterne, J. A C, I. R White, J. B Carlin, M. Spratt, P. Royston, M. G Kenward, A. M Wood, and J. R Carpenter. 2009. “Multiple Imputation for Missing Data in Epidemiological and Clinical Research: Potential and Pitfalls.” BMJ 338 (jun29 1): b2393–93. https://doi.org/10.1136/bmj.b2393.

Footnotes

1. Multiple imputation relies on several key assumptions. The primary assumption is that data are Missing At Random (MAR), meaning that the probability of missingness depends only on observed data, not on unobserved data. A stronger assumption is that data are Missing Completely At Random (MCAR), where missingness is unrelated to both observed and unobserved data. Both MAR and MCAR are valid scenarios for multiple imputation, with MCAR being a special case of MAR. Other important assumptions include proper specification of the imputation model, congeniality between the imputation and analysis models, and a sufficiently large sample size. While multiple imputation is robust to some violations of these assumptions, it’s crucial to carefully consider them when applying the method. For a comprehensive discussion, see (Sterne et al. 2009).

2. The quickpred() is a helper function that calculates correlations between variables and uses these correlations to determine which variables are most relevant as predictors for each incomplete variable.

3. Using 5 imputed data sets was done for example purposes. Typically a higher number of imputations are recommended (Azur et al. 2011).