Use counters or structured objects to assist debugging asynchronous loops

This is part of the Semicolon&Sons Code Diary - consisting of lessons learned on the job. You're in the javascript category.

Last Updated: 2021-05-15

A coworker had some code that looked roughly like this:

const dataEndPoints = []

await Promise.all(dataEndPoints.map((url) =>
  httpGet(url).then(json => JSON.parse(json))
))

It failed on JSON.parse, so he logged each dataEndPoint index and its parsed object. There were 120 in total, and he saw the numbers 1 to 120 in the Chrome DevTools log. Yet the code still blew up. What happened?

1 request
2 request
3 request
4 request
5 request
6 request
7 request
8 request
9 request
10 request
11 request
12 request
13 request
14 request
15 request
16 request
17 request
18 request
19 request
20 request
21 request
22 request
23 request
24 request
25 request
26 request
27 request
28 request
29 request
30 request
31 request
32 request
33 request
34 request
35 request
36 request
37 request
38 request
39 request
40 request
41 request
42 request
43 request
44 request
45 request
46 request
47 request
48 request
49 request
50 request
51 request
52 request
53 request
54 request
55 request
56 request
57 request
58 request
59 request
60 request
61 request
62 request
63 request
64 request
65 request
66 request
67 request
68 request
69 request
70 request
71 request
72 request
73 request
74 request
75 request
76 request
77 request
78 request
79 request
80 request
81 request
82 request
83 request
84 request
85 request
86 request
87 request
88 request
89 request
90 request
91 request
92 request
93 request
94 request
95 request
96 request
97 request
98 request
99 request
100 request
101 request
102 request
103 request
104 request
105 request
106 request
107 request
108 request
109 request
110 request
111 request
112 request
113 request
114 request
115 request
116 request
117 request
118 request
119 request
120 request

The issue was that the 42nd one actually did not log - he just thought it did, since he saw 1,2,3,4....120 in the logs (but didn't inspect the sequence for every individual item).

The moral of the story is that in concurrency work, the assumption that code screeches to a halt and crashes at the point in time where things are broken no longer applies... failures in the middle of the sequence can cause an exception, yet life goes on.

Lesson

When debugging asynchronous loops, instead of looking at the output sequence, instead have a dictionary with indexId and success true/false and use that to figure out where it went wrong. Or count the number of failures.