Shell commands in loops compete for STDIN in unpredictable ways

This is part of the Semicolon&Sons Code Diary - consisting of lessons learned on the job. You're in the bash category.

Last Updated: 2023-03-28

I had some code to split a large mp3 (all.mp3). It read from a csv file like the following:


Here's the code:

# Notice here, BTW, how IFS is set to `,` (comma) for just this loop, and how
# `read` accepts multiple variables, corresponding to the splits on each line
while IFS=, read -r name start_t end_t
  ffmpeg -i all.mp3 -n -acodec copy -ss "00:$start_t" -to "00:$end_t" -f mp3 "$output_file"
done < splits.csv

Aside on how read works:

Read will read a line of input into name name1 name2 ... splitting the line based on the contents of the Internal Field Separator (IFS)


When I ran the script, the first few tracks had normal file names, but the later ones had characters loped off the front (e.g. "Ad-te-levavi" might become "e-levavi")

Yet when I simply echoed the variables without ffmpeg, everything was fine

while IFS=, read -r name start_t end_t
  echo $name
  echo $start_t
  echo $end_t
done < splits.csv

So ffmpeg was somehow messing with the variables.

It transpired that ffmpeg reads from STDIN — which was "splits.csv" in this case due to the redirection — and its consumption of input was competing with the read -r consumption of input in the outer loop.


  1. The simplest (but least far reaching) solution was to disable reading from STDIN with ffmpeg
$ ffmpeg -nostdin ...
  1. A more general purpose solution involved redirecting the input of ffmpeg to dev/null
while IFS=, read -r name start_t end_t
  # new bit at end
  ffmpeg -i all.mp3 -n -acodec copy -ss "00:$start_t" -to "00:$end_t" -f mp3 "$output_file" < /dev/null
done < splits.csv
  1. Another general solution was to get the read loop to use a file descriptor unlikely to be in use elsewhere:
while IFS=, read -r name start_t end_t <&3; do
 # Here read is reading from FD 3, to which 'splits.csv' is redirected.
done 3<splits.csv