Shell commands in loops compete for STDIN in unpredictable ways

This is part of the Semicolon&Sons Code Diary - consisting of lessons learned on the job. You're in the bash category.

Last Updated: 2024-03-28

I had some code to split a large mp3 (all.mp3). It read from a csv file like the following:

Rorate-coeli,00:00,01:46
Universi-qui-te-expectant,01:46,03:46
Ad-te-levavi,03:47,06:01
Ex-Sion,06:02,07:51.7

Here's the code:

# Notice here, BTW, how IFS is set to `,` (comma) for just this loop, and how
# `read` accepts multiple variables, corresponding to the splits on each line
while IFS=, read -r name start_t end_t
do
  output_file="./output/$name.mp3"
  ffmpeg -i all.mp3 -n -acodec copy -ss "00:$start_t" -to "00:$end_t" -f mp3 "$output_file"
done < splits.csv

Aside on how read works:

Read will read a line of input into name name1 name2 ... splitting the line based on the contents of the Internal Field Separator (IFS)

--

When I ran the script, the first few tracks had normal file names, but the later ones had characters loped off the front (e.g. "Ad-te-levavi" might become "e-levavi")

Yet when I simply echoed the variables without ffmpeg, everything was fine

while IFS=, read -r name start_t end_t
do
  echo $name
  echo $start_t
  echo $end_t
done < splits.csv

So ffmpeg was somehow messing with the variables.

It transpired that ffmpeg reads from STDIN — which was "splits.csv" in this case due to the redirection — and its consumption of input was competing with the read -r consumption of input in the outer loop.

Solutions:

  1. The simplest (but least far reaching) solution was to disable reading from STDIN with ffmpeg
$ ffmpeg -nostdin ...
  1. A more general purpose solution involved redirecting the input of ffmpeg to dev/null
while IFS=, read -r name start_t end_t
  # new bit at end
  ffmpeg -i all.mp3 -n -acodec copy -ss "00:$start_t" -to "00:$end_t" -f mp3 "$output_file" < /dev/null
done < splits.csv
  1. Another general solution was to get the read loop to use a file descriptor unlikely to be in use elsewhere:
while IFS=, read -r name start_t end_t <&3; do
 # Here read is reading from FD 3, to which 'splits.csv' is redirected.
done 3<splits.csv

References