Concurrent programming in Bash with timeout and concurrency control

Bash is a popular Unix shell and command language that has been around for decades. It is widely used for scripting, automation, and system administration tasks. One of the lesser-known features of Bash is its ability to perform concurrent programming.

Recall of background running

#!/usr/bin/env bash

PIDARR=()
date
for c in {1..10};do
{
    echo $c
    sleep $c
} &
    PIDARR+=("$!")
done
wait ${PIDARR[@]}
date

Make an improvement

Add the following parameter, use fd to control concurrency and timeout to control max execution time.

• MAX_CONCURRENCY
• MAX_EXEC_TIME

Modern concurrent Bash script

#!/usr/bin/env bash

MAX_CONCURRENCY=10 # max running slots
MAX_EXEC_TIME=3 # Send SIGTERM after timeout

mkfifo testfifo
exec 10<>testfifo && rm -f testfifo
for _ in $(seq 1 ${MAX_CONCURRENCY}); do { echo >&10; } done
for T in {001..100}; do
    read -r -u10
    {
        ### do something

        ### or timeout something
        timeout ${MAX_EXEC_TIME} sleep $(( ( RANDOM % 10 )  + 1 ))

        echo "finish: $T with $?"
        echo >&10
    } &
done
wait
exec 10>&-
exec 10<&-

Buy Me a Coffee

Back to Table of Contents

Disclaimer

1. License under CC BY-NC 4.0
2. Copyright issue feedback me#imzye.com, replace # with @
3. Not all the commands and scripts are tested in production environment, use at your own risk
4. No personal information is collected.

Feedback

Send