Skip to content

Concurrency programming in bash script with timeout control

homepage-banner

Review of Concurrent Execution in Bash

#!/usr/bin/env bash

PIDARR=()
date
for c in {1..10};do
{
    echo $c
    sleep $c
} &
    PIDARR+=("$!")
done
wait ${PIDARR[@]}
date

Objective

To execute a task in a controllable, multi-process, concurrent manner, and stop the script after the maximum execution time.

Idea

  1. Use fifo pipeline files and pre-fill n values (equal to the number of concurrent processes).
  2. Record the script’s own PID and start the timer process (terminate the timer and kill the script’s own PID after timing out).
  3. Concurrent execution.
  4. After normal completion, kill the timer subprocess and exit the main process.
  5. Recycle fd.

Control Parameters

  • MAX_CONCURRENCY Number of processes
  • MAX_EXEC_TIME Maximum execution time

Concurrency without time control

#!/usr/bin/env bash

trap "exec 10>&-;exec 10<&-;exit 0" 2
mkfifo testfifo
exec 10<>testfifo
rm -f testfifo
MAX_CONCURRENCY=10
for x in $(seq 1 ${MAX_CONCURRENCY}); do
    echo >&10
done
for x in {001..100}; do
    read -u10
    {
        ### do something
        echo "success:$x"
        sleep 2
        echo >&10
    } &
done
wait
exec 10>&-
exec 10<&-
exit 0

Concurrency with time control

#!/usr/bin/env bash

## create fd
[[ -e ./fd1 ]] || mkfifo ./fd1
exec 10<>./fd1
rm -rf ./fd1

MAX_CONCURRENCY=3
for i in $(seq 1 ${MAX_CONCURRENCY}); do
    echo >&10
done

## Define max execuate time here
MAX_EXEC_TIME=30

## echo $$ -> PPID
## pass PID of this process to backgroud process
T_PPID=$(echo $$)

## time tic-toc
{
    sleep ${MAX_EXEC_TIME} && exec 10<&- && exec 10>&- && echo "[ERROR] ABORT and EXIT" && kill ${T_PPID}
} &

## echo $! -> PID
## remember latest backgroud process PID
T_PID=$(echo $!)

## task list
for TASK in $(seq 1 10); do
    read -u10
    {
        ## do something here
        echo "start_task_${TASK}: $!"
        sleep 3
        echo >&10
    } &
done
kill ${T_PID}
wait

## close fd read & write
exec 10<&-
exec 10>&-

## normal exit
echo "[INFO] NORMAL EXIT" && exit 0
Feedback







Disclaimer
  • Welcome to visit the knowledge base of SRE and DevOps!
  • License under CC BY-NC 4.0
  • Made with Material for MkDocs and improve writing by generative AI tools
  • Copyright issue feedback me#imzye.com, replace # with @