How to delete a line in a csv file after processing it with a Linux bash script

Question

I use a Linux bash script to create posts on my WordPress website with wp-cli. this is the example of the script that I use:

#! /bin/bash

while IFS="|" read -r rec_column1 rec_column2 rec_column3 rec_column4
do
cd /posts/ &&
wp post create \
    --post_type=post \
    --post_title="$rec_column1" \
    --post_status=publish \
    --post_category=2 \
    --url=http://127.0.0.1/ \
    --meta_input="{\"word\":\"$rec_column2\"}" \
    --tags_input="$rec_column3" \
    --post_content="$rec_column4"
done < <(tail -n +2 ./files/$1) > ./logs/$1.log

and this is an example of the csv file that I that the data from:

title|word|tags|content
Lorem|ipsum|dolor|sit
amet|consectetur|adipiscing|elit
Maecenas|sed|condimentum|est
in|fermentum|justo|Aenean

The issue that I'm facing is that if the post creation gets interrupted for some reason, I would than have to take a look at the log file to see what was the post id of the last post created, afterwards I go to the WordPress site, see the that last post, find the line in the csv file, delete all the line above that line and start over.

I would like to improve that process so, after the post is created, the line from which that post was created in the CSV file is deleted. so next time that the post creation process gets interrupted, I will be able to continue it from the place it was stopped in.

for example if the process was stopped after the first line the new csv file will look like:

title|word|tags|content
amet|consectetur|adipiscing|elit
Maecenas|sed|condimentum|est
in|fermentum|justo|Aenean

I'll appreciate any suggestions or code modification on how to make this work like I want it to.

Answer 1

In the while loop, store current line number in a variable $LINE_NUMBER .

Create a function delete_lines() which deletes processed lines from the input file:

delete_lines() {
  sed -i "1,${LINE_NUMBER}d" ./files/$FILE_NAME
  exit
}

Finally, set up a trap which calls this function on error, exit or script interruption:

trap delete_lines EXIT ERR INT TERM

Whole solution:

#!/bin/bash

LINE_NUMBER=1
FILE_NAME="./files/$1"

delete_lines() {
  if [[ "$LINE_NUMBER" -gt "1" ]]; then
    sed -i "2,${LINE_NUMBER}d" "$FILE_NAME"
  fi
  exit
}

trap delete_lines ERR INT TERM

while IFS="|" read -r rec_column1 rec_column2 rec_column3 rec_column4
do
  cd /posts/ &&
  wp post create \
      --post_type=post \
      --post_title="$rec_column1" \
      --post_status=publish \
      --post_category=2 \
      --url=http://127.0.0.1/ \
      --meta_input="{\"word\":\"$rec_column2\"}" \
      --tags_input="$rec_column3" \
      --post_content="$rec_column4"
  LINE_NUMBER=$((LINE_NUMBER+1))
done < <(tail -n +2 "$FILE_NAME") > ./logs/$1.log

delete_lines

Answer 2

trap delete_lines EXIT ERR INT TERM

delete_lines() {
  sed -i "2,${LINE_NUMBER}d" ${FILES_DIR}/$FILE_NAME
  exit
}

FILE_NAME="$1"
FILES_DIR="/path/to/files"
LOGS_DIR="/path/to/logs"

while IFS="|" read -r LINE_NUMBER rec_column1 rec_column2 rec_column3 rec_column4
do
cd /posts/ &&
wp post create \
    --post_type=post \
    --post_title="$rec_column1" \
    --post_status=publish \
    --post_category=2 \
    --url=http://127.0.0.1/ \
    --meta_input="{\"word\":\"$rec_column2\"}" \
    --tags_input="$rec_column3" \
    --post_content="$rec_column4"
done < <(awk 'NR!=1 { print NR"|"$0}' ${FILES_DIR}/${FILE_NAME}) > ${LOGS_DIR}/${FILE_NAME}.log

delete_lines