Bash - How can I cut blocks of text into individual files, then place them back in after manipulation?

Question

I have a file with this text in:

{{- if .Values.cxvxcxcvxcvx -}}
{{- if .Values.xcvxcvcxv.enabled }}
---
apiVersion: xx
kind: xcvxcv
metadata:
  name: xxxxxxxxxxxx
  namespace: xxxxxxxx
type: zxczxc
stringData:
  app-sql-host: gdsfgdfgdfg.dfgdfgdfgdg.zxczxczcc.xcvx.zxczxczc.com
  app-sql-database: xxxxxxxxxxxxxxxx
  app-sql-password: xxxxxxxxxxxxxxxx
  xx-sql-zxczx: xxxxxxxx
  xx-sql-zxczxc: xxxxxxxx
  sql-use-xx-user: 'no'
  app-sql-username: xxxxxxxx
{{- end }}
{{- if .Values.xcvxcv.enabled }}
---
apiVersion: xx
kind: xcvxcv
metadata:
  name: xxxxxxxxxxxx
  namespace: xxxxxxxx
type: xcv
stringData:
  xxx-url: xcvc:xcv://xxxxxxxxxxxxxxxxxxxxxxxxxxxx;
  username: xxxx
  password: xxxx
{{- end }}
{{- end -}}

Here is the pseudo-code/steps I need to perform:

1. Get the text located between the first --- and { .
2. Move/copy the text into a temporary file.
3. Repeat for each occurrence of text between any following --- and { .
4. Run cat -n <temp_file> > <temp_file>
5. Insert each file's content back between the --- and { where it came from.

Expected output:

{{- if .Values.cxvxcxcvxcvx -}}
{{- if .Values.xcvxcvcxv.enabled }}
---
apiVersion: xx
kind: xcvxcv
metadata:
  name: xxxxxxxxxxxx
  namespace: xxxxxxxx
type: zxczxc
encryptedData:
  app-sql-host: W8l4BFOTOzg4PDjjVVS7jrjhNy0WEcSjkWY0r93SK54cM1KM77bzSxAiNfwDW9liom8nESqezUcmX3V8A2IFMQJufNkOLUsUHAVPYWzAd0RqHhRHD6x2RkOsasGr46Jm3p14Dcz6oDYzETtwUhMJCVfAL7FXqqQWNOPXQyFhX7Oo05L7v34W4GGEDSwQo4ZqE9rJeYd4OXlsxkvfcIVqkZ5YsacczTXSsJWGdqPr3tsmplznaKZfkLqQgUzayXWdNI5H12FtBILqOr7TEdVEOVk2VkP7Nj4WZyrF00hbRhBi
  app-sql-database: W8l4BFOTOzg4PDjjVVS7jrjhNy0WEcSjkWY0r93SK54cM1KM77bzSxAiNfwDW9liom8nESqezUcmX3V8A2IFMQJufNkOLUsUHAVPYWzAd0RqHhRHD6x2RkOsasGr46Jm3p14Dcz6oDYzETtwUhMJCVfAL7FXqqQWNOPXQyFhX7Oo05L7v34W4GGEDSwQo4ZqE9rJeYd4OXlsxkvfcIVqkZ5YsacczTXSsJWGdqPr3tsmplznaKZfkLqQgUzayXWdNI5H12FtBILqOr7TEdVEOVk2VkP7Nj4WZyrF00hbRhBi
  app-sql-password: W8l4BFOTOzg4PDjjVVS7jrjhNy0WEcSjkWY0r93SK54cM1KM77bzSxAiNfwDW9liom8nESqezUcmX3V8A2IFMQJufNkOLUsUHAVPYWzAd0RqHhRHD6x2RkOsasGr46Jm3p14Dcz6oDYzETtwUhMJCVfAL7FXqqQWNOPXQyFhX7Oo05L7v34W4GGEDSwQo4ZqE9rJeYd4OXlsxkvfcIVqkZ5YsacczTXSsJWGdqPr3tsmplznaKZfkLqQgUzayXWdNI5H12FtBILqOr7TEdVEOVk2VkP7Nj4WZyrF00hbRhBi
  xx-sql-zxczx: W8l4BFOTOzg4PDjjVVS7jrjhNy0WEcSjkWY0r93SK54cM1KM77bzSxAiNfwDW9liom8nESqezUcmX3V8A2IFMQJufNkOLUsUHAVPYWzAd0RqHhRHD6x2RkOsasGr46Jm3p14Dcz6oDYzETtwUhMJCVfAL7FXqqQWNOPXQyFhX7Oo05L7v34W4GGEDSwQo4ZqE9rJeYd4OXlsxkvfcIVqkZ5YsacczTXSsJWGdqPr3tsmplznaKZfkLqQgUzayXWdNI5H12FtBILqOr7TEdVEOVk2VkP7Nj4WZyrF00hbRhBi
  xx-sql-zxczxc: W8l4BFOTOzg4PDjjVVS7jrjhNy0WEcSjkWY0r93SK54cM1KM77bzSxAiNfwDW9liom8nESqezUcmX3V8A2IFMQJufNkOLUsUHAVPYWzAd0RqHhRHD6x2RkOsasGr46Jm3p14Dcz6oDYzETtwUhMJCVfAL7FXqqQWNOPXQyFhX7Oo05L7v34W4GGEDSwQo4ZqE9rJeYd4OXlsxkvfcIVqkZ5YsacczTXSsJWGdqPr3tsmplznaKZfkLqQgUzayXWdNI5H12FtBILqOr7TEdVEOVk2VkP7Nj4WZyrF00hbRhBi
  sql-use-xx-user: W8l4BFOTOzg4PDjjVVS7jrjhNy0WEcSjkWY0r93SK54cM1KM77bzSxAiNfwDW9liom8nESqezUcmX3V8A2IFMQJufNkOLUsUHAVPYWzAd0RqHhRHD6x2RkOsasGr46Jm3p14Dcz6oDYzETtwUhMJCVfAL7FXqqQWNOPXQyFhX7Oo05L7v34W4GGEDSwQo4ZqE9rJeYd4OXlsxkvfcIVqkZ5YsacczTXSsJWGdqPr3tsmplznaKZfkLqQgUzayXWdNI5H12FtBILqOr7TEdVEOVk2VkP7Nj4WZyrF00hbRhBi
  app-sql-username: W8l4BFOTOzg4PDjjVVS7jrjhNy0WEcSjkWY0r93SK54cM1KM77bzSxAiNfwDW9liom8nESqezUcmX3V8A2IFMQJufNkOLUsUHAVPYWzAd0RqHhRHD6x2RkOsasGr46Jm3p14Dcz6oDYzETtwUhMJCVfAL7FXqqQWNOPXQyFhX7Oo05L7v34W4GGEDSwQo4ZqE9rJeYd4OXlsxkvfcIVqkZ5YsacczTXSsJWGdqPr3tsmplznaKZfkLqQgUzayXWdNI5H12FtBILqOr7TEdVEOVk2VkP7Nj4WZyrF00hbRhBi
{{- end }}
{{- if .Values.xcvxcv.enabled }}
---
apiVersion: xx
kind: xcvxcv
metadata:
  name: xxxxxxxxxxxx
  namespace: xxxxxxxx
type: xcv
encryptedData:
  xxx-url: W8l4BFOTOzg4PDjjVVS7jrjhNy0WEcSjkWY0r93SK54cM1KM77bzSxAiNfwDW9liom8nESqezUcmX3V8A2IFMQJufNkOLUsUHAVPYWzAd0RqHhRHD6x2RkOsasGr46Jm3p14Dcz6oDYzETtwUhMJCVfAL7FXqqQWNOPXQyFhX7Oo05L7v34W4GGEDSwQo4ZqE9rJeYd4OXlsxkvfcIVqkZ5YsacczTXSsJWGdqPr3tsmplznaKZfkLqQgUzayXWdNI5H12FtBILqOr7TEdVEOVk2VkP7Nj4WZyrF00hbRhBi
  username: W8l4BFOTOzg4PDjjVVS7jrjhNy0WEcSjkWY0r93SK54cM1KM77bzSxAiNfwDW9liom8nESqezUcmX3V8A2IFMQJufNkOLUsUHAVPYWzAd0RqHhRHD6x2RkOsasGr46Jm3p14Dcz6oDYzETtwUhMJCVfAL7FXqqQWNOPXQyFhX7Oo05L7v34W4GGEDSwQo4ZqE9rJeYd4OXlsxkvfcIVqkZ5YsacczTXSsJWGdqPr3tsmplznaKZfkLqQgUzayXWdNI5H12FtBILqOr7TEdVEOVk2VkP7Nj4WZyrF00hbRhBi
  password: W8l4BFOTOzg4PDjjVVS7jrjhNy0WEcSjkWY0r93SK54cM1KM77bzSxAiNfwDW9liom8nESqezUcmX3V8A2IFMQJufNkOLUsUHAVPYWzAd0RqHhRHD6x2RkOsasGr46Jm3p14Dcz6oDYzETtwUhMJCVfAL7FXqqQWNOPXQyFhX7Oo05L7v34W4GGEDSwQo4ZqE9rJeYd4OXlsxkvfcIVqkZ5YsacczTXSsJWGdqPr3tsmplznaKZfkLqQgUzayXWdNI5H12FtBILqOr7TEdVEOVk2VkP7Nj4WZyrF00hbRhBi
template:
  metadata:
    annotations:
      sealedsecrets.bitnami.com/<example-pathparameter>: "true"
{{- end }}
{{- end -}}

I think AWK is the best way to do this but I'm open to other Bash solutions. gawk is not available to me.

Answer 1

I'm really not convinced you need your command to modify the temp file so here's a solution that will work with any awk that implements fflush() (which is most of them and will be required by POSIX in the upcoming 2022 standards release, see https://www.austingroupbugs.net/view.php?id=634 for details) and uses a temp file (probably not necessary at all if you have GNU awk for co-processes) but assumes your command you run on it can print to stdout:

$ cat tst.awk
inBlock {
    if ( /^{/ ) {
        fflush()
        close(tmp)
        cmd = "cat -n \047" tmp "\047"
        while ( (cmd | getline line) > 0 ) {
            print line
        }
        close(cmd)
        inBlock = 0
    }
    else {
        print > tmp
    }
}

!inBlock

/^---/ { inBlock = 1 }

$ tmp=$(mktemp) && awk -v tmp="$tmp" -f tst.awk file && rm -f "$tmp"
{{- if .Values.cxvxcxcvxcvx -}}
{{- if .Values.xcvxcvcxv.enabled }}
---
     1  apiVersion: xx
     2  kind: xcvxcv
     3  metadata:
     4    name: xxxxxxxxxxxx
     5    namespace: xxxxxxxx
     6  type: zxczxc
     7  stringData:
     8    app-sql-host: gdsfgdfgdfg.dfgdfgdfgdg.zxczxczcc.xcvx.zxczxczc.com
     9    app-sql-database: xxxxxxxxxxxxxxxx
    10    app-sql-password: xxxxxxxxxxxxxxxx
    11    xx-sql-zxczx: xxxxxxxx
    12    xx-sql-zxczxc: xxxxxxxx
    13    sql-use-xx-user: 'no'
    14    app-sql-username: xxxxxxxx
{{- end }}
{{- if .Values.xcvxcv.enabled }}
---
     1  apiVersion: xx
     2  kind: xcvxcv
     3  metadata:
     4    name: xxxxxxxxxxxx
     5    namespace: xxxxxxxx
     6  type: xcv
     7  stringData:
     8    xxx-url: xcvc:xcv://xxxxxxxxxxxxxxxxxxxxxxxxxxxx;
     9    username: xxxx
    10    password: xxxx
{{- end }}
{{- end -}}

Obviously replace cat -n with whatever command you really want to run on the temp files.

---

EDIT: without using a temp file:

$ cat tst.awk
inBlock {
    if ( /^{/ ) {
        fflush()
        cmd = "cat -n"
        print buf | cmd
        close(cmd)
        buf = ""
        inBlock = 0
    }
    else {
        buf = (inBlock++ > 1 ? buf ORS : "") $0
    }
}

!inBlock

/^---/ { inBlock = 1 }

$ awk -f tst.awk file
{{- if .Values.cxvxcxcvxcvx -}}
{{- if .Values.xcvxcvcxv.enabled }}
---
     1  apiVersion: xx
     2  kind: xcvxcv
     3  metadata:
     4    name: xxxxxxxxxxxx
     5    namespace: xxxxxxxx
     6  type: zxczxc
     7  stringData:
     8    app-sql-host: gdsfgdfgdfg.dfgdfgdfgdg.zxczxczcc.xcvx.zxczxczc.com
     9    app-sql-database: xxxxxxxxxxxxxxxx
    10    app-sql-password: xxxxxxxxxxxxxxxx
    11    xx-sql-zxczx: xxxxxxxx
    12    xx-sql-zxczxc: xxxxxxxx
    13    sql-use-xx-user: 'no'
    14    app-sql-username: xxxxxxxx
{{- end }}
{{- if .Values.xcvxcv.enabled }}
---
     1  apiVersion: xx
     2  kind: xcvxcv
     3  metadata:
     4    name: xxxxxxxxxxxx
     5    namespace: xxxxxxxx
     6  type: xcv
     7  stringData:
     8    xxx-url: xcvc:xcv://xxxxxxxxxxxxxxxxxxxxxxxxxxxx;
     9    username: xxxx
    10    password: xxxx
{{- end }}
{{- end -}}