Extracting information from a string using regex in bash

Question

I have a string variable in bash which looks like so:

{"SOGoTimeFormat": "%H:%M", "SOGoMailShowSubscribedFoldersOnly": "0", "SOGoMailSignaturePlacement": "below", "SOGoLanguage": "English", "SOGoDayEndTime": "18:00", "SOGoDefaultCalendar": "selected", "SOGoFirstWeekOfYear": "January1", "SOGoFirstDayOfWeek": "0", "SOGoTimeZone": "Asia\/Kolkata", "SOGoContactsCategories": ["Business Partner", "Colleague", "Competitor", "Customer", "Family", "Friend", "Press", "Provider", "VIP"], "Vacation": {"enabled": 0, "endDate": 1374690600, "autoReplyEmailAddresses": ["testuser@testdomain.com"], "ignoreLists": 1, "autoReplyText": "", "daysBetweenResponse": "7", "endDateEnabled": 0}, "SOGoCalendarTasksDefaultClassification": "PUBLIC", "SOGoMailSortByThreads": "0", "SOGoMailMessageCheck": "manually", "SOGoMailMessageForwarding": "inline", "SOGoLoginModule": "Mail", "SOGoCalendarCategoriesColors": {"Customer": "#aaa", "Calls": "#aaa", "Favorites": "#aaa", "Meeting": "#aaa", "Ideas": "#aaa", "Miscellaneous": "#aaa", "Birthday": "#aaa", "Anniversary": "#aaa", "Vacation": "#aaa", "Travel": "#aaa", "Projects": "#aaa", "Suppliers": "#aaa", "Gifts": "#aaa", "Clients": "#aaa", "Issues": "#aaa", "Business": "#aaa", "Holidays": "#aaa", "Personal": "#aaa", "Status": "#aaa", "Public Holiday": "#aaa", "Follow up": "#aaa", "Competition": "#aaa"}, "SOGoBusyOffHours": "0", "SOGoCalendarCategories": ["Customer", "Calls", "Favorites", "Meeting", "Ideas", "Miscellaneous", "Birthday", "Anniversary", "Vacation", "Travel", "Projects", "Suppliers", "Gifts", "Clients", "Issues", "Business", "Holidays", "Personal", "Status", "Competition", "Follow up", "Public Holiday"], "SOGoCalendarEventsDefaultClassification": "PUBLIC", "Forward": {"enabled": 1, "forwardAddress": ["testuser1@testdomain.com", "testuser2@testdomain.com"], "keepCopy": 1}, "SOGoRememberLastModule": "0", "SOGoMailReplyPlacement": "below", "SOGoMailDisplayRemoteInlineImages": "never", "SOGoSieveFilters": [{"actions": [{"method": "fileinto", "argument": "INBOX\/spam"}], "active": 1, "rules": [{"operator": "contains", "field": "subject", "value": "[SPAM]"}], "match": "any", "name": "spam"}, {"actions": [{"method": "fileinto", "argument": "INBOX\/spam"}], "active": 1, "rules": [{"operator": "contains", "field": "subject", "value": "TESTTEST"}], "match": "any", "name": "new"}], "SOGoDayStartTime": "08:00", "SOGoMailComposeMessageType": "text"}

It is a single line of text, not wrapped or anything. What I am trying to achieve is, there is a field in this called "Forward" . If it's corresponding enabled value is 0, do nothing. If it's corresponding enabled value is 1, it should the parse the email addresses inside forwardAddress one-by-one inside and delete one based on some comparison (In this string, let's say we want to delete testuser2).

I have two questions:

• How do I achieve this using regex to find "Forward" and then check the enabled value?
• Should I extract them into a new string, edit it and then write it back or is there a more efficient method?

Answer 1

What you have is JSON and what you should be using is a JSON parser. Using regex is not a good substitute.

Here's some python that loads the string, and if enabled in Forward is 1, deletes any address with the substring "testuser2" from the forwardAddress list:

#!/bin/python
import sys
import json

thing = json.load(sys.stdin)
forward = thing["Forward"]

if forward["enabled"] == 1:
    forward["forwardAddress"] = \
        filter(lambda x: not "testuser2" in x, \
            forward["forwardAddress"])

json.dump(thing, sys.stdout)

You can run it with

echo "$yourvariable" | python thisfile.py

The json re-encoding process might shuffle the fields. This doesn't matter, as the strings still represent the same json objects.

Answer 2

jq is an excellent tool for parsing and editing JSON, easy to drive from shell.

# extract "enabled" field from "Forward"
enabled=$(jq '.Forward.enabled` <input.json)

...or, to do the whole thing in one pass:

jq '
  if .Forward.enabled==1 then
    .Forward.forwardAddress=[(
      .Forward.forwardAddress[] |
      select(. | test("testuser2") | not)
    )]
  else . end
' <input.json >output.json