Is there a way to copy Redshift workload management settings from existing one?
I have a complex WLM configuration in my redshift parameter group, which I want to copy to another configuration. Doing it manually is error prone. Is there a way to copy it automatically?
Here is my WLM config JSON, which I copied from Redshift console's WLM config page:
[
{
"query_group": [
"defaulton",
"lab4on"
],
"query_group_wild_card": 0,
"user_group": [],
"user_group_wild_card": 0,
"concurrency_scaling": "off",
"priority": "normal",
"queue_type": "auto",
"auto_wlm": true
},
{
"query_group": [
"lab1",
"lab2",
"lab3normal",
"lab4off",
"lab5",
"lab6reads",
"defaultoff"
],
"query_group_wild_card": 0,
"user_group": [],
"user_group_wild_card": 0,
"concurrency_scaling": "off",
"priority": "normal",
"queue_type": "auto",
"auto_wlm": true
},
{
"query_group": [
"lab3highest"
],
"query_group_wild_card": 0,
"user_group": [],
"user_group_wild_card": 0,
"concurrency_scaling": "off",
"priority": "highest",
"queue_type": "auto",
"auto_wlm": true
},
{
"query_group": [
"lab3lowest"
],
"query_group_wild_card": 0,
"user_group": [],
"user_group_wild_card": 0,
"concurrency_scaling": "off",
"priority": "lowest",
"queue_type": "auto",
"auto_wlm": true
},
{
"query_group": [
"lab6writes"
],
"query_group_wild_card": 0,
"user_group": [],
"user_group_wild_card": 0,
"concurrency_scaling": "off",
"priority": "highest",
"queue_type": "auto",
"auto_wlm": true
},
{
"query_group": [],
"query_group_wild_card": 0,
"user_group": [],
"user_group_wild_card": 0,
"priority": "normal",
"queue_type": "auto",
"auto_wlm": true
},
{
"short_query_queue": false
}
]
I could copy it to a new parameter group following below steps, but would like to simplify it if possible.
"
) with backslash doublequotes ( \\"
)modify_pg.json
[
{
"ParameterName": "wlm_json_configuration",
"ParameterValue": "<input_formatted_json_here>"
}
]
<input_formatted_json_here>
above with the formatted text in my editor (created in step# 3 above)myclusterparametergroup
aws redshift modify-cluster-parameter-group --parameter-group-name myclusterparametergroup --parameters file://modify_pg.json
Below is my final modify_pg.json
file, which I am able to use in my new parameter group. But is there a way to simplify this?
[
{
"ParameterName": "wlm_json_configuration",
"ParameterValue": "[{\"query_group\":[\"defaulton\",\"lab4on\"],\"query_group_wild_card\":0,\"user_group\":[],\"user_group_wild_card\":0,\"concurrency_scaling\":\"off\",\"priority\":\"normal\",\"queue_type\":\"auto\",\"auto_wlm\":true},{\"query_group\":[\"lab1\",\"lab2\",\"lab3normal\",\"lab4off\",\"lab5\",\"lab6reads\",\"defaultoff\"],\"query_group_wild_card\":0,\"user_group\":[],\"user_group_wild_card\":0,\"concurrency_scaling\":\"off\",\"priority\":\"normal\",\"queue_type\":\"auto\",\"auto_wlm\":true},{\"query_group\":[\"lab3highest\"],\"query_group_wild_card\":0,\"user_group\":[],\"user_group_wild_card\":0,\"concurrency_scaling\":\"off\",\"priority\":\"highest\",\"queue_type\":\"auto\",\"auto_wlm\":true},{\"query_group\":[\"lab3lowest\"],\"query_group_wild_card\":0,\"user_group\":[],\"user_group_wild_card\":0,\"concurrency_scaling\":\"off\",\"priority\":\"lowest\",\"queue_type\":\"auto\",\"auto_wlm\":true},{\"query_group\":[\"lab6writes\"],\"query_group_wild_card\":0,\"user_group\":[],\"user_group_wild_card\":0,\"concurrency_scaling\":\"off\",\"priority\":\"highest\",\"queue_type\":\"auto\",\"auto_wlm\":true},{\"query_group\":[],\"query_group_wild_card\":0,\"user_group\":[],\"user_group_wild_card\":0,\"priority\":\"normal\",\"queue_type\":\"auto\",\"auto_wlm\":true},{\"short_query_queue\":false}]"
}
]
I've done this several times for clients and you are nearly there. The same way you are modifying the PG using the AWS CLI, you can extract the JSON from the existing PG using the AWS CLI.
aws redshift describe-cluster-parameters ...
You will get back a JSON that has the parameter value for a specified PG. It is then just a matter of modifying the returned values to the string format you need. This requires some programming or scripting - I'll go with Linux scripting as this is a popular option. Command line tools you will likely want:
The process you will follow will be something like
Now you can do this in python using boto3 or in java or ... This is just one path.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.