简体   繁体   中英

Is there a way to setup a Snowpipe on multiple Azure Storage Accounts?

My company set up multiple storage accounts (about 30) for an app with the same container name on the same Azure tenant ID (ex. azure://myaccount1.blob.core.windows.net/samecontainername, azure://myaccount2.blob.core.windows.net/samecontainername, azure://myaccount3.blob.core.windows.net/samecontainername)

I've created Snowpipes in the past and the process was:

  1. Create a queue on the account and container
  2. Create an Event Grid linked to that account and container and with a destination set as the created queue
  3. Create a Notification Integration in Snowflake (to a queue related to that account and container)
  4. Authenticate the service principal app Snowflake created (if it is the first time on that tenant ID)
  5. Create a stage using the account and blob url (azure://myaccount2.blob.core.windows.net/samecontainername) and using the notification integration created
  6. Create a pipe using the notification integration created and copy from the stage created into a variant table (for Json files)

The question is, is there a way to simplify this so I won't have to do the steps above 30 times? Each account contains the same container name

Seems not as snowpipe can have one NOTIFICATION INTEGRATION per queue. However, we have setup a copy statement from all accounts into one not used for PROD on which we set up a NOTIFICATION INTEGRATION to Snowflake and then a PIPE using that integration

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM