是否有更快的方式从Heroku而不是水龙头中提取生产数据？

Question

I often need to clone production data to investigate bugs. Even with a trivial database size heroku db:pull (taps) takes 5+ minutes and seems to have a high chance of failing. Is there an alternative method to pull the database?

Libraries for alternative processes / articles would also be appreciated.

Answer 1

Check out pgbackups . It has replaced the Heroku bundle command and will give you a the postgres equivalent of mysqldump. This is far more civilized than Taps for large datasets.

heroku pgbackups:capture

Will create a dumpfile and store it. To download the dumpfile you need the url which you get with

heroku pgbackups:url b001 (or whatever the id number of the backup is)

That will return an url from which you can download your dump. You can paste it into Firefox if you want or use curl/wget like they suggest. The use pg_restore to load the dump file into your database as they say in the docs:

pg_restore --verbose --clean --no-acl --no-owner -h localhost -U test_user -d myapp_development /home/mike/Downloads/b001.dump

pg_restore: connecting to database for restore

Answer 2

I created a shell script that automates this process (based on Mike Williamson's answer).

https://gist.github.com/921535

#!/bin/bash

# Best use case is to create a file "update_local_db.sh" in your project folder and then     
# call the command with bash update_local_db

# Follow me: @jackkinsella

function LastBackupName () { 
  heroku pgbackups | tail -n 1 | cut -d"|" -f 1
}

# This part assumes you have a low limit on no. of backups allowed
old_backup=$(LastBackupName)
heroku pgbackups:destroy $old_backup 

heroku pgbackups:capture 
new_backup=$(LastBackupName)
curl $(heroku pgbackups:url $new_backup) > temporary_backup.dump
pg_restore --verbose --clean --no-acl --no-owner -h localhost -U REPLACE_WITH_YOUR_USER -d REPLACE_WITH_YOUR_DB_NAME temporary_backup.dump 
rm -f temporary_backup.dump

Answer 3

Mike's correct - PGBackups is the way to do this. When you create a backup with PGBackups, you get access to a standard pg_dump file. Here's the relevant section of the Dev Center PGBackups article.

Answer 4

This post is quite old right now.

The newest and easiest method right now is using Heroku's pg:pull/pg:push

Answer 5

An update to Jack's script, with Heroku's recommendation as of Jan 2015.

The first part is due to running on different computers, hence my Postgres dbs has different names.

#!/bin/bash

# Run the following command: bash update_local_db.sh

# Getting computer name, which is the same as username in Postgres db
echo "Please enter name of Computer"
read input_variable
echo "You entered: $input_variable"

# Make a backup on Heroku
heroku pgbackups:capture --app APP_NAME
echo "== Created a new backup =="

# Download the backup and name it latest.dump
curl -o latest.dump `heroku pgbackups:url --app APP_NAME`
echo "== Downloaded the backup =="

# Restore local db with latest.dump
pg_restore --verbose --clean --no-acl --no-owner -h localhost -U $input_variable -d my_db_name latest.dump
echo "== Replaced db with downloaded =="

# Delete downloaded db latest.dump
rm -f latest.dump
echo "== Deleted downloaded db =="
echo "== Done! :) =="

Answer 6

Here is a script I wrote that utilizes pg:pull , as mentioned by Lomefin, to pull down a db from Heroku and replace a local one with it:

#!/bin/bash

export MAIN_DB=NAME_OF_LOCAL_DB
export TMP_DB=NAME_OF_TEMPORARY_DB

function delete_db () {
    psql -d ${1} -c "SELECT pg_terminate_backend(pg_stat_activity.pid)
FROM pg_stat_activity
WHERE pg_stat_activity.datname = '$1'
  AND pid <> pg_backend_pid();" || true

    dropdb ${1} || true
}

delete_db ${TMP_DB}

heroku pg:pull DATABASE_URL ${TMP_DB} || exit 1

delete_db ${MAIN_DB}

psql -c "ALTER DATABASE $TMP_DB RENAME TO $MAIN_DB;"

Since pg:pull clones to a new database, your work will not be interrupted (only once it renames the db, which takes a fraction of a second). The script can, of course, be easily customized to your liking.