[英]Read slave, read-write master setup
I have a Flask, SQLAlchemy webapp which uses a single mysql server.我有一个 Flask、SQLAlchemy webapp,它使用一个 mysql 服务器。 I want to expand the database setup to have a read-only slave server such that I can spread the reads between both master and slave while continuing to write to the master db server.
我想扩展数据库设置以拥有一个只读的从属服务器,这样我就可以在继续写入主数据库服务器的同时在主从服务器之间传播读取。
I have looked at few options and I believe I can't do this with plain SQLAlchemy. Instead I'm planning to create 2 database handles in my webapp, one each for master and slave db servers.我已经查看了几个选项,我相信我不能用普通的 SQLAlchemy 来做到这一点。相反,我计划在我的 webapp 中创建 2 个数据库句柄,每个用于主数据库服务器和从数据库服务器。 Then using a simple random value use either the master/slave db handle for "SELECT" operations.
然后使用一个简单的随机值使用主/从数据库句柄进行“SELECT”操作。
However, I'm not sure if this is the right way to go with using SQLAlchemy. Any suggestion/tips on how to pull this off?但是,我不确定这是否是使用 SQLAlchemy 到 go 的正确方法。关于如何实现这一点的任何建议/提示?
I have an example of how to do this on my blog at http://techspot.zzzeek.org/2012/01/11/django-style-database-routers-in-sqlalchemy/ . 我在http://techspot.zzzeek.org/2012/01/11/django-style-database-routers-in-sqlalchemy/上的博客中提供了有关如何执行此操作的示例。 Basically you can enhance the Session so that it chooses from master or slave on a query-by-query basis.
基本上,您可以增强会话,以便在逐个查询的基础上从主服务器或从服务器中进行选择。 One potential glitch with that approach is that if you have one transaction that calls six queries, you might end up using both slaves in one request....but there we're just trying to imitate Django's feature :)
这种方法的一个潜在故障是,如果您有一个事务调用了六个查询,则您可能最终在一个请求中使用了两个从属。...但是我们只是在尝试模仿Django的功能:)
A slightly less magic approach that also establishes the scope of usage more explicitly I've used is a decorator on view callables (whatever they're called in Flask), like this: 一种稍微少用的魔术方法(也可以更明确地确定使用范围)是视图可调用项(无论在Flask中是什么)上的装饰器,如下所示:
@with_slave
def my_view(...):
# ...
with_slave would do something like this, assuming you have a Session and some engines set up: 假设您有一个Session并设置了一些引擎,with_slave会做类似的事情:
master = create_engine("some DB")
slave = create_engine("some other DB")
Session = scoped_session(sessionmaker(bind=master))
def with_slave(fn):
def go(*arg, **kw):
s = Session(bind=slave)
return fn(*arg, **kw)
return go
The idea is that calling Session(bind=slave)
invokes the registry to get at the actual Session object for the current thread, creating it if it doesn't exist - however since we're passing an argument, scoped_session will assert that the Session we're making here is definitely brand new. 这个想法是,调用
Session(bind=slave)
调用注册表以获取当前线程的实际Session对象,如果不存在则创建它-但是由于我们传递了一个参数,scoped_session将断言Session我们在这里制造的绝对是全新的。
You point it at the "slave" for all subsequent SQL. 您将其指向所有后续SQL的“从属”。 Then, when the request is over, you'd ensure that your Flask app is calling
Session.remove()
to clear out the registry for that thread. 然后,当请求结束时,您将确保Flask应用正在调用
Session.remove()
以清除该线程的注册表。 When the registry is next used on the same thread, it will be a new Session bound back to the "master". 下次在同一线程上使用注册表时,它将是绑定到“主服务器”的新会话。
Or a variant, you want to use the "slave" just for that call, this is "safer" in that it restores any existing bind back to the Session: 或者是一个变体,您只想为该调用使用“从属”,这是“安全的”,因为它将所有现有的绑定还原回Session:
def with_slave(fn):
def go(*arg, **kw):
s = Session()
oldbind = s.bind
s.bind = slave
try:
return fn(*arg, **kw)
finally:
s.bind = oldbind
return go
For each of these decorators you can reverse things, have the Session be bound to a "slave" where the decorator puts it on "master" for write operations. 对于这些装饰器中的每个装饰器,您都可以撤消操作,将会话绑定到“从属”,其中装饰器将其置于“主”上以进行写操作。 If you wanted a random slave in that case, if Flask had some kind of "request begin" event you could set it up at that point.
如果在这种情况下您想要一个随机的从属设备,则如果Flask发生某种“请求开始”事件,则可以在该时间进行设置。
Or, we can try another way. 或者,我们可以尝试另一种方式。 Such as we can declare two different class with all the instance attributes the same but the
__bind__
class attribute is different. 例如,我们可以声明两个不同的类,并且所有实例属性都相同,但是
__bind__
类属性不同。 Thus we can use rw class to do read/write and r class to do read only. 因此,我们可以使用rw类进行读/写操作,使用r类进行只读。 :)
:)
I think this way is more easy and reliable. 我认为这种方式更加简单可靠。 :)
:)
We declare two db models because we can have tables in two different db with the same names. 我们声明两个数据库模型,因为我们可以在两个不同数据库中具有相同名称的表。 This way we can also bypass the 'extend_existing' error when two models with the same __tablename__ .
这样,当两个具有相同__tablename__的模型时,我们还可以绕过'extend_existing'错误。
Here is an example: 这是一个例子:
app = Flask(__name__)
app.config['SQLALCHEMY_BINDS'] = {'rw': 'rw', 'r': 'r'}
db = SQLAlchemy(app)
db.Model_RW = db.make_declarative_base()
class A(db.Model):
__tablename__ = 'common'
__bind_key__ = 'r'
class A(db.Model_RW):
__tablename__ = 'common'
__bind_key__ = 'rw'
Maybe this answer is too late!也许这个答案为时已晚! I use a
slave_session
to query the slave DB我使用
slave_session
来查询从数据库
class RoutingSession(SignallingSession):
def __init__(self, db, bind_name=None, autocommit=False, autoflush=True, **options):
self.app = db.get_app()
if bind_name:
bind = options.pop('bind', None)
else:
bind = options.pop('bind', None) or db.engine
self._bind_name = bind_name
SessionBase.__init__(
self, autocommit=autocommit, autoflush=autoflush,
bind=bind, binds=None, **options
)
def get_bind(self, mapper=None, clause=None):
if self._bind_name is not None:
state = get_state(self.app)
return state.db.get_engine(self.app, bind=self._bind_name)
else:
if mapper is not None:
try:
persist_selectable = mapper.persist_selectable
except AttributeError:
persist_selectable = mapper.mapped_table
info = getattr(persist_selectable, 'info', {})
bind_key = info.get('bind_key')
if bind_key is not None:
state = get_state(self.app)
return state.db.get_engine(self.app, bind=bind_key)
return SessionBase.get_bind(self, mapper, clause)
class RouteSQLAlchemy(SQLAlchemy):
def __init__(self, *args, **kwargs):
SQLAlchemy.__init__(self, *args, **kwargs)
self.slave_session = self.create_scoped_session({'bind_name':
'slave'})
def create_session(self, options):
return orm.sessionmaker(class_=RoutingSession,db=self,**options)
db = RouteSQLAlchemy(metadata=metadata, query_class=orm.Query)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.