How could I avoid repeating myself by passing the ToSql
data as a closure, telling tokio-postgres which columns of data, from my various struct models, to COPY
/ SELECT
/etc?
For instance, ideally I could pass various table/column args, like:
// see MRE, "copy_in" performs boilerplate to copy binary format in postgres table
copy_in(
&mut client, // the postgres client setup in tokio
"table_name", // meta info about what table/columns to insert
&["col1", "col2", "col3", "col4"],
&vec![Type::INT4, Type::VARCHAR, Type::INT4, Type::INT4],
&rows_of_struct_data, // the vec of data I plan to insert
// then here's the hard part: which fields to extract from each row... (not works)
|m| &[&[&m.id], &[&m.name], &[&m.range_begin, &m.range_end]] as &[&[&(dyn ToSql + Sync)]]
);
tokio_postgres
requires a type for writing binary data during a copy , Vec<&(dyn ToSql + Sync)>
, and though hardcoding what gets push
ed into that Vec works, I am puzzled how to pass an argument that decides dynamically what to push
into it.
I have two ideas for this, but what is recommended approach?
expected
&dyn ToSql + Sync
, found trait objectdyn ToSql
Vec
to my closure for it to push columns data into for each row, but that had several lifetime issues, like:reference to
row
escapes the async closure body here
Out of my two approaches, I finally got #1 working, though I'm not accepting my own answer yet in case a better recommendation comes along.
Here is the working MRE
Here is what I was missing in my understanding:
ToSql
type was violated if a single Vec contained both i32
and String
Interested to see for purely functional programming learning what others might recommend. This is really helpful understanding how closures can be used. #2 approach has lifetime challenges I'm interested if others have insights on.
I don't know how to return tokio_postgres &dyn ToSql + Sync type. But I resolve this problem this way below. I am inserting the data using postgres's binary copy.
Table:
CREATE TABLE IF NOT EXISTS test (
id SERIAL PRIMARY KEY,
name VARCHAR NOT NULL
)
To populate the table, I use:
use tokio_postgres::types::{Type, ToSql};
use tokio_postgres::binary_copy::BinaryCopyInWriter;
use tokio_postgres::{NoTls, Error};
use futures::{pin_mut};
async fn main() -> Result<(), Error> {
//...
let config = format!("host={} user={} password='{}' dbname={} port={} connect_timeout=10",host, user, passwd, dbname, port);
let (client, connection) = tokio_postgres::connect(&config, NoTls).await?;
tokio::spawn(async move {
if let Err(e) = connection.await {
eprintln!("Connection error: {}", e);
}
});
let data: Vec<(i32,&str)> = vec![(1i32,"hello,"),
(2i32,"world!"),
(3i32,"how"),
(4i32,"are"),
(5i32,"you"),
(6i32,"today?")];
let sink = client
.copy_in("COPY test (id, name) FROM STDIN BINARY")
.await
.unwrap();
let writer = BinaryCopyInWriter::new(sink, &[Type::INT4, Type::VARCHAR]);
pin_mut!(writer);
for i in 0..data.len() {
let (id, name) = data[i];
writer.as_mut().write(&[&id, &name]).await.unwrap();
}
writer.finish().await?;
Ok(())
}
I use tuples to store the data, Then I unpack the tuple to insert the data on the database. The dependencies are tokio-postgres = "0.7.6", tokio = { version = "1.20.0", features = ["full"] } and futures = "0.3".
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.