[英]Can't store and then read Files in PostgreSQL as large objects
我正在嘗試使用NpgSQL v3.0.4.0向PostgreSQL數據庫V9.4.x寫入和讀取大對象,因此我制定了一種將本地文件作為大對象存儲在數據庫中的方法,如下所示:
public static async Task<uint> InsertLargeObjectFileToDB(string theFilePath)
{
// connecting to DB
string connstring = MakeDatabaseConnectionString();
// make a connection object
NpgsqlConnection Conn = new NpgsqlConnection(connstring);
try
{
await OpenDatabaseConnection(Conn); //open database connection
}
catch (Exception Ex)
{
throw (Ex);
}
uint oid; // to store object ID number
try
{
// Reading and writing Large Objects requires the use of a transaction
using (FileStream fs = new FileStream(theFilePath, FileMode.Open))
{
using (var transaction = Conn.BeginTransaction())
{
// Retrieve a Large Object Manager for this connection
var manager = new NpgsqlLargeObjectManager(Conn);
// Create a new empty file, returning the identifier to later access it
oid = manager.Create();
using (var DbStream = manager.OpenReadWrite(oid))
{
long theFileSize = GetFileSizeInBytes(theFilePath);
StreamReader sr = new StreamReader(fs);
byte[] buffer = new byte[1024 * 1024];
while (sr.BaseStream.Position < theFileSize)
{
await fs.ReadAsync(buffer, 0, buffer.Length);
await DbStream.WriteAsync(buffer, 0, buffer.Length);
}
}
transaction.Commit();
return oid;
}
}
}
catch // any error
{
// exception
Exception ex = new Exception();
ex.Data.Add(ex.Data.Count, "some error message");
throw ex;
}
}
然后,我制作了一種方法來讀取大對象並將其存儲在temp目錄中的隨機命名文件中,如下所示:
public static async Task<string> GetLargeObjectFileFromDB(uint oid)
{
// connecting to DB
string connstring = MakeDatabaseConnectionString();
// make a connection object
NpgsqlConnection Conn = new NpgsqlConnection(connstring);
try
{
await OpenDatabaseConnection(Conn); //open database connection
}
catch (Exception Ex)
{
throw (Ex);
}
// getting a temorary file name from the system to use it to store the fetched file
string TempFileName = GetRandomFileNameFromSystem();
try
{
using (FileStream LocalStream = new FileStream(TempFileName, FileMode.Create))
{
using (var transaction = Conn.BeginTransaction())
{
// create a Large Object Manager for this connection
var DbLargeObjectManager = new NpgsqlLargeObjectManager(Conn);
using (var DbStream = await DbLargeObjectManager.OpenReadAsync(oid))
{
byte[] buffer = new byte[1024 * 1024];
// get the length of the database object
long LengthOfDbObject = DbStream.Length;
while (DbStream.Position < LengthOfDbObject)
{
// read from the database to buffer
await DbStream.ReadAsync(buffer, 0, buffer.Length);
//write from buffer to local file
await LocalStream.WriteAsync(buffer, 0, buffer.Length);
}
}
transaction.Commit();
return TempFileName;
}
}
}
catch // any error
{
// exception
Exception ex = new Exception();
ex.Data.Add(ex.Data.Count, "Error inserting object in database");
throw ex;
}
}
如您所見,我一直在異步編寫。 問題是我對這2種方法進行了測試,並且此測試將6MB文件寫入數據庫,但是當我再次從數據庫中讀取該文件時,文件大約大了400 kb,(當然)MD5哈希不匹配。 不要忘了說,沒有例外發生過。 如果您關心的話,這是測試:
private async void button3_Click(object sender, EventArgs e)
{
listBox1.Items.Clear();
// getting the MD5 hash of the source file
string FirstMd5Hash = GetMd5OfFile(tbSourceFile.Text);
// performance measurment ##########################################
DateTime dt1 = new DateTime(DateTime.Now.Ticks);
listBox1.Items.Add("Uploading file to database");
//storing that file into database
uint oid = await InsertLargeObjectFileToDB(tbSourceFile.Text);
// performance measurment #########################################################
DateTime dt2 = new DateTime(DateTime.Now.Ticks);
TimeSpan ts = new TimeSpan(dt2.Ticks - dt1.Ticks);
listBox1.Items.Add("Large object (oid = " + oid + ") inserted in " + ts.Seconds + "." + ts.Milliseconds + " seconds");
// performance measurment ##########################################
dt1 = new DateTime(DateTime.Now.Ticks);
listBox1.Items.Add("reading file back from the database");
// get that object back from the database into temporary file
string ReturnedFileName = await PostgresqlLargeObject.GetLargeObjectFileFromDB(oid);
// performance measurment #########################################################
dt2 = new DateTime(DateTime.Now.Ticks);
ts = new TimeSpan(dt2.Ticks - dt1.Ticks);
listBox1.Items.Add("reading done in " + ts.Seconds + "." + ts.Milliseconds + " seconds");
//calculate md5 of that file
string SecondMd5Hash = GetMd5OfFile(ReturnedFileName);
// compare the 2 hashes
if (FirstMd5Hash == SecondMd5Hash)
{
listBox1.Items.Add("the hashes are match . MD5 = " + FirstMd5Hash);
}
else
{
listBox1.Items.Add("failed with oid = " + oid);
tbFileBack.Text = ReturnedFileName;
}
}
怎么了?
好的,我已經解決了這個問題,事實證明(除了考慮Emil的答案),您還必須異步讀取然后同步寫入。 我不知道為什么。 此代碼的工作原理:
using (FileStream LocalStream = new FileStream(TempFileName, FileMode.Create))
{
using (var transaction = Conn.BeginTransaction())
{
// create a Large Object Manager for this connection
var DbLargeObjectManager = new NpgsqlLargeObjectManager(Conn);
using (var DbStream = await DbLargeObjectManager.OpenReadAsync(oid))
{
byte[] buffer = new byte[262144]; //256KB
// query the database stream length
long DatabaseStreamLength = DbStream.Length;
while (DbStream.Position < DatabaseStreamLength)
{
// read from the database to buffer (async)
int bufferByteCount = await DbStream.ReadAsync(buffer, 0, buffer.Length);
//write from buffer to local file (sync)
LocalStream.Write(buffer, 0, bufferByteCount);
}
}
transaction.Commit();
當執行await <stream>.WriteAsync(buffer, 0, buffer.Length);
您應該寫入由上一個read方法實際讀取的字節數(將返回該值)。
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.