[英]A producer consumer queue with an additional thread for a periodic backup of data
[英]producer/consumer get different data by Queue
解决了!!! 谢谢
小心放入Queue的对象的类型。如果您输入一个值(例如int),则入队将进行复制,每个人都很高兴。 如果您放置一个引用(例如byte [],string),将其放入队列,那么就会出现问题。 如果在使用者读取该引用之前对其进行了更改 ,那么使用者将读取更改后的数据版本。
为避免此问题,请在入队后立即在rxThread中获取新版本的帧引用 。 码:
public void rxThreadFunc()
{
byte[] data = new byte[datalen];//declare for the first iteration.
int j = 0;
while (true)
{
for (int i = 0; i < rxlen; i++)
{
data[j] = (byte)i;
j++;
if (j >= datalen)
{
j = 0;
mQ.Add(data);
using (StreamWriter fwriter = new StreamWriter("C:\\testsave\\rxdata", true))
{
for (int k = 0; k < datalen; k++)
{
fwriter.Write(data[k]);
fwriter.Write(",");
}
fwriter.Write("\n");
}
data = new byte[datalen];//create new reference after enqueue/Add
}
}
}
}//rxThreadFunc()
Update1我刚刚编写了另一个简单的代码,因此每个人都可以在没有串行端口硬件的情况下对其进行测试。 单击按钮运行程序。 我认为线程优先级会导致此问题,而无需更改rxthread的线程优先级,则dataProc线程将获取正确的数据。 但我仍然不知道为什么。
rxThread.Priority = ThreadPriority.Hightest
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using System.Collections.Concurrent;
using System.Threading;
using System.IO;
namespace WindowsFormsApplication1
{
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
}
private void button1_Click(object sender, EventArgs e)
{
Thread rxThread = new Thread(rxThreadFunc);
rxThread.Priority = ThreadPriority.Highest;//this causes problem
rxThread.Start();
Thread procThread = new Thread(dataProc);
procThread.Start();
}
BlockingCollection<byte[]> mQ = new BlockingCollection<byte[]>();
int datalen = 30;
int rxlen = 200;
public void rxThreadFunc()
{
int j = 0;
while (true)
{
byte[] data = new byte[datalen];//is this in the right place?
for (int i = 0; i < rxlen; i++)
{
data[j] = (byte)i;
j++;
if (j >= datalen)
{
j = 0;
mQ.Add(data);
using (StreamWriter fwriter = new StreamWriter("C:\\testsave\\rxdata", true))
{
for (int k = 0; k < datalen; k++)
{
fwriter.Write(data[k]);
fwriter.Write(",");
}
fwriter.Write("\n");
}
}
}
}
}//rxThreadFunc()
public void dataProc()
{
byte[] outData = new byte[datalen];
while (true)
{
if (mQ.Count > 1)
{
outData=mQ.Take();
using(StreamWriter fwriter=new StreamWriter("C:\\testsave\\dataProc",true))
{
for (int i = 0; i < datalen; i++)
{
fwriter.Write(outData[i]);
fwriter.Write(",");
}
fwriter.Write("\n");
}
}
}
}
}
}
说明:我编写了这个包含两个线程的应用程序。 RxThread从串行端口接收数据,按标题标记0x55 0xaa对其进行排序,将以下30个字节放入FrameStruct类中,然后将该FrameStruct放入Queue中。 dataProcess线程从Queue获取帧,然后将其存储到磁盘。
的SerialPort =(RXBUFF)=> RxThread =(rxFrame,队列)=> dataProcess ==>磁盘
问题:由dataProcess线程接收并保存到磁盘的数据以某种方式损坏。
尝试:这是我尝试供您参考的内容。
我也认为dataProcess收到的时间也是正确的。
5.如果我在RxThread的Monitor.Pulse()之后放置Thread.sleep(20),问题就解决了,但我不明白为什么? 如果我换到另一台电脑怎么办?
这是代码快照。
//declared:
//Queue<FrameStruct>messageQ=new Queue<FrameStruct>;
//object _LockerMQ=new object();
private void RxThread()
{
int bytestoread, i;
bool f55 = false;//55 flag
bool fs = false;//frame start flag
int j=0;//data index in FrameStruct
int m_lMaxFram=32;
bytestoread = 0;
FrameStruct rxFrame = new FrameStruct((int)m_lMaxFrame);
while (true)
{
if (Serial_Port.IsOpen == true)
{
if ((bytestoread = Serial_Port.BytesToRead) > m_lMaxFrame*2)//get at least two frames
{
rxbuff = new byte[bytestoread];
Serial_Port.Read(rxbuff, 0, bytestoread);
for (i = 0; i < bytestoread; i++)
{
if (rxbuff[i] == 0x55)
{
f55 = true;
continue;
}
if (rxbuff[i] == 0xaa && f55)
{//frame header 0x55, 0xaa
//new frame start
fs = true;
f55 = false;
j = 0;//rxframe index;
rxFrame.time = DateTime.Now;//store the datetime when this thread gets this frame
continue;
}
if (fs && j < m_lMaxFrame - 2)
{//frame started but not ended
rxFrame.data[j] = rxbuff[i];
j++;
}
if (j >= (m_lMaxFrame - 2) && fs)
{//frame ended if j=30, reaches the end of rxFrame.data
fs = false;
lock(_LockerMQ)
{
messageQ.Enqueue(rxFrame);
Monitor.Pulse(_LockerMQ);
}
//Thread.Sleep(20);//if uncomment this sleep, problem solved
using (StreamWriter fWriter = new StreamWriter("c:\\testsave\\RXdata", true))//save rxThread result into a file rawdata
{
fWriter.Write(rxFrame.time.ToString("yyyy/MM/dd HH:mm:ss.fff"));
fWriter.Write(",");
for (int k = 0; k < m_lMaxFrame - 2; k++)
{
fWriter.Write(rxFrame.data[k]);
fWriter.Write(",");
}
fWriter.Write("\n");
}
}
}
}//if ((bytestoread=Serial_Port.BytesToRead) > 0)
rxbuff = null;
Thread.Sleep(20);
}//(Serial_Port.IsOpen==true)
Thread.Sleep(100);
}//while(true),RxThread sleep
}//private void RxThread()
DataProcess线程:
public void dataProcess()
{
while (true)
{
lock (_LockerMQ)
{
while (messageQ.Count < 1) Monitor.Wait(_LockerMQ);//get at least one frame data
f_NewFrame = messageQ.Count;
if (f_NewFrame > 0)
{
procFrame = messageQ.Dequeue();
using (StreamWriter fWriter = new StreamWriter("c:\\testsave\\dPdata", true))
{
fWriter.Write(procFrame.time.ToString("yyyy/MM/dd HH:mm:ss.fff"));
fWriter.Write(",");
for (int i = 0; i < m_lMaxFrame - 2; i++)
{
fWriter.Write(procFrame.data[i]);
fWriter.Write(",");
}
fWriter.Write("\n");
}
}//if(f_NewFrame>0)
}//lock(messageQ)
}
}
FrameStruct包含时间成员和数据[30]
class FrameStruct
{
public FrameStruct(int m_lMaxFrame)
{
time = DateTime.Now;
data = new byte[m_lMaxFrame - 2];
}
public DateTime time;
public volatile byte[] data;//volatile doesn't help
}
RxThread保存的rxData是正确的,显示:
2015/07/18 18:40:26.125,127,255,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,111,51,204,
2015/07/18 18:40:26.177,128,0,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,112,51,204,
2015/07/18 18:40:26.177,128,0,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,113,51,204,
2015/07/18 18:40:26.297,127,255,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,114,51,204,
2015/07/18 18:40:26.298,127,255,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,115,51,204,
2015/07/18 18:40:26.298,127,255,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,116,51,204,
2015/07/18 18:40:26.299,127,255,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,117,51,204,
2015/07/18 18:40:26.420,127,255,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,118,51,204,
//^this columns is accumulated number
dataProcessThread保存的dPdata是WRONG,显示:
2015/07/18 18:40:31.904,127,255,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,227,51,204,
2015/07/18 18:40:31.905,127,255,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,228,51,204,
2015/07/18 18:40:31.905,127,255,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,229,51,204,
2015/07/18 18:40:32.026,128,0,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,231,51,204,
2015/07/18 18:40:32.026,128,0,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,231,51,204,
2015/07/18 18:40:32.147,128,0,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,232,51,204,
2015/07/18 18:40:32.148,128,0,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,233,51,204,
2015/07/18 18:40:32.148,128,0,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,234,51,204,
2015/07/18 18:40:32.269,128,0,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,236,51,204,
2015/07/18 18:40:32.269,128,0,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,236,51,204,
2015/07/18 18:40:32.510,128,0,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,237,51,204,
2015/07/18 18:40:32.512,128,0,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,240,51,204,
2015/07/18 18:40:32.512,128,0,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,240,51,204,
2015/07/18 18:40:32.514,127,255,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,240,51,204,
2015/07/18 18:40:32.514,127,255,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,241,51,204,
2015/07/18 18:40:32.635,128,0,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,243,51,204,
2015/07/18 18:40:32.635,128,0,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,243,51,204,
//^this accumulated number is not correct
请帮忙!
谢谢!
FrameStruct是一个类(而不是结构),排队时,您一次又一次使用相同的引用。
更新2
这是基本的生产者使用者,只需添加您的逻辑即可:
class Program
{
static BlockingCollection<int> mQ = new BlockingCollection<int>();
static void Main(string[] args)
{
Thread rxThread = new Thread(rxThreadFunc);
rxThread.Priority = ThreadPriority.Highest;//this causes problem
rxThread.Start();
Thread procThread = new Thread(dataProc);
procThread.Start();
Console.ReadLine();
}
static public void rxThreadFunc()
{
for (int i = 0; i < 10; i++)
{
mQ.Add(i);
}
}
static public void dataProc()
{
foreach (int outData in mQ.GetConsumingEnumerable())
{
Console.WriteLine(outData);
}
}
}
回答更新1:
现在,由于使用了BlockingCollection,使用者(dataProc)可能非常简单(只需删除循环和计数检查,它将为您完成所有这些同步操作):
foreach (byte[] outData in _taskQ.GetConsumingEnumerable())
{
using(StreamWriter fwriter=new StreamWriter("C:\\testsave\\dataProc",true))
{
for (int i = 0; i < datalen; i++)
{
fwriter.Write(outData[i]);
fwriter.Write(",");
}
fwriter.Write("\n");
}
}
现在,问题可能与原始帖子中的有所不同。 也许是因为生产者在这里也将数据写入文件?
原版的:
它有很多解释,但是解决方案很简单,只需将FrameStruct的初始化添加到第二个“ if”语句中:
rxFrame = new FrameStruct((int)m_lMaxFrame);
如保存的数据文件中所示:在损坏的文件中,每当您缺少值(例如230)时,您就会重复其他值(例如231)。 缺失值的计数等于重复值的计数。
这样做的原因是您将对同一对象实例的引用添加到队列中。 让我们看一下以下情况:RxThread循环N次,在上下文切换到dataProcess线程之前,它将N个引用添加到队列的同一FrameStruct实例。 在这种情况下,数据将是上下文切换之前最后一次读取循环迭代中的数据。 现在发生了上下文切换:dataProcess在上下文切换回RxThread之前循环M <N次,因此它从队列中读取M个元素,但所有元素都指向同一实例,因此它将M行同一行写入M元素。文件(如前所述)
现在,为什么Thread.Sleep有帮助。 简短的答案:每次RxThread将1个元素添加到队列中时,上下文切换到dataProcess线程的可能性都很高。 所以它实际上是:读一个->上下文切换->写一个...然后再次做同样的事情。
长答案是:dataProcess线程执行Monitor.Wait之后,它进入等待队列,上下文切换调度RxThread。 现在,线程将第一个元素添加到队列中,然后执行Monitor.Pulse。 这会将dataProcess线程移至就绪队列。 但不必安排立即运行,因此RxThread可以进行另一次迭代。 但是,如果您执行Thread.Sleep-现在很有可能会安排上下文切换和ataProcess线程。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.