[英]improve temporary data store
I need your idead to improve our implementation. 我需要您的想法来改善我们的实施。
we have about 10MB
data need to be processed in our application for each analysis
. 我们需要为
each analysis
在应用程序中处理大约10MB
数据。 that data is transferred from another application packet by packet. 该数据是从另一个应用程序的数据包中逐个传输的。 there are about
24000 pakcets
for each analysis
because the data is created packet by packet in that application and then sent to us immediately. each analysis
大约需要24000 pakcets
,因为数据是在该应用程序中逐包创建的,然后立即发送给我们。
our current implementation is like the following: 我们当前的实现如下:
communication module
receives data and puts data into an object (an instance of a data class) when a packet is received. communication module
接收到数据包时, communication module
接收数据并将数据放入对象(数据类的实例)中。 Then it will fire an event
which will contain that data object
. 然后它将
fire an event
contain that data object
fire an event
。
data process module
receives event and puts the received data object into an arraylist
. data process module
接收事件并将接收到的数据对象放入arraylist
。
after all data for one analysis are received, data process module
will start data process to analyze the data. 接收到一个分析的所有数据后,
data process module
将启动数据处理以分析数据。 At the end the data will be stored to database and those data objects will be cleanup for next analysis. 最后,数据将存储到数据库中,这些数据对象将被清理以进行下一步分析。
time between each analysis
is more than 10 minutes. each analysis
之间的时间超过10分钟。
I have removed the clone or copy operation during the internal data transfer process. 我已在内部数据传输过程中删除了克隆或复制操作。 So I think we are actually passing reference around.
所以我认为我们实际上是在传递参考。 right?
对? I have also created a pool for those events and data objects.
我还为这些事件和数据对象创建了一个池。 so those events and data objects will be reused again and again.
因此这些事件和数据对象将一次又一次地被重用。
Another suggestion is create a shared reusable memory space
(internal share, between modules) for those analysis data and pass its reference around with the event. 另一个建议是为那些分析数据创建一个
shared reusable memory space
(模块之间的内部共享),并将其引用随事件一起传递。 I like this idea better but more code change from me. 我比较喜欢这个主意,但是我需要更改更多代码。 What I did now are almost the same in terms of memory usage.
就内存使用而言,我现在所做的几乎是相同的。 am I right?
我对吗?
What's your opinion? 你怎么看? any other better ideas?
还有其他更好的主意吗? for the
shared reusable memory space
solution, what's the correct way to implement it? 对于
shared reusable memory space
解决方案,实现它的正确方法是什么?
thanks, 谢谢,
You can make it thread safe by doing something like this. 您可以通过执行以下操作使其线程安全。
public sealed class SafeBytes
{
private readonly List<byte> _data;
private readonly object _lock;
public SafeBytes()
{
_data = new List<byte>();
_lock = new object();
}
public void Add(byte[] data)
{
lock (_lock)
{
_data.AddRange(data);
}
}
public byte[] ToArray()
{
lock (_lock)
{
return _data.ToArray();
}
}
}
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.