跨程序/线程共享变量的最佳方法？

Question

I currently have a program that runs an algorithm that processes real-time data, and once an hour a function is run that optimizes the parameters of the algorithm based on the new historical data.

Since this is running in a single thread, when the optimizer is running it pauses the real-time-data-processing algorithm, but it only takes about a minute so it's fine.

However, I want to greatly expand the optimizer and it's going to really increase the runtime. So I'm thinking of separating it into it's own program. Then it'll run in its own thread and won't interrupt execution of the algorithm.

To share the results of the optimizer with the algorithm, I guess I'll save them to a file each time the optimizer is run, and then the algorithm can periodically read the file.

How would I ensure that the algorithm doesn't attempt to read this file at the same time as the optimizer is writing to it? Does the OS file system already not allow that?

Also, is there a better way to share variables across different programs? Or maybe it would be better to split the optimizer off into a separate thread within the same program? Any suggestions would be appreciated, thanks!

Answer 1

first of all to share variables between programs you have to consider using named pipes.

If you want to use anything other than a named pipe, maybe just an ordinary file you have to handle race condition or reading wrong data from the file, you have to implement a lock mechanism, maybe a lockfile, if you are about to write to the file, you create a lock file, the routine that reads the file will check for the existence of the lock file, if the lock file exists it won't read from it, when the routine that writes to the file is done, you can remove it.

You can also use the cluster module , you can easily communicate between two process ( child and parent ) without creating a file.