around this issue From your description, it looks like OMG's Data Distribution Service could be a good fit. It is related to message queueing technologies, but I would rather call it a distributed data management infrastructure. It is completely distributed and supports advanced features that give you a lot of control over how the data is distributed, by means of a rich set of Quality of Service settings. Not knowing much about your problem, I could guess what an approach might be. DDS is about distributing the state of strongly-typed data-items, as structures with typed attributes. You could create a data-type describing the state of an automaton. One of its attributes could be an ID uniquely identifying the automaton in the system. If possible, that would be assigned according to a scheme such that every automaton knows what the ID's of its neighbors are (if they are present). Each automaton would publish its state as needed, resulting in a distributed data-space containing the current state of all automatons. DDS supports so-called partitioning of that data-space. If you took advantage of that, then each of the nodes in your machine would be responsible for a well-defined subset of all automatons. Communication over the wire would only happen for those automatons neighboring a different partition. Since automatons know the ID's of their neighbors, they would be able to query the data-space for the states of the automatons it's interested in.
Are single producer/consumer Tornado queues thread-safe?
this will help Use chain_future. Chain queue.get's future with some other future that is in your control (some kind of indicator/flag), then simply resolve flag future on close:
from tornado.concurrent import Future, chain_future
# of course you can create that in get as well
self.close_indicator = Future()
queue = self.getQueue() # returns a tornado.queues.Queue
qget_future = queue.get()
_ = yield qget_future
# set close indicator by resolving future
# result will be copied to chained future (qget_future)
Creation of Multiple Message Queues, with Unique Keys for the queues
it should still fix some issue We create multiple queues for message passing from one process to other. But sometimes the keys of the message queues get overlapped ( same keys for separate queues). In order to avoid this I thought of using IPC_PRIVATE with the function msgget() which is used in the creation of Message queues. , Please tell me the syntax for msgget with IPC_PRIVATE
int flags = /* ... permission bits ... */;
int my_mq = msgget(IPC_PRIVATE, flags);
WebSphere MQ/MQSeries - Possible to send a message to multiple queues with single call?