In RabbitMQ, or without, customize the classic line?

Hi!
For a long time thinking over the architecture of one project, but was faced with technical issues in the implementation plan.

Description: there is a 100k client computers at, say, 50 of Internet clubs around the country. After each session working on these computers, they must connect to send the report with the main server No. 2, which in turn connects with the main server # 1 to pass the report received from the client, get the response and pass back to server number 2, and the-client. The time client receives a response from the server number 2 is 10 seconds, if no response received -- connection is broken.

Also there are 2 kinds of operation: online and offline.
Online: the classic queues, each report is handled according to the date. The client began and ended the session -> open the server connection No. 2 to send the report with waiting 10 seconds> server # 2 received and recorded in the queue -> from the queue is taken and reports are sent to the server # 1 for verification and entry -> server number 1 receives and sends the response to server number 2 -> server number 2 sends the response of successful completion of the operation to the client, which waits for 10 seconds.

Offline: the case is that the server # 1 can fall off at any moment. And then the client receives no response within 10 seconds, he says that he is now working in offline mode. But still continues to send reports to server No. 2. Server No. 2 also takes records, but records them in place-offline, which should be processed as soon as server number 1. And other reports that come from the client, should expect. Because the server is No. 1 takes consistently.

Question: how to organize this architecture and on what?
Until I came to use RabbitMQ and microservices. But I think that it would be bad manners for each client (100k computers) to create a turn, and will be two stages: online and offline. Online will not be processed until offline turn. But for each computer and everything should be processed sequentially by date: it turns out this "classic" Soviet line. But in parallel with multiple computers: 100k computers processed in parallel, but the reports from each of them -- consistently. On the basis of the calculations: 100k customers and for each 2 turns = 100 * 2 = 200 queues 2 queues on every computer.

How can you do? Ideas, tips?
June 5th 19 at 21:53
2 answers
June 5th 19 at 21:55
from the queue are taken and reports are sent to the server No. 1 to check and record

Yeah, this is the place where is decided, offline server # 1 or online.

Because the server is No. 1 takes consistently

And this is the main requirement.

Option number 1.
A common place one worker. If the server No. 1 is offline, the worker just stops running as long as the server returns. For the current message it is necessary to perform amqp reject, then it will return to the beginning of the queue.
The order of messages observed, but there is a single point of failure, it is also the bottleneck of performance (however, depends on the speed of report processing)

Option # 2.
A common place, a lot of workerb. The scheme is the same, the server is offline, the processing stops. Online renewal. The bottleneck in performance and point of failure away, but two consecutive report from one of the computers can get simultaneously on adjacent worker, i.e., the requirement about "consistently" may not work.

Option # 3.
HashRing and other hashing algorithms that allow to reduce the number of queues and each queue to hang your worker. The need for manual load balancing, no single point of failure on a separate line.

Something like that. Immediately figure out how to get some workarou to process a single turn, preserving the order of messages, I figured it out.
Option #2 seems more optimal. Any idea how to make the connection with every client is not lost while waiting for a response from the server number 2? Well, consider the sequence from each client, but multiple clients in parallel. - Emanuel_Ritchie commented on June 5th 19 at 21:58
And the server number 2 there generally is not involved. His task is to accept the request from the computer and put in a queue in RabbitMQ, I understand. - addie_Ziema commented on June 5th 19 at 22:01
server number 2 accepts the connection from the client keeps the connection open 10 seconds until report received from the client is handled by the server number 1, and then server # 2 need to send OK to the client and close the connection. Have a look here: How to RabbitMQ or without, to set the classic line?
Can't come to a standard solution, without loss of consistency and performance. - Emanuel_Ritchie commented on June 5th 19 at 22:04
looked at 500 messages per minute with 100K clients with the cluster of rabbits, which the authors boast on the Internet, manages end-to-end.
Still rabbit is not necessary to load the storage period offline work from non-empty queues he starts to slow down significantly, and a long offline server # 1 will lead to OutOfMemory.

I would take a hard look at the stream data processing, a La Kafka, but he is close to them do not know. Try to reset the question to the experts on this topic. - addie_Ziema commented on June 5th 19 at 22:07
June 5th 19 at 21:57
It is not clear why turn on each computer? In fact, the queue is only needed on PC # 2. All clients send him a report, the reports are placed in the queue, the PC # 1 takes them from the queue. And turn and the need to be independent, the network is the computer No. 1. All messages will be queued until the first doesn't take them. If at any time the message is not taken, the client is given the answer that the first is not available and the client using a timeout sends another message.

Each client has an ID, each report is its own identifier, and if the queue were several messages from the same client with the same report ID, it takes into account the last by date. Well basically and all.
In order not to slow down other customers. Look, we have 100k customers, and each customer should be processed in parallel, so they are not forced to wait for each other, but the reports from a single client must be processed sequentially. Actually why there are two queues for each client. XS somehow. How is it to be? - Emanuel_Ritchie commented on June 5th 19 at 22:00
But Yes, if the server number 1 is broken, then, in fact, all clients will go offline. - addie_Ziema commented on June 5th 19 at 22:03
How often to send reports with the client? Even if every hour. And under it to allocate a separate place with his handler? Unless of course they will send every minute and it's all in real-time to process, then I agree. But You gave the example of the computer club, from this and repel.
But if the mind, that rebit should handle 100k messages per hour in the same queue, and process these messages concurrently. But, of course, if reports are not MB.
You're the first the computer took a bunch of messages, say once a minute, he takes the pile. In this stack You can all messages to sort and group as you wish: the client, identificatio report on time. And then treat clients in parallel, the reports consistently.
It is certainly possible for ease of processing to make a turn under areas or clubs but the essence does not change. - Emanuel_Ritchie commented on June 5th 19 at 22:06
with each client the reports will come in about 200-500 per minute. And you need to handle multiple clients in parallel, and the reports from them-consistently. Can't come to the point where everything works fine, even if to divide them into smaller microservices. Because the service server No. 1 -- one of the entire continent. - addie_Ziema commented on June 5th 19 at 22:09
Yes , then you need for each turn. But as far as I remember, rabbit guarantees delivery but does not guarantee the order of delivery. So we need to be more samaraditya buffer with sorting by date, once the desired processing sequence.

And why a separate queue for offline?

I understand that report processing can only comp No. 1, and the second only collect them from customers. And if the first off, nothing can be done, everything will be offline. But that will not be lost and in the end, their first taken away by anyone. And all the parallel processing on the first computer.

To retrieve messages it is only necessary in batches, instead of one process to always understand the report queue from one client. - Emanuel_Ritchie commented on June 5th 19 at 22:12
in fact the problem is that we don't have access to the server No. 1, and sends a message to our server number 2. These your API, and we adapt to them. - Emanuel_Ritchie commented on June 5th 19 at 22:15

Find more questions by tags .NETDesigning softwareC#Message queuesRabbitMQ