Highly loaded system, what are the design principles?

Hi all, what are the principles of high load systems?

And here's some books cite the use of cgi-scripts in python, using python from php scripts, etc. whether it is Safe in General in terms of stress situations when users very much?

How to design network services in the socket in cases of high load? How?

What languages are best to use for this? What not to use?

Thank you
July 8th 19 at 11:35
4 answers
July 8th 19 at 11:37
let so, there are two types of problems:

- CPU bound - the various algorithms, math, coding/decoding/encryption... in short, all that CPU.
- I/O bound - in fact when we have a lot of operations with I/o, where 90% of the tasks related to WEB and server development.

For CPU bound should use languages like C, Rust, Dlang, Go, etc. Word languages which are compiled into efficient machine code.

For I/O bound - Go, NodeJS, Erlang, Java.... Yes, in principle it does not matter what language, the main thing that used non-blocking calls and there was no lock.

There is still a queue of tasks, scaling out, etc. the Architecture and algorithms used in the system impact is often much stronger than the programming languages.

What languages are best to use for this? What not to use?

No restrictions, just common sense. That is unlikely we are going to write complex math to node.js but problems related a lot less. Also, no one says that the system needs to write strictly in one language. Now it is fashionable to use microservices, each of which can be implemented in your language and to its database, which are ideal for specific tasks.

Do not forget about the algorithms. They too must be optimal. For example, take a simple task - clustering placemarks on the map. Imagine that you have database of millions of objects that we have displayed on the map. Because to do this on the client would be problematic - we have to do it on the server and return to the client as much data as he needs.

And at these volumes, even if we took si, if our algorithm has a complexity of O(N^2) as if there is nothing particularly to do. And and so slowly. But if we take any algorithms with complexity O(NLogN) it is already possible that this algorithm can be at least in php/python/ruby to implementit. So for example I have this algorithm implemented in Java and not in the most efficient manner. Cope.

Improve the speed of development (any ruby/python/node in this plan is a good one), the cost of support (C support much more expensive than Go for example, while it is always possible to write so bad that it is easier to throw than to support), the cost of the developer.... For example to find a cheap strong developers on the Go or Rust will be very problematic.

Also do not forget that the server now are not so expensive. Sometimes in business it is easier to pay for a dozen servers than to write all the pros.

Actually the main rule of high-loaded systems - load testing and then optimization
July 8th 19 at 11:39
About cgi, python, php, apache should be immediately forgotten. It is better to forget about http and websocket. And look at C/C++ node.js Go, in-memory databases, TCP sockets.
Thank you, in-memory database is something like SQLite? I heard somewhere that the DBMS contrary to the best

And TCP sockets to use multi-threading or asynchronous sockets? - Emiliano_Becker commented on July 8th 19 at 11:42
Architecture is more important than the algorithms and languages, as has already been said, but more importantly is to build good data structures and to choose a good Protocol. From the data structures on algorithms, which will be processed, with poor structures, you cannot write a good algorithm. And protocols depends on the server performance and the speed of data exchange between system components (e.g., between the DBMS and the server, the server and client on two different servers or services). Here HTTP and other technologies of the old web as not fit for heavy tasks. It often makes sense to choose a language and platform that has a good memory Manager and is able to keep in memory and efficiently handle the large data structures, there are languages to query in-memory data, meaning that the DBMS does not pull, the drive does not pull and do not make any I/O, in addition to working with the client over the network. - rosemarie.Balistre commented on July 8th 19 at 11:45
: what can you say about HTTP2? - Allene_Crona78 commented on July 8th 19 at 11:48
will say that this is a very complicated solution to a simple problem, take TCP + JSON and get a thousand times more flexible solution, bi-directional connection-oriented and guaranteed delivery transport, to which is appended an hour reconnection, disconnection, remote procedure calls and translate events. - rosemarie.Balistre commented on July 8th 19 at 11:51
tcp + messagepack you want to say?) why waste time on deserialization of data. - Allene_Crona78 commented on July 8th 19 at 11:54
in General, it makes sense to go for things like zeromq (not the bare socket to use, right?) only when http is the bottleneck, because debajit and support easier. How many teams got burned from these premature optimizations. - Allene_Crona78 commented on July 8th 19 at 11:57
I used ZMQ for two years, but in the end and got rid of it and now on the bare TCP chase not even JSON, and JavaScript. To parse it and make V8. HTTP always the bottleneck, it is so poorly designed that a go socty raises productivity even at times, and several orders of magnitude. Service HTTP much more overhead than writing your Protocol in more than 10 languages, which we did. - rosemarie.Balistre commented on July 8th 19 at 12:00

> not even JSON, and JavaScript

what was that sorry? Type with native extensions structure chase raw? Well, it's at the root kills the ability to theimplementing another part of the system on something other than js. And I don't think this method is more efficient messagepack or thrift. - Allene_Crona78 commented on July 8th 19 at 12:03
Not the native structure and syntax of js. more than 10 implementations: a few js and node, some c/c++, some java, objective-c, swift, c#, python, haskell, php, go. Well, with the ability to both drive objects/hashes and arrays to drive that on the remote side using pre-prepared prototype of cling getters and setters to access the fields by name. - rosemarie.Balistre commented on July 8th 19 at 12:06
: again, what does "syntax. JS" in the context of data structures? Chase data in binary or text form? If the first - what is different from messagepack/thrift? If the second - that it's json, no? - Allene_Crona78 commented on July 8th 19 at 12:09
Yes, in JS syntax, a text. JSON is not JS, on JS it is possible to describe a lot of things in the JSON it is impossible or not convenient. For example, in the JSON [null,null,null,100], and in JS it's [,,100] and you can write expressions, functions, designers, anything and absolutely safe to parse them in v8 sandboxes:
 name: ['Marcus', 'Aurelius'].join(' '),
 passport: 'AE' + '127095',
 birth: {
 date: new Date('1990-02-15'),
 place: 'Rome'
 age: function() {
 var difference = new Date() - birth.date;
 return Math.floor(difference / 31536000000);

Of course, usually all this does not apply, and instead, position transfer, i.e., the arrays that the assignment of the prototype are transformed into objects ['Ukraine','Kiev','03056','Pobedy','37',1,158] - rosemarie.Balistre commented on July 8th 19 at 12:12
: somehow I'm confused about the 'Ukraine' and the 'Pobedy', is ocresolate and got to the issue with high-load systems... - Emiliano_Becker commented on July 8th 19 at 12:15
: The address of the National order of Lenin Technical University of Ukraine named after 50 th anniversary of the great October Socialist Revolution Kiev Polytechnic Institute of Emperor Alexander II - rosemarie.Balistre commented on July 8th 19 at 12:18
: and, uh, hmm. Well, the name. - Emiliano_Becker commented on July 8th 19 at 12:21
in short under-json. Clear. It is not clear just why. Smacks of bad architectural decisions and castelani. - Allene_Crona78 commented on July 8th 19 at 12:24
is JSON an under-JavaScript, and JavaScript parsers are everywhere, in any system, and it is possible to use V8 as the parser, we do and it works quickly, with all the optimizations, much faster JSON. - rosemarie.Balistre commented on July 8th 19 at 12:27
: you mean binding to use v8 as the parser? Try messagepack, I think it will be easier and faster. - Allene_Crona78 commented on July 8th 19 at 12:30
Yes , if the system is already built v8, then why do something else, it makes for hidden classes that parse, and JS is well supported everywhere, nothing special should not be written in Qt and iOS and in Android and in Java and C#, only the wrapper to do, and parsing it is not difficult, we have compared two dozen ways of parsing the different methods and on different platforms. Binary serialization will not let you win, if it's not the native format of the data in memory v8 or another engine. But the engine attached is not good. - rosemarie.Balistre commented on July 8th 19 at 12:33

> Binary serialization will not let you win, if it's not the native format of the data in memory v8 or another engine.

I somehow suspect that all overhead it's good to not less than the regular JSON parsing. But we need benchmarks. The whole idea is interesting and I'll give it a try. - Allene_Crona78 commented on July 8th 19 at 12:36
and I'll play around with messagepack, suddenly it will be faster zmq and protobuf.
When the SDK is not in beta, then lay the article on Habr. - rosemarie.Balistre commented on July 8th 19 at 12:39
July 8th 19 at 11:41
I will add that to improve performance you can use the microservices written in C/C++ and controlled by the Manager. And to contact the Manager is through a socket or through a database, giving the task.
And all pages form - to give PHP (for example). This will result in a synchronous system with a semi-automatic partial load on the CPU, which can be reduced to automatic by creating a balanced distribution rules of CPU load different modules in different situations.
July 8th 19 at 11:43

Find more questions by tags Network administrationHighloadSocketsDesigning software