Whether in the background tornado run method?

Good afternoon.

There is a task for a get request to get data from the database. But if the database has not the desired value, then give the answer self.write'have, the user will receive the downloaded page, and this time run the background parser that will get the desired values and write them to the database. And when you re-query the answer data will be recorded.

I look towards Asynchronous requests? Type:

from tornado.httpclient import AsyncHTTPClient

def asynchronous_fetch(url, callback):
 http_client = AsyncHTTPClient()
 def handle_response(response):
callback(response.body)
 http_client.fetch(url, callback=handle_response)


And yet, here we have threads to connect?
July 9th 19 at 13:02
1 answer
July 9th 19 at 13:04
If we proceed from the assumption that the missing data are not rare, and parsing with and add to the database takes tangible time, and the user can wait, I would probably do the following.
1) had a persistent job queue, for example, using Redis or would existing base.
2) through the mechanism PeriodicCallback periodically checks this queue and ...
3) start parsing using Subprocess or joblib

You can do it all and inside the Tornado, adding tasks in IOLoop - but there is a risk that at some point the server will be busy mainly with parsing, not processing requests from the frontend.
And if all the same to pass on to the tornado, giving it a more server resources, right?) - Jermey.Legr commented on July 9th 19 at 13:07
Resources is gigahertz, cores and memory? Tornado is running on a single core and if multiple cores can be use multiprocessing or wrapper on it - joblib.
If the kernel is allowed to use strictly only one and parsing will actually need 2-3 times a day - city asynchronous coroutine, do everything "inside". - Kathryn5 commented on July 9th 19 at 13:10

Find more questions by tags Asynchronous programmingPythonTornado