How best to run time-consuming operations in the backend flask?

I have a website in flask, the only task is to receive a request from the user to return him some visualized data. In the process, the script is drawn using urlopen to various api and loads the data, what is the problem, because you may need treatment to several thousands of addresses in a loop. Processing 300 is on a home computer about 7 minutes, and of course, the website waits for a response all this time. For user convenience, I can transfer the processing status to the client with websocket, but:

Is it possible to somehow speed up this operation? For example, using multithreaded programming? I used this area did not face, and I tips, or at least directions where to dig.
July 9th 19 at 12:57

Related questions

More answers about "How best to run time-consuming operations in the backend flask?"
2 answers
July 9th 19 at 12:59
Solution
Maybe look at Celery?
thanks, didn't know about this. Potestiruyu. - nikita.Stracke commented on July 9th 19 at 13:02
July 9th 19 at 13:01
To force the user to wait 7 minutes you can't. Therefore, it is necessary to send request to some page of expectations, where will Aaham kicking the status of the work that you will put in place, and, upon successful completion, forward it to the page with the results.

To accelerate, you can use aiohttp, eventlet or write your own simple functions for threading.Thread.

Still needs to tie some kind of message broker (RabbitMQ is reliable, fast Redis) to store and transmit these same "work" with the necessary parameters.
the idea is that it's all done on one page, not necessarily a few, it is because of Ajax. It's not too often will be necessary, because the cut-you are saved in the database and will then appear there, but any new requests will need to consider. About the implementation of the treding, or aiohttp more? Do I understand correctly that for each query I have to create a new thread? - nikita.Stracke commented on July 9th 19 at 13:04
:
the idea is that it's all done on one page, not necessarily a few, it is because of Ajax.

Then don't forget to change the URL in the address bar.
About the implementation of the treding, or aiohttp more?

It is possible to esteem docks. - nikita.Stracke commented on July 9th 19 at 13:07
: docks I read, but it is far from the practice. I want in addition to this write, it still worked. I just overclocked quite far, including not are, first of all as it is at least to design) So I'm asking, one thread in one long query urlopen is this normal? - glennie_Mer commented on July 9th 19 at 13:10
No, it is not normal. Better yet, rock some eventlet. Will not have to pile the bikes. - nikita.Stracke commented on July 9th 19 at 13:13

Find more questions by tags FlaskPythonmultithreading