How best to run time-consuming operations in the backend flask?
I have a website in flask, the only task is to receive a request from the user to return him some visualized data. In the process, the script is drawn using urlopen to various api and loads the data, what is the problem, because you may need treatment to several thousands of addresses in a loop. Processing 300 is on a home computer about 7 minutes, and of course, the website waits for a response all this time. For user convenience, I can transfer the processing status to the client with websocket, but:
Is it possible to somehow speed up this operation? For example, using multithreaded programming? I used this area did not face, and I tips, or at least directions where to dig.
2 answers
To force the user to wait 7 minutes you can't. Therefore, it is necessary to send request to some page of expectations, where will Aaham kicking the status of the work that you will put in place, and, upon successful completion, forward it to the page with the results.
To accelerate, you can use aiohttp, eventlet or write your own simple functions for threading.Thread.
Still needs to tie some kind of message broker (RabbitMQ is reliable, fast Redis) to store and transmit these same "work" with the necessary parameters.
Find more questions by tags FlaskmultithreadingPython