How best to break the script?

Hello,
There is a php script that runs through the console phantomjs, you need to parse the page every few minutes. Pages ~500 on every page for 2 seconds, how best to break the script?
Here's an idea:
1) Divided into several parties, and to run consecutively, at the time the same
2) to Divide into parties, but to run multiple jobs.
3) Use some kind of handler, enter the Url in the database/memory/file, and 1 single handle
July 9th 19 at 13:59
3 answers
July 9th 19 at 14:01
Without specifics kosaemo of all, you can only guess what you want. Want, to do one job and do it or take a few, depends on the implementation of the script, the capabilities of the scheduler and your capabilities.
And what need specifics?
I need to parse the results of Google, every half hour. 15 different cities, a few different queries, the first 5 pages. ~500 pages is impossible. Now run it all in one script, each page is processed in 1.5-2 seconds = ~12 minutes, I wonder how best to implement that is not loaded with the server, in order to increase the number of processed pages to 5000. - kaylie_Casper commented on July 9th 19 at 14:04
if your goal is not to load the server, then you need a script to cut and optimize. If you owe someone $100, because no matter will you give them dollar bills or one - the result is the same. - aniya commented on July 9th 19 at 14:07
July 9th 19 at 14:03
Run 20 handlers via supervisor (to be a little more docente), for each select a list of URLs, sobsno all. P - Parallelization
July 9th 19 at 14:05

Find more questions by tags PHPPhantomJS