Is it true that threads are limited to the number of cores and why run thousands of goroutines — bad?

I have some code that gets the list with a couple thousand URLS, each of which he goes and gets the right info, synchronous this code was working 5 minutes probably, and when I copied the address to each URL on goroutines and channels make the code to handle all this for 20 seconds. The profit is obvious to me.

But I got a rebuke from another person, in his opinion, the launch of such number of goroutines is a bad decision, explaining it by the fact that the number of threads depends on the number of CPU cores and Violetta each goroutine stack is equal to the count of cores, and the other type in the memory hang, spending her.

But I find it hard to believe about the number of cores and large memory overhead.
Can something about this to say?

My code by the way - https://github.com/d0kur0/webm-grabber
April 3rd 20 at 17:28
2 answers
April 3rd 20 at 17:30
Solution
The man is not right, Gosha normally can roll millions of goroutines. Goroutine is not a flow.
In your problem most of the time spend in the channel in a locked state, wait for a response from the external service, this is the intended use of goroutines, which you can think of. They are performed only in those microseconds when the service answers come.
The goroutine stack size is only 8 kilobytes of memory much they take will not be.
April 3rd 20 at 17:32
Solution
The statement is true for cpu-bound tasks. You have not that situation

However, to create goroutine on every request is clearly bad. Edinovlastno you can send millions of requests. Either put the service on the other side, or you get in ban. Not to mention the fact that there is a clear bereached memory.
You need to create a pool of goroutines with a finite number of them and feed them tasks via select. Example https://gobyexample.com/worker-pools
Code runs as a daemon that runs on a timer for the updated data. He knocks on different resources to their API, which I did not authorize this and limit the number of requests.
It can be folded in General.

The statement is true for cpu-bound tasks.

Do not quite understand: it is true that a lot of goroutines is bad. Or the fact that the number of threads is limited to cores? - Lenore_Rohan commented on April 3rd 20 at 17:35
@Lenore_Rohan, is it True that threads are limited to the number of cores - threads are limited by the number of cores. But if the tasks occupy all cpu time, meaning from spawn goroutines more than you have cores - no.

If you have a block of operacii, as waiting for the response from the server, they do not create burden on Prots. So you can create 100 goroutines for parallel processing, because they will no longer wait for a response than to process information. However, it is not necessary to generate an unlimited number of them.

In another response written
The goroutine stack size is only 8 kilobytes of memory much they take will not be.

This is true, but ö in addition to the stack itself goroutines, memory is required and what you have allocated pamyatyu . your code goroutines - Ayden.Beaha commented on April 3rd 20 at 17:38

Find more questions by tags Gomultithreading