if you enable torrents, the rest of the Internet falls to almost zero
Torrent is multiple data transmission.
Regular download - single flow pumps with a single host. In the case of the torrent open a lot of network connections.
Your router may stupidly not withstand the load. And that's all.
Especially if you limit the bandwidth router and so cannot cope with the work, because he still rules shaping process is necessary.
May be the problem is not the router, and bad channel - if the channel is clogged with packages torrent other will not pass.