How to use the dataset in jQuery?

Good afternoon. I have a question, how can I remove the blocks through a dataset. Below each block was removed separately with a click. I have is slightly different. The first block removes the button, but the second is not. All two blocks are removed.Please tell me

<div class="cart" data-product-id="1">
<span class="remove"><a href="#"><i class="fa fas-ban"></i></a></span>`
</div>
<div class="cart" data-product-id="2">
<span class="remove"><a href="#"><i class="fa fas-ban"></i></a></span>
</div>

$('.remove').on('click', function (){
$('[data-product-id="1"]').remove();
 $('[data-product-id="2"]').remove(); 
 });