It’s always good to keep the composition/ care labels attached in garments so that future wearers know what the fabric they are buying is and how to take care of the garment so they can keep wearing it longer.
One of the biggest hindrances to being able to recycle the cloth in garments that are no longer wearable is that they have to be sorted into textile groups. If the label has been removed it can be impossible to guess the composition which means the textiles can then only be re-used for something very low grade, like roofing felt. Whereas, if it is known that the garment is 100% cotton it can be remade into another garment. I have heard of one brand which has started using different coloured markings inside their garments to make it easier to sort. It would be great if this could become standard practice.
In the meantime try and keep the care labels attached and if you do remove keep and attach to the garment when you get rid of it.