It's actually the old pvc piping you need to worry about, made before 1977. They can release vinyl chloride monomer (VCM), VCM is a known carcinogen, causing cancerous tumors in the brain, lungs, liver and various tissues in humans. Since then improvements have been made and pvc pipes are used for most people's drinking water, and as Megan mentioned, in fish tanks. There are different types of pvc piping however, only the ones made for drinking water are safe. PVC pipe marked “NSF-61” or “NSF-PW” indicate that they have passed testing for potentially harmful chemicals leaching into the water. That said if the pipe is chewed up into powder or dust (unlikely but possible I guess), and inhaled or swallowed it can be a problem, so just like fleece if they chew it it has to go. A little bit of chewing isn't going to kill them though, unlike other plastic it doesn't break off in pieces when chewed, more like tiny shavings, so pieces that are swallowed will most likely pass. But as I said if they do chew it take it away, or cover it.
.
