Fleece is the only chin safe (or as close as you can get to chin safe) fabric since it does not shred or fray like other fabric if chewed on a little. However even with fleece if they chew it it needs to be removed. A small amount if swallowed normally passes without a problem, so you can remove it before too much is eaten. Also even if it's a fleece shirt it would also pose a problem of the thread holding it together being accessible and swallowed, thread can cut right through guts. That being said though, chins aren't like cats and dogs in terms of smell, since he is new to you your smell wouldn't be comforting yet.
The best thing to do to make him feel safe would be letting him be, just sit near watching, and talking to him. Also, if you haven't already, get a hidy house, those help a lot by giving him somewhere to hide. It takes most chins at least a week to get use to a new home.