You want actual Popsicle sticks not "craft sticks". If the package says "Food Safe" of "Food Grade" on it they are fine, if not there is no guarantee. If you got them in the food section of the craft store (where you find the cake and candy molds, and decorations stuff) then yes they are safe. The ones in the craft section sometimes have stuff on them or may be made from recycled wood or mystery wood that might not be safe.
I just buy them at the grocery store that way I know for sure they are food safe since they are sold to make food with.
Just like to add that they can and do splinter much more then other wood because they are so thin. Most chins are fine with them, I've been given them to my chins as toy parts for years, and may people sell them as part of toys, but it is something to keep in mind. I know some people don't like to use them for fear of a mouth injury.