Yes, its totally okay to talk to a girl casually at the gym. Just dont interrupt her while shes working out and dont follow her around or anything like that. However, you dont go up to her randomly and try to force a conversation. Its not going to work, and its kind of a creepy and rude thing to do.

Is it creepy to approach a girl in the gym?

Sure theres a difference between the casual gym creeper who goes with the intent of picking up women and those who are there to train seriously, but you can usually spot whos who by the difference in their physiques. That said, approaching women in the gym *can* be a delicate situation.