The rubber can get scuffed (and see slight discoloration on some of the brighter colorways), as would any other stiffer polyurethane band on a Freestyle or G-Shock, but as for scratches and gashes, you'd have to get in a pretty gnarly garage sale to mess up the band.
The rubber is ridiculously comfortable - a more than amicable trade-off. Don't think - swoop one!