I have had it just come off with just water. The paint is old and can be delicate but it is really cheap and very easy to repaint them.
Thanks to someones advice I was able to repaint my adjustable numbers with a black extra fine oil based sharpie marker like this one, worked so well. and looks amazing.
Baking soda is pretty gentle but if the paint loose then it will come off. If you've used this razor a few time and the paint survived a few cycles of hot water under the running tap it should be alright. If not you'll have to repaint it. Good luck.