Weird. When I use it, about 3 x week, I leave it on for a few minutes and then rinse with cold water. It's the first thing I apply after a shave. I find creams and balms are less creamy if I put them on directly over a layer of alum.

First time with my new alum block last night...

I had a decently close SE shave (I normally stick to DE, but what are rotations for?) and rinsed with warm water.
My face had a few tingly spots where I might have scraped a bit with the razor so I was apprehensive about how the alum would feel.
I was expecting the proverbial "salt in the wound" sensation... but it was actually just a mild sting here and there and a very very slight tightening of the skin.
After a minute or two, I rinsed thoroughly with cold water and dried my face before applying aftershave.

So far, so good. I think I'll make it a part of my routine for a few weeks and see how it works out.