Mould on coffee grounds

I can’t wash coffee grounds down the sink, so I have to keep them aside till I dispose of them.
I try to let them dry, but a white mold forms on them.
Am I breathing in some dangerous stuff with this sitting around?
Thanks.