Just rinsing the o-ring in clean water while running it through your (clean) fingers should usually be enough to remove any dirt or old grease. Some o-ring greases (i.e., Ikelite's) are "stickier" than others, but you will be able to feel whether there is any grit or residue that is not coming off easily. When you are cleaning the O-ring, you are also inspecting it (visually and tactile-ly) for any racks, splits, or other problems.
After washing, run the o-ring through a clean lintless cloth - 3M's ScotchBrite "High Permance Cloths" are widely available in supermarkets, and cost a fraction of what essentially the same piece of cloth sold in some high-end photographic and electronic equipment stores would.
If something more than just water is needed, I would go with a mild detergent (such as J&J Baby Shampoo), properly diluted, rather than soap solution, which has a high ph.
Cleaning o-rings is a very basic and important issue w/ any u/w photo gear. This question probably should have been posed in the "Beginners' Forum" rather than "Digital SLRs/Housings".
Frogfish