I always take mine on the dive. I've often got multiple bailout tanks, scooter, dive in current, etc. and like to protect the lens on long deco stops, etc.
When I'm taking photos, I clip off the cap either to the housing (I've added some wheel weights to make it negative) or a D ring on my harness. Never been an issue.
Similar to @Nemrod, I've attached a bolt snap to mine and I've also drilled holes in it to facilitate equalization. I would sometimes find that I would get to the bottom and the lens cap would be suctioned onto the lens and a bit hard to remove, especially with dry gloves.