* How close could I get to my subject?
Really close. The port will get very close to the subject. The min focus distance is 8.75" and the lens is 2.9". About 5" from the end of the lens.
* How big can I make a very small subject appear?
A quarter will fill up most of your frame and get some of it chopped off the top and bottom.
* If I took a photo of a large subject at a specific distance, say, 10ft, how much of that subject would I capture? (in other words, when that big palegic swims by, am I going to get a decent photo?)
You can get the whole subject, but the photo will be far from decent. Ten feet is far away, as in no strobe light. You would have to open up the aperture to get even a remotely decent exposure. It will lack color and depth of field.
You can get very nice fish portraits from a reasonable distance, say 3-4ft. If the big guy comes by, you can still get a decent shot, just not a full body shot. I've photo'd many a barracuda with the 105mm lens, just not the entire fish.
Assume I use the 28-105mm f/3.5-4.5D Macro with a dome port and a +4 diopter close up lens.
* Using this lens, how close can I get to my subject for macro work? (with and without the +4 diopter)
The min focus distance is 1.7'. The repo rate is 1:2.74 w/o diopter, not very good, but still macro range on the D100. The diopter will get you closer. Remember, the dome will be several inches away from the end of the lens, which uses up a lot of the working space. Getting domes close to macro subjects can be problematic as can positioning your strobes for macro while using a dome port. I shoot the 17-35mm behind the dome. It is the equivalent of 52.5mm with a minimum focus distance of 12". So I can get the dome almost right up to the subject.
While you can do some macro, this setup you are talking about is far from optimal.
* How big can I make a very small subject appear? (with and without the +4 diopter)
Well, you can't make it bigger than it is. How big is big?
* With the lens zoomed out to 28mm, and I were to take a photo of a large subject at a specific distance, say, 10ft, how much of that subject would I capture?
Plenty, depending on how large the subject is.

This was taken at 35mm I believe, as was the next one.

"Is there some formula that I can apply to help me answer these types of questions for any lens?"
Of course, but I don't do math anymore. It gets in the way of good photography.
Define what it is you want to do. Choose the best tool for that job. If you have another job, get the best tool for that job. If you are only going to get one lens to start, get the 60mm and work on macro and fish portraits until you perfect it. If you can afford a second lens and need something more versatile for above water shooting, then get the 28-105mm or better yet the 24-85mm G lens or 18-36mm. You can any of those for wide angle work underwater. Not as wide as some would like, but pretty workable.
I believe a common mistake, especially new photographers, is to want to go too wide too soon. The result is often poor compositions that are too wide, or excessive cropping. (Cropping can not always cure poor framing) I spent years taking bad photos with the 20mm. As soon as I switched to the 24mm, the quality of the composition improved dramatically. Only then did I start moving wider again.