Jeff
Sorry if I sounded harsh. Morning rush syndrome. Plus my lens physics vernacular is severely lacking sometimes. I understand and agree with your point about focal distant. I'll illustrate with Stephen's pics:

Here you can see he's got the focal plane covered from center to right which are roughly equidistant. Then the left side, is where he brings in real world conditions whereby subjects are not always on a flat plane parallel to the sensor plane (eg. in Doug's case, baitfish wrap around the camera etc).
I think his testing method, if not perfect, is sound in testing dome compatibility to a lens,. There are so many issues regarding lens testing such as how the lens performs on its own. Quite a few UWA lenses do well on the focal flat plane distortion wise but the near field can suffer. Also at 12mp, you may not see some things that you can see at 21-24mp.
Drew - your theory sounds correct, but in the case of this photo, the sailfish is closer to the lens than the diver. I suspect that at those wider apertures, nothing at the edge of the frame would be sharp no matter what distance it was at (although there would certainly be an optimal distance at which something might be "less unsharp".)
It would be interesting to set up some tests with a camera on tripod focused on a central target, and then peripheral targets set up at various distances forward of and behind the central target to see where the best focus would hit.
Doug, my wording was utterly wrong although the concept was somewhere there. The diver was at the edge of the frame farther out and his body was OOF as the wide aperture's thinner DoF just hit at that distance to the virtual image.
When will Wetpixel establish its official Underwater Testing Center?
Actually it's not cost effective for the consumer (the staff at wetpixel pay like the rest of you) to setup something like that. Should the onus of testing be on the manufacturer, dealer or the consumer? I know a few people who test and not publish their findings without coaxing.
The 2 industry standouts on Wetpixel at least are Stephen Frinks of SeaCam and Jean Bruneau of Aquatica. Jean occasionally publishes tests with different setups and results and is pretty much on hand to field customer support questions on Aquatica products. Then there's Stephen Frinks, who is probably the most prodigious tester dealer who publishes his findings for all, including non-customers.
The problem is compounded by the time needed to test properly and the availability of equipment. Can you imagine the cost of getting all the cameras and lenses? Backscatter and Reef Photo also publish a few tests on their own website. Then again I shouldn't lament the situation too much as I think it's the lack of facts that Wetpixel has become successful in providing on some issues. 