Quality of WA zoom lens for UW use

Hi,

I plan to acquire a WA setup in 2017 (camera/lens/housing; not tomorrow, not next week, but within the next few monthes ). I am still not shure wich sensor platform (MFT, APS-C, VF) to choose, but in the meantime I look which lenses are popular/available, the sensor platform I choose will also depend on this...

When one compares three popular UW WA lenses using DXO benchmark:

and looks at "sharpness" (to my opinion a very important criterion for a lens), the comparison is as follows:

Panasonic 7-14mm (MFT):

at f8 - f11 the sharpness in "perceptual Mpix" is between 3 to 6 Mpix. Only at larger apertures sharpness increases to >10 Mpix at shorter focus length.

Sony SEL 10-18mm (APS-C):

at f8 - f11 the sharpness in "perceptual Mpix" is also between 3 to 6 Mpix. Sharpness does not improve with larger apertures, hence this lense is the worst.

Zeiss Variotessar 16-35mm (VF):

>12 Mpix, almost throughout.

My question: Does someone have direct experience and can compare these lenses' performance UW?

Is there indeed such a hughe difference in image sharpness between these lenses (is this DXO benchmark misleading?)?

Does someone know about optical testing of these lenses other than by DXO?

Wolfgang

P.S.: the Sony 7-14mm was not (yet?) tested by DXO...

 

 

The DxOMark score for Sharpness is based on the Perceptual Megapixel (P-Mpix) concept that weights the Modulation Transfer Function (MTF) of the lens with the human visual acuity. Read more about Perceptual Megapixels.

https://www.dxomark.com/Reviews/Looking-for-new-photo-gear-DxOMark-s-Perceptual-Megapixel-can-help-you

 

 

A number of factors can cause this loss in megapixels and resolution, including such lens defects as optical aberrations, light diffraction, or an ineffective anti-aliasing filter. The difference in number between a sensor’s megapixels and Perceptual MPix quantifies this loss.

You will find that no lens are perfect - it's quite the opposite.

Also the sharpness is not constand over the frame. Things allways gets worse at the edges. So a simplified averate rating doen't tell the whole picture.

Thus these sharpness figures are perhaps not the best figure to use for evaluating a system performance.

Generally 35 mm frame cameras and large optics are better. They are simply bigger and easier to make.

Still far from perfect. Find a few 36 - 50 MP FF cameras on DXO mark and check the "sharpness" factor with several lenses.

The camera might be 36 MP, DXO mark will say that lens XXX is only 10 to 15 MP "sharp". ;-)

Go figure.

Hi EspenB,

Thanks for the information. Looks to me that the advantage of VF is more the better optical quality of bigger lenses and, to a lesser extend, the higher sensor sensitivity...

And yes, I believe that sharpness is a very important criterion, as others, like distortion, are small and do not really spoil the image in most cases...

The perceptual Mpixels sharpness, by the way, is not average, but maximum (usually at center). The performance in all regions of the image can be seen under "measurement/sharpness/Field map" in % of maximum at: https://www.dxomark.com/Lenses/Compare/Side-by-side/Sony-E-10-18mm-F4-on-Sony-A6000-versus-Panasonic-LUMIX-G-Vario-7-14mm-F4-ASPH-on-Olympus-OM-D-E-M5-versus-Sony-FE-Carl-Zeiss-Vario-Tessar-T-STAR-16-35mm-F4-ZA-OSS-on-Sony-A7R-II__1084_942_536_793_1467_1035

At fully open aperture and in the periphery of the image, the Sony 10-18mm has <50% of those 3 - 6 pMpix only, much worse than both the MTF and the VF WA lenses, that perform substantially better in this regard...

=> as first conclusion I would say the (rather small) improvement of APS-C over MFT, regarding sensor sensitivity and dynamics is more than compensated by the worse optical quality of the WA lenses compared (the Sony 10-18 mm seems to be very popular and some kind of standard for e.g. Sony A6500 cameras; I see no candidate for this camera that would perform better).

=> the second conclusion is that VF with 24 Mpix sensor & >12 pMpix throughout is better than MFT with 16-20Mpix sensor & 3 - 10 pMpix, depending on focal lebgth and aperture. In addition VF sensor is more sensitive and provies better dynamics.

=> So the crucial question remains: does this "benchmark" advantage of VF over MFT justify to carry around the extra volume and weigth in equiment, when compared to MTF, when real life IQ is compared???

(But 3 pMpix with 20 Mpix sensor does NOT sound like excellent IQ to me).

Wolfgang

Hi EspenB,

 

(But 3 pMpix with 20 Mpix sensor does NOT sound like excellent IQ to me).

 

Wolfgang

Yeah, I have also been puzzled by the DXO results measuring a lens to 3-4-5 MP on a 16 MP 4/3 sensor...

Can't say that I understand the real world implications. If taken figuratively all M4/3 optics are basically crap!

The best lens is the Olympus 75 mm f1.8

https://www.dxomark.com/Lenses/Olympus/Olympus-MZUIKO-DIGITAL-ED-75mm-F18-mounted-on-Olympus-OM-D-E-M5__793

Sharpness is 13 MP.

This is followed by the more underwater friendly 12 mm f2.0 with 11 MPix sharpness.

I have the Oly 12 mm but have just tried it underwater once. Clearly I should use it more!

 

Yeah, I have also been puzzled by the DXO results measuring a lens to 3-4-5 MP on a 16 MP 4/3 sensor...

 

Here is my explanation:

You know the sampling theorem maybe:

f < 0,5 * fs

To sample audio up to 20 kHz you need a sampling frequency of more than 2 x 20 kHz = 40 kHz.

Its the same for images which can be described as sampled when its dvided up into pixels.

The sensor is the sampler and the sampling frequency is the pixel raster, i.e. 5184 x 3888 pixels (20 MP sensor).

Before the sampler/sensor there has been an optical low pass filter in basically all modern digital cameras up until recently.

Removing the optical low pass filter results in moiré and aliasing because of the sampling theorem limitations.

The bayer filter which is usually used to get color from a single sensor compunds the problem further. This gives you color moiré as well.

[Given the example above the low pass filter should soften the image down to 2592 x 1944 pixels, effectively give you a 5 MP image. :-) ]

[The nyquist freqency for the image sampler seems to be dictated by the number of pixels at each mm on the sensor:

http://www.edmundoptics.com/resources/application-notes/imaging/camera-resolution-for-improved-imaging-system-performance/ ]

I don't know the details of the actuall level of optical low pass filters (anti alising filters) aka softening used in a system camera , but you can see an example here so the effect is quite dramatic:

http://thedigitalcamera.net/what-are-the-optical-low-pass-filter-aa-filters-and-how-do-they-affect-your-photographs/

On cameras which has removed the optical low pass filter you have two possibilities:

1. Reduce the offending moiré and aliasing manually in post processing.

2. Use an DSP which analyzes the image and removes/soften problem areas directly in camera. At least the new GH5 does this, probably the other M43 cameras from Panasonic as well.

Here are some more info on the AA-filter: http://www.outdoorphotographer.com/photography-gear/cameras/can-you-go-no-low-pass/

"In high-quality digital maging systems, optical low-pass filters (OLPF) are used to eliminate color Moire fringes. An OLPF cuts off the lens MTF above the sampling frequency of the imager resulting an overall MTF curve that approximates a step function in spatial domain." http://www.optics-online.com/lpf.asp

The short summary here is that the lens+sensor resolution will never be the same as the pixel count on the sensor...

I think we also need to step into the "real" underwater world too.

Firstly, we shoot through water, which will inevitably effect image sharpness.

Secondly, the critical factor is actually how the lens will interact with a dome port. Some very good, sharp (and expensive) lenses perform worse than their cheaper brethren behind a dome port. Not relevant to this discussion specifically, but the Nikon 16-35mm f4 will significantly outperform the 14-24 f2.8 behind a dome for example.

I would specifically ask people for examples of image taken with the lenses you have chosen and decide based on their results. When doing so, it is also important to find out how they have set up their port too.

All the best

Adam

@EspenB: Sounds interesting, but I admit I do not understand your explanation (I understand the Nyquist theorem when sampling analog voltage signals with A/D converters, but the extrapolation to image sampling is too much for me...)

If your explantation is rigth, then using the Zeiss 16-35mm with a7II 24 Mpix or even the a7sII with 12 Mpix will result in a great loss in image sharpness due to severe "undersampling", compared with the Sony and Olympus lenses (that just perform optically much worse and provide much unsharper images by themselves). I cannot believe this...

Your initial explanation that the "small" lenses are technically more difficult to construct seems to me much more likely (all the lenses we are talking about are "high end" both with respect to quality and price). Probably this phenomenon is related to the decrease of image sharpness at high aperture values (due to the diffraction of ligth at rim of the aperture). Becomes a real life problem with MFT cameras already at apertures > 8 (depending also on focal length) so that many people avoid higher aperture values with these cameras. With the bigger lenses and sensor in VF, this problem becomes substantial only at the highest aperture values possible ...

@adamhanlon:

"Firstly, we shoot through water, which will inevitably effect image sharpness." of course, but what would you estimate would then be the highest resolution in Mpix that makes sense in UW-photography: 2 Mpix, 6 Mpix, 10 Mpix? Any resolution above this is then just "overkill" in equipment??

"Secondly, the critical factor is actually how the lens will interact with a dome port. Some very good, sharp (and expensive) lenses perform worse than their cheaper brethren behind a dome port. Not relevant to this discussion specifically, but the Nikon 16-35mm f4 will significantly outperform the 14-24 f2.8 behind a dome for example.": This is very true. The practical examples from people using camera/lens/domeport combinations (ideally the same photographer) would be great and very helpfull in decision making (no matter what system one will choose this is an investment of 5-10 K for camera/lens/housing&domeport (without arms, flash(es) and ligth(s)). Certainly worth extensive research before ordering). I could, however, not find such a comparison anywhere so far. I think such a comparison would be of great interest to many of us...

Wolfgang

Wolfgang,

Doing a comparison like you suggest is very expensive and time consuming so it is unlikely that you will find this answer. I know of one underwater photographer who has and I believe still is shooting with 24x36mm, APS-C, and m4/3. That person is Alex Mustard. He has discussed this a bit here on Wetpixel as well as in his book. Keep in mind that the rules keep changing. I would say that the Nikon D500 is the current state of the art APS-C camera for underwater use. So conclusions based on earlier cameras may no longer be valid. This raising of the bar applies to all formats as well as camera types - Almost no one can keep with it and have extensive side by side experience with all of them.

A number of people bought into the high MP Nikon D800 and its derivative models and find the high MP useful - discussed here on Wetpixel. Keep in mind that water itself is somewhat like an AA filter in that light is scattered and thus resolution is diminished. However unlike a filter the effect of water is quite variable - some things to consider are water distance, what is dissolved in the water, and what is suspended in the water.

There are a lot of choices out there. One really needs to know very specifically what ones requirements are to make the best selection. For example, I make use of high ISO - have shot a lot at ISO 12800 in the last couple of years. I am shooting close-up action in poor light that can also be low contrast (e.g. salmon spawning behavior) so need the most sensitive and fastest AF there is. As well, I am shooting in cold water. I measured a water temp of 34F last month when I was shooting - this is about 1 degree C. I shoot for prolonged periods - not so much in December but I have had the camera powered up in one spot in a stream for 12 hours continuously during the long summer days. The only cameras that meet my requirements are the full sized sports oriented DSLRs. BTW, mirrorless cameras all seem to have tiny batteries that die rather quickly.

If I lived in Austria I would be temped to photograph Huchen spawning. This would probably require the same gear I am using with salmon in Alaska. If you are only going to shoot in the Red Sea then your requirements might be quite different.

Here are another view on the resolution limits of sensors, courtesy of the age old Kell factor:

https://en.wikipedia.org/wiki/Kell_factor

Thank you all for the interesting comments...

Seems to be very difficult to compare the different WA lenses and camera systems by objective means under real life and UW conditions.

Wolfgang

Here is more information, maybe of interest for discussion:

Also on another webside benchmark tests of lenses are performed and made public: http://www.photozone.de/

The measure of sharpness there is Linewith/Pictureheigth (I assume that the numbers they give is actually PH/LW, as these numbers are quite high and not a tiny fraction of 1 as expected for LW/PH).

The results for the lenses under discussion is similar, but by far nor identical to the DxO results. Here are their results, the range is best resolution to worst resolution in the aperture range indicated, in all regions of the image (including center and corners):

Zeiss 16-35mm (F4-F11):and also i

Sorry, I pressed the TAB button, here come the results:

Zeiss 16-35mm (FF; F4-F11): 4459-2987

Sony 10-18mm (APS-C; F4-F11): 3469-1910

Zuiko 7-14mm (MFT; F2,8-F11): 3125-2147

other MFT lenses of interest:
Pana 7-14mm (MFT; F4-F11): 2400-1800

Zuiko 9-18mm (MFT; F4-F11): 2400-1600

=> The Sony 10-18mm lens is rehabilitated to some extend, but still the sharpness is closer to the best MFT lens than to FF (for the "older" MFT super-WA zoom lenses, it would just be inbetween). The FF lens is, again, the best, no matter how one considers the results.

As some of you pointed out, optics UW may be different, turbidity/visibility and also the dome port as active component of the optics having a lot of influence on image quality (worsening IQ, never improving it of course).

=> While visibility is uninfluencable we can select different domeports. The larger the port, the further away the virtual image and the less curved the virtual image is. This means better image quality (higher real DOF and higher sharpness in the corners).

May it be, that taking a very large diameter domeport (e.g. 200mm or even 240mm) made from high quality optical glass (when e.g. compared to e.g. an 6" acryl domeport) has appreciably more influence on IQ (especially sharpness) than the optical benchmarks above (that are made on the surface)??

Wolfgang

The Photozone numerical results depend in part on the resolution of the sensor. You can see this more clearly for some of the lenses listed under Canon and Nikon where the same lens was tested with different cameras of the same sensor size. Looks at the numbers on the Y-axis of the graphs when doing the comparison - you will see the scale varies by camera model used. Photozone results are more useful IMHO for understanding how a particular lens model behaves so, for example, you could pick out optimal apertures to use for that lens.

I see - so the better resolution of FF and APS-C lenses tested are partially because the sensors have more Mpixels. This explains the difference between DxO and photozone benchmarks...

You also need to take into account some more dome optic essentials, such as minimum focus distance of the lens. The lens focuses on a virtual image which is much closer than the actual image and is also curved. The larger the radius of the dome, the larger the radius of the curved image, so big domes are better. Generally you are wanting to stop down to get enough DOF so that the corners of this curved virtual image are in focus when the centre is. This I have seen as at least part of the explanation of soft corners in wide angle lenses. Many domes for wide angle lenses (non fish eye) are designed with a segment of a much larger sphere for instance the ZEN dome for M4/3 is 170mm dia but is a segment of about a 230-250mm dome so can only accommodate up to about a 7mm lens (14mm 35mm equivalent). So a lens may be super sharp but won't focus close enough to do the CFWA shot you want.

With micro 4/3 you have more DOF due to lower magnification to achieve a given image size and the lenses are designed around this and the diffraction characteristics of the 16-20MP sensors and peak at f5.6 mostly. The problem is that while the scene you are shooting may have enough DOF at f5.6 without a dome you're needing to stop down to to give acceptable corner sharpness due to dome optics as discussed above. So this places limits on optical quality of the lens behind a dome, which you would not perhaps see on larger formats.

The other factor is proper placement of the lens behind the dome. You will note that a number of companies quote multiple lens can be placed behind specific domes. I'm sure all these lenses don't have the nodal centre at the exact same distance from the sensor plane, but the domes all place the radius of the dome at the same distance, you are supposed to place the nodal centre at a distance equal to the radius of curvature of the dome, so there are more compromises there. How close does it need to be to the nodal centre.... I don't know, presumably the manufacturers design it so the average person will be happy with the results. The point is mulling over the finer nuances of resolution is all well and good but the truth is you would be hard pressed to tell the difference in actual UW shots.

Additional considerations include:

Price: A M4/3 housing can be around 1/2 the price of a DSLR housing and the lenses are significantly cheaper and more compact

Size:

• A DSLR, particularly a full frame and big dome have a lot more surface area so harder to push through water
• You can probably carry on a M4/3 setup for travel but would have to check a full DSLR setup for air travel

AF: A DSLR will have snappier AF for macro and in low light - probably not a problem for WA. M4/3 apart from newer top line models like the EM1 use contrast detect AF which is slower and more prone to hunting. DO you also want to macro?

Dome and zoom gear availability may be a factor and the choice of housings for the camera will also factor in.

Also you don't state what you want to do with the WA setup. CFWA is often reckoned to be the domain of fisheye lenses as these will often focus right on the dome. A rectilinear wide angle in a big dome generally won't focus as close and also you physically can't get the big dome in to take the shot. Rectilinear Wide angles may be better suited to wrecks and large animals like sharks.

To my mind you are approaching this backwards or maybe sideways, the practicality aspects of getting all this gear under water and getting everything right seem more important to me than the absolute sharpest lens. Sure you want a sharp lens and there some poor to average performers out there but to take advantage of all this sharpness requires everything to be exceptional and for the most part any of the high ranked lenses in DxO will do the job and be hard or impossible to tell apart from each other.

Thank you all again for the valuable contributions, that make decision making easier for me...

Here is the bottomline of the contributions:

#1.: Generally high end big WA lenses perform better (at least with regard to sharpness) as their small format (MFT) counterparts do (same applies to the size of the sensor).

#2.: Direct objective comparison, how much better the IQ is in reality, is difficult.

#3.: Most important: In rectilinear WA the optical dome is the weakest element in the optical chain and the most limiting factor for IQ in practice. Hence a dome with large radius of curvature and exact alignment with respect to entrance pupil of the lens is much more essential to real life IQ than type of lens used (VF, APS-C or MFT). A "segmented dome" as ChrissCross describes here: http://wetpixel.com/forums/index.php?showtopic=59408is probably the optimal solution one can get.

#4.: Optimum dome curvatur for fisheye is still not clear to me (the larger the radius, the better, as with rectilinear?)...

Wolfgang