Judging lenses on the web....

[rjlhughes]

How well can you judge lenses by pictures you see on the web?

When the subject came up ion another thread sirhc55 noted:
"I tend to go to http://www.pbase.com/cameras/sigma/18-50_28_ex_dc and check out what other people have achieved with the lens (the address is for the Sigma 18-50). In this way I can check on all manner of pic types (in the 18-50 there are over 3,000 pics). But none of this compares to putting the lens on your camera and trying it for yourself."


I've gone to photosig for the same exercise as Chris suggests for every lens I've bought. You see the quality of the photographer who has chosen to use the lenses as much as the quality of the pictures. That may be a better indicator, in the long run.

I notice too on flickr that someone shooting with a Tamron 70-300 is getting magnificent results with the 350D, although the photosig shots are very ordinary. And some of the shots I've seen here taken with that lens put me off it.

Fredmiranda has reviews that can be useful - but often you're seeing the lens through the lens of the photographer's experience (or inexperience).

There's the technical specs, of course, too.

So I'm very influenced by what I see on the net.

But I'm beginning to think it's not a very smart strategy.

Your views?

[Matt. K]

There is no way you can make any meaningful judgement on lens quality from images on the web.

[rjlhughes]

I suppose we could just leave it there then.

But I doubt that will happen.

[Sheetshooter]

In order to conduct any useful evaluative test of performance (of anything) it is essential first and foremost to minimise the number of variables influencing the result.

Quite apart from actual lens performance, which itself encompasses many areas requiring specialised investigation (Resolution, Acutance, Coma, Sherical Aberration, Chromatic Aberration) there are too many other factors that can make either a poor performing lens look good or a good performing lens look a right lemon.

There are issues of mechanics such as front- or back-focus, focus repeatability, colimation and so forth. Then there is the matter of the Image Capture itself: RAW, JPEG, Parameters of Saturation, Sharpening, Contrast etc. Add to this the presence or absence of flare and the level of skilled precaution to guard against it. Camera Movement and Subject Movement will also influence the result as will the style, direction and intensity of lighting.

After all this is taken into account we are presented with about 72dpi of resolution with which to make our assessment. Monitor Calibration, Signal Interference and other artefacts raise their noxious little heads also.

The best we can hope for is some sort of comparative trend or consistency displayed in a given lens over a broad range of users. But even that is not definitive because sloppy practices and casual shooting are never the stuff of truly evaluative testing.

As I have written previously, I use a Putora Sharpness Indicator Test Chart: 7A9 to evaluate the performance of my lenses on film but even this can ONLY make an informed indication of just one aspect of optical performance: RESOLUTION - and that too is subject to film structure, acutance and development.

[rjlhughes]

All well noted.....many thanks. And sharpness isn't all, of course.

I thought someone would say how good the lens seems depends on your monitor.

I expected someone else to say that if you were shooting for the web then the web might be a reasonable place to make an evaluation. But if you were shooting for print.....

But I think you might have gone a step or three further.

I've got to get that old TV down from the attic - the one we used to watch the Bill with on Saturday night - the new one has other programmes on it.

[genji]

As I have written previously, I use a Putora Sharpness Indicator Test Chart: 7A9 to evaluate the performance of my lenses on film but even this can ONLY make an informed indication of just one aspect of optical performance: RESOLUTION - and that too is subject to film structure, acutance and development.

sheets

what is your take on this method DxO .a method that Michael Reichmann of Luminous Landscape employs to evaluate digital gear.

[Sheetshooter]

Genji,

I am familiar with Lu-La and Mr. M. R. but could hardly say taht I am a fan - although I do enjoy the Mike Johnston column wghen it happens (soemtimes).

A colleague purchased DxO and we have discussed it but according to him it requires data fo the focus distance and his Canon 300D does not output that data.

If you have further and better particulars I'd be all ears.

Cheers,

[Aussie Dave]

Isn't it fun trying to compare lenses etc.. by reading reviews. If you read enough of them, they all contradict themselves eventually.

Just like a camera, the lens is only another tool we use and as such, is heavily dependant on the person using it. I'm sure everyone has seen remarkable shots taken with the 70-300G Nikon lens, however it is crystal clear that this lens ain't no 70-200VR. Good lighting, along with well chosen camera settings can make even the most lacking lens look good, and sometimes great.

Now, give the 70-200VR to someone who doesn't know what they're doing and it will quickly become evident that it's not the lens....it's the photog.

Of course, better lenses employ better quality glass, manufacturing, specifications etc... From my take on things (and this is only my perspective), the better the lens is, the more available options you will have to play with. (eg. having a telephoto lens that is capable of holding f2.8 throughout it's entire range gives you many more option when light levels drop. Try using a variable aperture lens that starts play at around f5.6 and already you are behind the 8-ball, before you begin - in the same low-light situation).

Using online images as the comparison cannot be effective, as we don't know how much PP has been done to the images, which will greatly affect it's outcome.

I guess the old saying, "you get what you pay for" somehow always rings true....

[rjlhughes]

Well that's right Dave,

and I wonder how much better photographs are taken by someone using a lens called Nikon compared to one called Tamron when they are the same lens?

I don't know that all reviews contradict themselves, however. Some reviewers may disagree with the majority view, of course. But in professional journals there are far more lens and camera 'praisers' than camera 'critics'.

As for DxO - it's a shortcut to what people would do in PP I suppose....but how does Luminous Landscape use it to evaulate lenses?

To be a bit clearer I wonder if anyone has any strategies for discerning the useful truths about potential lens purchases from all the information that's available on the net?

[genji]

As for DxO - it's a shortcut to what people would do in PP I suppose....but how does Luminous Landscape use it to evaulate lenses?

sorry bob,

i posted the wrogn link try this it explains how DoX is used...

http://www.luminous-landscape.com/revie ... ined.shtml

[genji]

But in professional journals there are far more lens and camera 'praisers' than camera 'critics'.

with Pro mags, the lens/camera manufacturer advertise in those mags...

[rjlhughes]

Genji

I think that might be one important answer to my question - if only it was widely used for the 350D - I'll go and have a good look at LL.....

And I'll be interested in what Sheetshooter has to say.


thanks

[robboh]

Isn't it fun trying to compare lenses etc.. by reading reviews. If you read enough of them, they all contradict themselves eventually.

Yeah, its enough to drive you stark raving mad!!

I find the best ones to find are a comparison-type of review, such as the 80-200 v's 70-200VR on Camera Hobby. There you have a fairly qualitative differentiation as they are run through the same tests side-by-side. To be fair, not particularly scientific testing, but a good 'real-world' test IMHO. It especially helps if you have one of the lenses in the comparison already and can run your own similar tests against your lens, v's whats in the review.

Thom Hogan also seems reasonably reliable and doesnt seem to mind pointing out Nikon's failings.

One of the problems I find is trying to find someones opinion I take note of that has also done reviews of things like the Sigma's etc AS well as the name brands.

Using online images as the comparison cannot be effective, as we don't know how much PP has been done to the images, which will greatly affect it's outcome.

This is the biggest problem IMHO. You can take a very mediocre pic and PP it to come up with something pretty stunning.

Then you have to take into account someones testing and their general knowledge. For example, someone states 'no sharpening' on this pic. How do you know that? For example, if they just process through NC, then there WILL be in-camera sharpening unless they specifically turn it off under the RAW options (or have disabled in-camera). Same for AdobeRAW, the sharpening is hidden in a sub-tab. Some (i seem to recall Capture One) even if you turn off sharpening, will still sharpen a pic to a certain extent.

I think at the end of the day, the 'get what you pay for' does ring true and that the photographer/light will typically make more difference to a picture than the differences between lenses in a similar class. I also think, that like cars etc, there are far fewer true lemon models out there these days.

[birddog114]

I don't have any trust in reading the review of lens on the web or seeing it thru a photos on a web.

The best part I'm doing was, tried it in my hand, seeing what is your real needs and your style of shooting, lens and camera bodies are the two difference parts and they don't quite match your taste and my taste or other each individual + your skills + your experiences.

The review on the web are other factors make people believe it's good, excellent or worst, in the real term it's totally difference once it's in your hand.

Judging the lens thru a photo on the web is unrealistic! IMHO.

[stubbsy]

There is NO substitute for evaluating a lens by using it. Any other method is a poor substitute.

From personal experience I've seen great web pics with lenses that don't work for me (70-300G is an example). I have had the luxury of try before buy for all my lenses. That's one of the big pluses to mini meets be they in Adelaide, Brisbane, Canberra, Melbourne, Sydney or wherever.

[rjlhughes]

Gentlemen,

thanks for your thoughts.


Hey Birddog,

love to see some of your pics!

Stubbsy,

I guess my post presumes that you can't put the lens on your camera, which is true for most people.

Incidentally I'm surprised at the lack of complaints on these pages about bad copies of lenses. Perhaps Nikon's QA is that good.

Robboh

I will check out Thom. Thanks for the tip. I wonder at what point post production will make lens quality far less important? DxO is heading there I guess.

[Glen]

Bob, I would wholeheartedly agree that you cant judge a lens totally without having it in your hands. For me, the measure of a lens is how much it helps you in adverse conditions and one can't tell under what conditions the shot was taken.

That said, there are a number of reviewers who are good. I and others have posted links here. http://www.d70users.com/viewtopic.php?t=2888 I would pick a reviewer whose opinions concur with your own experience. In looking at lenses I tend to read the quality reviews, then investigate further culminating in seeing the lens. The first reviewer, Bjorn Rorslett, I find to be very accurate. All these links are Nikon based, so you will need to find a Canon equivalent. Have a look at Bjorn's reveiw site to see how extensive and good they can be

[robboh]

Agree with Glen here, Bjorn Raslett also is worth a read. He's the only review on the net where Ive seen comments on the 80-200 2touch that match my own experience; ie not very good at near min focus distance at wide apertures.

[rjlhughes]

Thank you gentlemen for the tips,

I had looked at some of these sites in the past, but thanks for the refresher.

Wish Photodo was still current, of course, since those figures don't lie.

And think I adopted Bjorn's brick wall method at one stage to test a lens. And yes wish there was a Canon equivalent.

Interesting that he also complains about the lack of quality testing even with Nikons, yet we don't see complaints on these pages, as I noted before.

[MHD]

without fully reading the thread I will add my thumbs up to Bjorn's page..

[rjlhughes]

Bjorn:

"The fact is that you run a real risk of getting a lemon sample even of highly expensive professional lenses, in particular zoom lenses. From a reviewer's point of view, the basic problem isn't sample variability, but the non-feasibility of testing it. "


Does that shock everyone as much as it does me?

[Glen]

Bob, it shocked me the first time I read it In fact made me go out and test my kit lens which was in the middle of the D70 backfocus issues just to be sure. I also tested my 70-200 after getting it as a couple of the first days shots weren't as sharp as I expected. Luckily I got to blame that bugger 'operator error', the lens was fine. I didn't bother testing my older zooms as frankly I wouldn't pay to fix them. Also didn't test all the primes as they usually never dissappoint.

I think one of the reasons for less complaints about lenses is most people aren't bothered to go through the testing. Also if you move from a cheap consumer lens to a sub standard pro lens, the pro lens is probably still a step up

[rjlhughes]

Good thought Glen,

Our expectations do make a difference. And as SS says the professional world and its standards are far higher than they are in the amateur area.

I suspect that there are many unjustified complaints about lenses registered on sites like DPReview, too.

I suspect that the manufacturers just replace the lenses without query - and then I trust don't recycle the 'defective' lenses to new customers.

There is also the issue of calibration of course, which seems to be important and could explain why so many amateurs think their lenses are soft.

I also mentioned in another thread that one of Sydney's leading lens repairers advised me very strongly against image stabilisation technology.

The more complex things become, the more fragile they are.