Won't find any pixel width detail in a film, and anything anywhere near a pixel in width looses contrast.
Where are the forum flame fests about how the director "added blur" by not getting the in-focus circle of confusion smaller than a pixel or where the textures used in the Ironman CG shots lack detail?
Compare to a possibly doctored "perfect" driver SSAA override screen grab from Skyrim posted on Neogaf (again click images for full size),
The industry status quo is to push ultra high display resolution, ultra high texture resolution, and ultra sharpness.
IMO a more interesting next-generation metric is can an engine on a ultra-highend PC rendering at 720p look as real as a DVD quality movie? Note, high end PC at 720p can have upwards of a few 1000's of texture fetches and upwards of 100,000 flops per pixel per frame at 720p at 30Hz.
EDIT,
Another interesting comparison which provides a numerical answer, is to down-sample with un-filtered decimation shots from BR movies, and see at what point edge gradients start to match high quality MSAA or CSAA shots from games. For instance with the same Ironman shot, one can use nearest neighbor down-sample at 33% of the original size and get something which has visually similar edge gradients to high quality AA on games,
And the full size un-filtered down-sample,
In practice, it seems as if somewhere between a 50% and 33% nearest neighbor down-sample would be typically required to make BR movies have the sharpness of the high quality AA found in games. Intuitively this makes sense as "50%" is the natural lower bound: detail under 2 pixels in width will have aliasing under motion. Inversely, it seems as if 1080p BR movies typically have somewhere between only 350p and 540p of effective "game resolution". Or alternatively in game AA terms, BR movies get an extra 2x2 or 3x3 OGSSAA but instead of getting resolved to the 350p to 540p resolution, they get a DOF "blur" applied at 1080p...
Great post Timothy, couldn't agree more.
ReplyDeleteIt's not about the amount of pixels, it is about the quality of the pixels and how the overall (moving!) picture looks like. Less aliasing = less noise for your brain to interpret = more pleasing & easier to see visuals.
Though games as they are interactive and not a passive medium may do require does require some more sharpness in some areas (like UI).
Agreed completely. I think there are many things in game graphics that could be helped immensely by doing serious image comparisons versus photos and movies. What you point out is a great example. Something else that bothers me in most games these days is how much contrast there is in the textures. Having physically based material guidelines help but the artists seem to try everything they can to create higher contrast. The result in my opinion is crunchy, noisy and often nasty looking images. I'd call the status quo ultra sharpness, ultra contrast.
ReplyDeleteI found an interesting comparison that demonstrates your point perfectly.
ReplyDeleteWALL-E Bluray
http://hq55.com/disney/walle/walle-disneyscreencaps.com-521.jpg
WALL-E HD production shot
http://0.tqn.com/d/kidstvmovies/1/0/I/H/walle008.jpg
Use a box filter or resize with your browser and it is very easy to tell that Pixar chose to make the image less sharp for bluray resolution than what a game would normally do.
Brian sneaked in the point I wanted to make: The better comparison is between Games and Animated films - since we are also required to fully render our frames.
ReplyDeleteHowever! That Wall-E "production shot" is rendered at 4961x2070 - which means that frame is from Marketing and not from the movie. Almost all of our films are at 1920x____ (the few exceptions are lower res, not higher) The Blu-Ray is an accurate representation of the softness in your average film frame.
We do what is essentially MSAA. Then we do a lens distortion that makes the image incredibly soft (amongst other blooms/blurs/etc.) Softness/noise/grain is part of film and something we often embrace. Jaggies we avoid like the plague and thus we Anti-Alias the crap out of our images.
In the end it's still the same conclusion: games oversample vs film. I've always thought that film res was more than enough res. I don't know how you will get gamers to embrace a film aesthetic, but it shouldn't be impossible.
Brian, great example with the WALL-E shots, the point even survives the poor JPEG conversion process. WALL-E is an interesting case given that CG animated films do not have to match the same larger circle-of-confusion common of digital HD or 35mm film, especially on shots taken in challenging situations (low-light, mixing in-focus ultra-foreground with in-focus background elements). Animated CG definitely tends to push the practical limit of sharpness just before the point of visible temporal aliasing in motion. Here are some more BR shots from WALL-E,
ReplyDeletehttp://www.cinemasquid.com/blu-ray/movies/screenshots/sets/wall-e/e60f0af1-9fca-4ade-9084-fa569ca346ed
Johan, awesome description, "Less aliasing = less noise for your brain to interpret = more pleasing & easier to see visuals"!
ReplyDeletehttp://disneyscreencaps.com is where I found the bluray shot. They have tons from many films.
ReplyDeletedistastee:
Yes, that was what I was trying to show. The marketing shot is very close in content to the final shot but at a much higher resolution which allows the comparison to the actual bluray frame which shows the correct filtering for 1080p. It clearly demonstrates that a pixel size box filter was not used which is currently considered by some the holy grail for game AA.
Timothy:
The way you describe the filtering process sounds like upsampling to me but the way I understand how offline supersampling works is something like a tent filter (or something fancier) where neighboring samples affect the current pixel. In other words a single sample can contribute to more than one pixel. The size of the kernel is what determines the sharpness and amount of aliasing. Maybe this is what you meant by effective resolution but I wanted to make clear you weren't meaning the source data was actually lower resolution. I do like your technique for finding the effective kernel size though.
You can do 1080p at movie quality on current gen PC hardware
ReplyDeleteAdvance warning: I'll be going off on at least one tangental rant here... (In fact, I think there are even tangents to my tangents, but I digress.)
ReplyDeleteI think there are several interrelated aspects at work here that have built up this fixation on display resolution and sharpness.
In the consoles arena, one thread is that people want their 720p+, and somewhat justifiably. They've invested in HD (TV, consoles), with the promise of, well, high definition, ergo sharper images, etc. That's been the big differentiator, the selling point, and TV manufacturers know this all too well. They need to stand out, so what do we see as out-of-the-box settings? Of course: artificial sharpness and contrast (not to mention saturation, too), because that's what helps to shift units in the showroom. They're *all* 'juicing'!
Aside 1: Film purists naturally hate this stuff. I read one account a while back of a movie guy going around to a friend's house and insisting that they calibrate their home theatre setup for them. Lo and behold, on their return some weeks later, all the settings were back as before! Apparently things just "weren't the same" without all of that artificial boosting. This is conditioned!
But whilst this tweaking can be somewhat detrimental to films, it can really destroy games, where sampling quality is currently borderline at best (certainly in the console space) and our hard work is then undone by these shenanigans. Yes, TVs typically have various content modes, but people are lazy, me included.
Anyway, whether it's TVs or sub-par edge AA or a combination, I can see why some people (a vocal minority, mind you) get upset when console games are sub-HD, because the effect is often noticeable (*at the moment*). But even when it's not, it's a convenient (read: lazy) metric to judge quality on. Surely 720 > 640, that simply *must* be true! Right?! Rare is the discussion about pixel *quality*.
Aside 2: The same issue shows up when trying to argue that your heavily-modified UE2-powered game is as good as a UE3 game. Even experienced game developers lazily pigeon-hole like this.
The art style of some games even plays on this sharpness/contrast = resolution association as well, with highly detailed or strong normal maps and geometry. Coincidentally, Epic really pushes the envelope here! ;)
So, to steal a line from Mad Men, we need to "change the conversation". My meta point is that although the focus will naturally shift anyway as we get more power and frame resolution plateaus, there's a conscious shift of mindset that needs to happen, and first of all that has to happen on the the development side.
There's a similar conditioning at work (pardon the pun), whereby a lot of artists in games just *expect* aliasing of various forms and don't even try to fight it (exhibit a: chain-link fences - oh how I hate thee). Then there's the 'juicing' of texture content that Brian points out, often to counter low-quality texture filtering. This is even before we get on to physically-based shading, shader AA, *proper* shutter effects, and everything else that we're missing quality-wise.
Many current practices need to be re-evaluated, expectations need to be raised, the focus needs to shift. So, indeed, the next few years are going to be *extremely* interesting and a very fun ride! :)
Brian, the point I was attempting to make about the filtering process is that a game could render at native 1080p resolution (with extra optional hardware AA or SSAA) then use a fast post effect filter (optionally combined with post AA to "estimate" shaded samples which don't physically exist) to apply a film like DOF and get a result similar to a 1080p BR film. And that this process would have a filter kernel which would be at least 2 to 3 (of 1080p) pixels wide (and likely larger in practice to get a good window).
ReplyDeleteThe filter kernel shape roughly approximates the circle of confusion (COC), which is an over simplification (the shape isn't always a circle, could be cropped by the lens, and changes across the frame, etc). Fine detail contrast is effected by the hardness of the edge on the COC shape. The larger and softer the edge on the COC, the lower the contrast on fine detail.
> that people want their 720p+, and somewhat justifiably
ReplyDeleteThere is a good argument to always at least output native resolution for HDTVs simply because non-native resolution like 720p on a 1080p HDTV will, I believe always, incure an extra frame of latency in the HDTV due to scaling.
Going off my now vague memory of the PS3, seems like rendering a letter boxed 960 lines at 1080p, with 960 wide hardware scaled to 1920 would provide better latency and be the same cost as 720p.
Some of the blurring comes from the films chemical response right, not just rays being out of focus? If so, that means the filter won't necessarily be a disk.
ReplyDeleteHere's a ton of info on the antialiasing filtering in PRman
http://www.renderman.org/RMR/st/PRMan_Filtering/Filtering_In_PRMan.html
I know I'm replying to a month old post, but maybe this will be of use to someone.
DeleteThe effect that Lottes is describing here is called microcontrast - or more so the lack of it in films when compared to games. It can be impacted by the by film itself, but the lens is the main thing that matters. For a good explanation on this, see this page - http://www.luminous-landscape.com/tutorials/understanding-series/lens-contrast.shtml
Now the really fun thing is that you can get rid of most of the ridiculously high microcontrast in games by doing a sort of reverse unsharp mask (and even a shader, which should work with the fxaa injector) An unsharp mask is actually a boosting of microcontrast, NOT the overall sharpness of the image but instead an increase in the APPARENT sharpness. That is something important to understand, and a lot of people don't seem to get it - sharpness is made up of several factors.
But, any who, running screen shots of a game through a "backwards" unsharp mask knocks down microcontrast without destroying smaller tonality and makes things look much more cinematic. Crysis, especially its vegetation, benefits immensely from such a treatment. I can post a few sample screens of this if anyone is still interested in this topic.
Anyways, as someone else was saying the conversation in games really does need to change. But I think the best way to deal with this problem would be to teach photography before 3d graphics to people going into the video game industry. How can you expect to create photo real images without understand how photography and film actually work is beyond me. And you do see the consequences of this "3d first" education almost everywhere in gaming.
Although I guess I'm kinda biased as I learned how to shoot and process film long before I started studying 3d graphics and rendering... and you can make a strong argument that its a purely subjective aesthetic argument. However I definitely think cinema can teach gaming, especially given how much though is put into how films are actually displayed. A projector screen or TV isn't the human eye. Not only does it lack the dynamic range of what we see day to day, but it also has a finite resolution and the like. The problems related to displaying moving images have largely been solved or compromised well by cinema over the past 100 years.
I don't mean to ramble but all I know is I'm really sick of seeing hyper sharp and high contrast games. Like a gravel road or brick wall where the texture has crushed shadows and blown out highlights. Bleh
Besides the Luminous-Landscape, Norman Koren's page on MTF is also quite awesome (along with the rest of his site): http://www.normankoren.com/Tutorials/MTF.html
DeleteApologies if I derailed the thread a bit earlier. It was partly in response to your "industry status quo" remark and a general need to vent. ;)
ReplyDeleteI'm really pleased that we'll be able to seriously contemplate these subtleties going forward, instead of just trying to make things *run* with a decent number of lights and objects in frame!
I've been beating the Image Quality drum a lot lately. To me it's clear that along many axes, simply adding *more* – particularly when it comes to geometry, which can actually exacerbate some IQ issues – isn't going to achieve a whole lot in comparison to better filtering, sampling and so on. I should have thought to really drive the point home with direct evidence like this.
Braid is a game that is relevant to this discussion. It was the first game that I've come across that actually embraced a blurry art style.
ReplyDeleteIt has absolutely no sharp edges, especially when running at 2560x1600.
It's done very artistically, and it works well.
Sorry, I meant to say Limbo, not Braid, in my comment above. >.< Although it's true for Braid also, but the effect is really exaggerated in Limbo.
ReplyDeleteDeus Ex: Human Revolution is erring on the side of softness, to great effect. It's full of "concept art come alive" moments, which are very soft, dreamy, non-game-graphicsey.
ReplyDeleteWhat it would great is to put some science behind these intuitions. We should start running studies to understand how sharpness and aliasing affect the perception of a synthesized image.
ReplyDelete