Timothy Lottes

Old School: Modems

2012-02-11T23:48:00.000-08:00

This brings back memories, years ago used to manually force modem handshake using my voice...

Random Text Mode Demo Tube

2012-02-11T18:12:00.000-08:00

Trapcode Gallery

2012-02-07T00:06:00.000-08:00

Trapcode - Gallery

Another Hawken Tube

2012-02-06T10:06:00.000-08:00

A Survey of Non-linear Pre-filtering Methods for Efficient and Accurate Surface Shading

2012-02-01T18:48:00.000-08:00

A Survey of Non-linear Pre-filtering Methods for Efficient and Accurate Surface Shading

Cyril Crassin's Ph.D thesis on GigaVoxels

2012-01-25T22:52:00.000-08:00

Cyril Crassin's Ph.D thesis on GigaVoxels

Rasterized Bounding Volume Hierarchies

2012-01-23T11:04:00.001-08:00

Rasterized Bounding Volume Hierarchies

p.i.d.

2012-01-22T11:30:00.000-08:00

Practical Real-time Micro-Polygons?

2012-01-21T22:17:00.000-08:00

Is it Possible to do Real-time Micro-Polygons Today?

Answer has been no in the past, but is this still the case today on current GPUs?

The concept is not new, and has some good things going for it. Shade at vertices only, with a pixel shader which simply interpolates color (possibly better than linear interpolation), with triangles small enough so that this works visually. Decouples shading rate from resolution, could decouple shading rate from refresh rate. Pre-tessellation of mesh can be a form of importance sampling for shading. Easily works with high quality hardware AA. Closer match to film rendering pipelines. Etc.

Simplified GPU Primitive Rates

Lets assume a 1 GHz GPU clock to make things simple and {1, 2, or 4} parallel raster units per GPU, so a maximum of {1, 2, or 4} billion triangles per second. Divide this by 60, provides {16, 33, or 66} million triangles per frame. Compared to around 2 million pixels at 1080p.

The Fermi GTX480 demos had some amazing triangle rates on dynamic and simulated geometry,

Starting to think this idea is not as crazy as it sounds...

Automatic LOD

The "Automated Level of Detail Generation for HALO: REACH" presentation from the 2011 GDC Vault shows one method for automatic LOD generation for vertex shaded models. Video,

Misc Demo Tubes From 2011

2012-01-20T14:28:00.000-08:00

Many View Rasterization

2012-01-20T10:10:00.001-08:00

Many View Rasterization

Screen Space Bent Cones

2012-01-19T21:12:00.001-08:00

Screen Space Bent Cones

Dear Esther

2012-01-18T22:50:00.000-08:00

As fellow landscape photographers know, the most interesting light is when the sun is not to be found.

To Extremes and Back To Reality

2012-01-12T21:38:00.001-08:00

Entertaining NeoGAF Thread: Developers Discuss Benefits Of Targeting 720p/30fps Next Gen, Using Film Aesthetics captures a lot of the concerns of the vocal PC and console gamer, responding to comments in the Games vs Film post.

Ok, Now Back to Reality

My prior comment, "IMO a more interesting next-generation metric is can an engine on a ultra-highend PC rendering at 720p look as real as a DVD quality movie?" is a rhetorical question asking if it is possible for a real-time engine to start to approach the lower bound of a DVD movie in realism.

To make this clear, I'm not suggesting that games should compromise interactive experience just to get visual quality. If I was going to develop a title for next generation consoles I would output 1080p and run frame locked to 60Hz with no dropped frames period. I still believe developers will be able to start to reach the quality of film for next generation consoles and current generation PCs, and I'm intending to develop or prove out some of the underlining techniques and technology which gets us there.

At the same time, certainly expectations for next generation consoles should at least be grounded in some rough realistic estimates for performance. Using public information found on the internet, lets nail down a realistic estimate of what next generation console performance will be, by looking at how ATI/AMD has evolved GPU performance after the Xbox 360,

(1.) Next gen console games will be outputting at 1080p. I can say this with full confidence simply because HDTV typically adds a frame of latency when it needs to convert from 720p to 1080p.

(2.) Using HD6970 as a proxy for a high end PC version of the Xbox 360, lets compare specs. Going from the typical 720p @ 30Hz on Xbox360 to 1080p @ 60Hz on HD6970 with 2x the geometry would take roughly 4x the performance (2x the pixels and geometry times 2x the frame rate) just to provide a similar experience at the higher resolution and frame rate with similar average pixels/triangle.

HD6970 has roughly another 2x over the 4x required to maintain same look at the full HD experience,

Xbox360 =  240 Gflops : 22.4 GB/s :    8 Gtex/s :    4 Gpix/s
HD6970  = 2703 Gflops :  173 GB/s : 84.5 Gtex/s : 28.2 Gpix/s
-------------------------------------------------------------
roughly    11x Gflops :   7x GB/s :  10x Gtex/s :   7x Gpix/s

(3.) What about process scaling, lets attempt to get an idea of what future technology might have, lets compare HD6970 to HD7970. Looks like AMD managed around a 1.4x on-paper spec increase except they did not scale Gpix/s.

HD6970  = 2703 Gflops :  173 GB/s :  84.5 Gtex/s : 28.2 Gpix/s : 250 Watt
HD7970  = 3789 Gflops :  264 GB/s : 118.4 Gtex/s : 29.6 Gpix/s : 250 Watt
-------------------------------------------------------------------------
roughly   1.4x Gflops : 1.5x GB/s : 1.4x Gtex/s :    1x Gpix/s

(4.) What about power scaling? The latest shrink of the Xbox 360 hardware uses a 115 Watt power supply (for the entire system, not just the GPU). So lets assume that next generation consoles won't have huge power supplies like PC GPUs. Taking what I'm wild guessing to be a really liberal estimate for possible GPU power for a 115 Watt system, lets compare a medium power modern proxy for the Xbox 360, the HD6750 (which is a 86 Watt TDP on paper). These numbers suggest if Microsoft launched a Xbox update around last year, that it would not be able to do 1080p at 60 Hz with the same look as current 360 games (because the HD6750 isn't 4x the 360).

Xbox360 =  240 Gflops : 22.4 GB/s :    8 Gtex/s :    4 Gpix/s
HD6750  = 1008 Gflops : 73.6 GB/s : 25.2 Gtex/s : 11.2 Gpix/s
-------------------------------------------------------------
roughly   4.2x Gflops : 3.3x GB/s : 3.2x Gtex/s : 2.8x Gpix/s

(5.) Next generation console performance will be a function of how much power the machine uses and what process technology each vendor adapts. Launch date of the console is going to hint at what process is used. Process scaling is not constant, but for the sake of making this simple, lets just assume each process gets 1.4x the performance. Then lets look at estimated performance scaling from the HD6750 to keep closer to current "console" power levels. This will provide some very rough estimate on what future consoles might have. Lets estimate process technology road maps by looking at google image search results,

2011   : 40nm : HD6750 :  4.2x Gflops :  3.3x GB/s :  3.2x Gtex/s : 2.8x Gpix/s
2012   : 28nm : ?????? :  5.8x Gflops :  4.6x GB/s :  4.4x Gtex/s : 3.9x Gpix/s
2013.5 : 20nm : ?????? :  8.2x Gflops :  6.4x GB/s :  6.2x Gtex/s : 5.5x Gpix/s
2015   : 14nm : ?????? : 11.5x Gflops :  9.0x GB/s :  8.6x Gtex/s : 7.7x Gpix/s

(6.) EDIT: Given an estimate that process technology will continue to advance during the lifetime of a console, a vendor could adapt higher power for launch, with the expectation of reducing this later in the product cycle. For example the first 360 had a 203 Watt power supply at launch and is now at 115 Watts (according to Wikipedia). This leaves a big window of possibility for performance.

Given the window of possible launch dates and power targets it would be hard to know exactly what will end up in next generation consoles, however a 2011 high-end single-GPU card seems like a possible proxy for next generation console, and at least a good start to understanding what could be possible.

UPDATE

The latest IGN Rumor is 100% fail, as they say the next Xbox is a HD6670 with 6x the perf of 360. The HD6670 at 40nm is clearly not 6x the performance, and even a 28nm die shrink of HD6670 will likely not be 6x the performance Xbox360.

Xbox360 =  240 Gflops : 22.4 GB/s :    8 Gtex/s :    4 Gpix/s
HD6670  =  768 Gflops : 64   GB/s : 19.2 Gtex/s :  6.4 Gpix/s
-------------------------------------------------------------
roughly   3.2x Gflops : 2.8x GB/s : 2.4x Gtex/s : 1.6x Gpix/s

However assuming the 6x number is correct then HD6870 or HD6850 is a possible development proxy,

Xbox360    =  240 Gflops : 22.4 GB/s :    8 Gtex/s :    4 Gpix/s
Xbox360 x6 = 1440 Gflops :  134 GB/s :   48 Gtex/s :   24 Gpix/s
----------------------------------------------------------------
HD6850     = 1488 Gflops :  128 GB/s :   37 Gtex/s :   24 Gpix/s
HD6870     = 2016 Gflops :  134 GB/s :   50 Gtex/s :   28 Gpix/s

Single Pass SRAA

2012-01-10T21:48:00.000-08:00

Here is a better description of a possible single geometry pass SRAA. I'm using OpenGL in this example because that is what I work in most often,

Use "layout(early_fragment_tests)". Described in GL_EXT_shader_image_load_store,

"When early per-fragment operations are enabled, the depth bounds test, stencil test, depth buffer test, and occlusion query sample counting operations are performed prior to fragment shader execution, and the stencil buffer, depth buffer, and occlusion query sample counts will be updated accordingly. When early per-fragment operations are enabled, these operations will not be performed again after fragment shader execution."

The render pass will be writing Z as standard and writing out the lower 8-bits of gl_PrimitiveID into an MSAA render target.

The render pass will fetch the coverage mask, gl_SampleMaskIn[], and check if the shaded fragment covers the sample associated with the non-MSAA g-buffer. If the sample is not covered, the shader exits here.

I do not remember off-hand if the gl_SampleMaskIn[] has pre or post-z test coverage when "early_fragment_tests" is enabled (this is not strictly defined in the GL specs). If coverage is post-z test, then all is good, if it is pre-z test, then there is the possibility that the fragment shader will be executed even if the associated "non-MSAA sample" is occluded. But at least in this state one of the samples in the pixel is non-occluded. Can attempt to correct for this case later by storing the associated lower 8-bits of the gl_PrimitiveID in the g-buffer also.

If the fragment shader does not early exit, then the fragment shader writes G-buffer output using image stores to a set of non-MSAA images (not using ROP).

In theory there is a possible ordering problem here, in that there is no ordering on image stores, but in practice this I'm guessing this isn't a problem on current hardware. Would have to test this to be sure. If it is a problem there are multiple work arounds....

SBAA Paper, SRAA, and Texture-Aligned Deferred Shading (TADS)

2012-01-09T22:05:00.000-08:00

I'm going to describe a new deferred shading technique, Texture Aligned Deferred Shading (TADS) below, but first some notes on the SBAA Paper by Marco Salvi and Kiril Vidimce and SRAA,

(1.) SBAA has two rendering passes. The paper describes the first drawing pass as similar to a z pre-pass, but leaves out some important details. Z pre-pass can on many GPUs be rendered at 2x the fill rate, and often Z pre-pass can be lower poly conservative proxy geometry, etc. SBAA requires full geometry for both passes. Two full geometry passes is likely a bad idea for tessellated geometry (or really high polygon non-tessellated geometry) and bad for the number of required draw calls. A single pass solution is highly desired.

(2.) SBAA uses the same primitiveID trick mentioned in the NVIDIA SIGGRAPH presentation on SRAA. Matthaus and I came up with the primitiveID option for SRAA as an optimization, but didn't have time to fully explore the best solution when mixed with tessellation as primitiveID is per patch instead of per triangle. When used with vector displacement, likely one would want to create a modified hash value in the domain shader for each triangle based on primitiveID and some parametric coordinates.

Side note on SRAA, should be able to do Matthaus's SRAA in one pass with DX11 and surface writes. In the MSAA pass which generates primitiveID (or something better for tessellation) also write the G-buffer, except early exit in the pixel shader when the coverage for the pixel doesn't intersect the "center sample" used for SRAA. For pixels which intersect the center sample, write out the G-buffer using surface writes.

(3.) SBAA has the same problem MSAA has, screen-aligned shading, or pre-shading filtering, is the wrong way to shade for moving pictures.

Texture Aligned Deferred Shading (TADS)

I'm describing this here, because right now I don't have time to actually try this...

Ideally for visual quality per number of shaded samples, shading would be aligned to the texels used to texture shaded surfaces. Filtering, to re-sample shading results to the screen grid, would ideally happen after shading for temporal stability. TADS is a method to do exactly this.

TADS requires "software" virtual texturing (like mega-texture), or some other method where all surface texture data can be index from one position in a set of textures (or texture layers) referred to as the "physical texture" below. This position will be referred to as "physical position" below. For a 8K by 8K sized set of textures, a 32-bit physical position provides only 3-bits of sub-pixel precision (which might not be enough, so might need to use more bits).

TADS pipeline works as follows,

(1.) Only one rendering pass, a modified G-buffer generation pass. Write out {normal, binormal, and physical position}.

(2.) Find unique samples pass. This is a full screen pass which reads the G-buffer's physical position, and computes the number of texture aligned samples which need to be shaded, and writes this information to Z. The number of unique samples to be shaded is packed in the high bits, and a 4-bit bitmask of the samples to shade in the 2x2 texel footprint in the physical texture is packed in the low bits. Each physical position can require between one to four shaded texels, and in most cases neighboring pixels have physical positions which share parts of the shaded texel footprint. To insure samples are not shaded multiple times, each pixel fetches the physical position of the North, North West, and West pixels. Then removes any texels shared by those 3 neighbors from this pixels list of samples to shade.

(3.) Deferred shading passes. Each pixel has a variable number of samples to shade (much like mixing MSAA and deferred shading). To optimize for this case, use depth bounds test to take the Z value generated in (2.) and split the shading up into 4 passes. First pass for pixels which have one sample, second pass for pixels which have two samples, etc. Shaders for each of the 4 passes are optimized to shade {1,2,3,4} samples at the same time respectively. This pass shades texels in physical texture, and uses surface stores to write the results into a "shade results physical texture". AMD hardware doesn't support depth bounds test, so use an alternative method.

(4.) Filtering pass. Each pixel fetches the physical position from the G-buffer, and samples the filtered results from the "shade results physical texture".

Adding in LOD and anisotropy to improve filtering,

(5.) Modify the G-buffer generation pass (1.) and write out gradient vector to provide a line to filter along, and the thickness of this line (LOD or simulation of mip-mapping).

(6.) Modify the shading passes (3.) to output the lower bits of a frame counter into the alpha channel.

(7.) Modify the filtering pass (4.) to fetch the gradient vector and thickness from the G-buffer, and manually sample the anisotropic footprint (simplify to a few samples along the vector) and for LOD a few samples to do a low-pass filter. Compare the filtered alpha value of texture fetches to the frame counter. As the alpha value starts to deviate from the frame counter value, reduce the weight of the associated texture fetch when blending for the final filtered value. The deviation signifies that the filtered result is fetching from a texel which was last shaded in prior frames (temporally stale).

Reducing the shading rate,

(8.) One can reduce the frequency of shading by modifying the "find unique sample" pass (2.) and just marking texels as shaded. The engine will automatically use the prior shaded value for the texel. It is better to mark tiles of texels as shaded instead of random texels. There are a lot of possibilities here, which I will leave for a future post.

Games vs Film Part 2

2012-01-07T08:52:00.000-08:00

Working from the Filtering In PRMan that Brian Karis posted in the comments of the last post, recommendations for film rendering lower bounds,

0.25 Shading Rate: Or on average 4 object space shaded samples per pixel. Seems like this 2x2 super-sample sample rate intuitively makes sense as providing the resolution to transition between 2 texture resolution levels without something similar to mip-mapping. One can see on the first or left shot that lower shading rates start to miss samples on the diagonal,

The 4xSS looks great, and the 2xSS (0.5 shading rate) already looks quite good. Would be interesting to see what the minimum bound on shading rate would be if using mipmap filtering on texture space shading results. Could 2.333xSS (2xSS with mipmaping) be good enough for games? Also might be able to do pure 2xSS with some manual filtering to simulate the mip transition.

8 Pixel Samples: This translates to 8x8 super-sampling on visibility. Without stochastic sampling, this would provide exactly 8 different shades as an edge transitions between exact pixel centers if a poor pixel sized box filter was used during filtering. Or alternatively one could think of this as only 3-bit precision on sub-texel position on GPU filtering (without any magnification). Which in the context of texture space shading would suggest an upper bound of a 8192x8192 source texture with a 32-bit position (13-bits for texel, 3-bits for sub-texel position). Magnification cases clearly would either require more bits of sub-texel position, or trade bits for noise and use stochastic sampling. For comparison with typical GPU real-time rendering, 4xMSAA provides 4 shades as an edge transitions between exact pixel centers, but poor quality on some diagonals.

Smooth Shading Interpolation: GPU MSAA with the standard resolve filter would represent non-smooth shading, as samples on a triangle edge all have the same value. With PRMan those visibility samples can be a "filtered after shading" sample.

Mitchell 4.0 Pixel Filter: The Image Magick docs has some great comparisons (look at the bottom of that page) which show the difference in quality from various filters for down-sampling (copied the images below). Mitchell provides a negative lobe which increases sharpness without having extra lobes like a full sync filter which would add ringing. A cubic filter in contrast has no negative lobes and thus looses some sharpness.

The 4.0 in "Mitchell 4.0" means that the filter kernel width is +/-2 output pixels in size. With 8x8 pixel samples this kernel is 32x32 samples in size. Mitchell 4.0's center positive lobe is around 2 pixels in size. Again this is completely intuitive in that pixel size details will show aliasing in sub-pixel motion. This large filter kernel is a core component of why CG has no aliasing (and looks less sharp) compared to games.

Crazy Thought for Reducing Shader Aliasing

2012-01-01T15:22:00.000-08:00

Just a thought, not sure if this concept or a variation of this concept would work in practice, but seems possible...

In line with the prior post, this is maybe applicable if one was rendering and shading say at a higher frequency than could be observed in the final resolved image. The idea is to attempt to fix some of the problems with standard screen aligned deferred shading by using texture aligned shading (or effectively object space shading).

In the G-buffer creation pass, one would sample textures using point filtering with nearest (biased towards sharper) mipmap, but also write out a screen projected offset of where the actual filtered texel center for the sample would be on the screen under the assumption that all screen neighbors are also point filtered. Likely also have to store out the projected texel radius to aid in correct filtering.

Shading would be computed as standard with deferred shading. After shading there would be a reconstruction pass which would use the current and prior frame's shaded samples and above mentioned offsets to re-construct the proper filtered sample.

One could extend this technique by storing a screen projected edge gradient for each sample during g-buffer creation pass, so that the reconstruction kernel would have an even better idea of texture gradient. This would be of great benefit for things like hair or thin objects for example.

Games vs Film

2012-01-01T01:10:00.000-08:00

2012 through 2014 could be a very interesting time for real-time graphics if game developers realize that even at native 1920x1080 without MSAA they are already effectively super-sampling when compared to movies. In case this is not obvious, go and look at some direct screen grabs from BluRays. For example, here is a crop then the full direct 1080p screen grab from Ironman (click images in this post for full size),

Won't find any pixel width detail in a film, and anything anywhere near a pixel in width looses contrast.

Where are the forum flame fests about how the director "added blur" by not getting the in-focus circle of confusion smaller than a pixel or where the textures used in the Ironman CG shots lack detail?

Compare to a possibly doctored "perfect" driver SSAA override screen grab from Skyrim posted on Neogaf (again click images for full size),

The industry status quo is to push ultra high display resolution, ultra high texture resolution, and ultra sharpness.

IMO a more interesting next-generation metric is can an engine on a ultra-highend PC rendering at 720p look as real as a DVD quality movie? Note, high end PC at 720p can have upwards of a few 1000's of texture fetches and upwards of 100,000 flops per pixel per frame at 720p at 30Hz.

EDIT,

Another interesting comparison which provides a numerical answer, is to down-sample with un-filtered decimation shots from BR movies, and see at what point edge gradients start to match high quality MSAA or CSAA shots from games. For instance with the same Ironman shot, one can use nearest neighbor down-sample at 33% of the original size and get something which has visually similar edge gradients to high quality AA on games,

And the full size un-filtered down-sample,

In practice, it seems as if somewhere between a 50% and 33% nearest neighbor down-sample would be typically required to make BR movies have the sharpness of the high quality AA found in games. Intuitively this makes sense as "50%" is the natural lower bound: detail under 2 pixels in width will have aliasing under motion. Inversely, it seems as if 1080p BR movies typically have somewhere between only 350p and 540p of effective "game resolution". Or alternatively in game AA terms, BR movies get an extra 2x2 or 3x3 OGSSAA but instead of getting resolved to the 350p to 540p resolution, they get a DOF "blur" applied at 1080p...

FXAA 4.0 with 1.78x SSAA

2011-12-25T01:31:00.000-08:00

Below is an example of 2xMSAA (so no AA on foliage and HDR/LDR edges) with 1.78x SSAA and prototype FXAA 4.0 spatial-only algorithm doing the down-sample with sharpness turned off to simulate the larger filter typical of CG film. Temporal FXAA 4.0 without 2xMSAA but with 1.78x SS would look a little better than this, as each output pixel gets roughly 3.56 shaded samples.

8xMSAA roughly same cost as 2xSSAA

2011-12-22T10:16:00.000-08:00

Was looking at some random internet sources comparing various AA modes on a few games and apparently 8xMSAA is around the same cost as 2xSSAA. Both trade the lead spot depending on GPU and title. A very rough ballpark being both run maybe 65-85% slower than NO-AA.

Seems like for games with modern AA solutions, 8xMSAA is not really an important option. For instance 4xMSAA + FXAA 4.0 should easily be much faster and higher quality than 8xMSAA. Or FXAA 4.0 2xSSAA should be similar cost with much better shading quality. Or FXAA 4.0 2xSSAA+2xTSSAA (temporal SS) should be similar cost with much better quality everywhere. Or 4xMSAA + FXAA 4.0 2xTSSAA which will be faster and much higher quality. With all the non-8xMSAA cases taking up less memory too.

Looking forward even 4xMSAA and 2xMSAA are likely to get marginalized in the step towards CG quality as they have some serious problems. For instance it is impossible to do a high quality post process DOF or motion blur after MSAA resolve as pixels end up with samples merged from various depths and post needs to be done pre tone-mapping while resolve needs to be post tone-mapping. In the race for CG quality, my bet is on 2xTSSAA with scaling in quality via fractional and non-fractional SSAA.

FXAA 4.0 Development Update (Stills)

2011-12-22T01:49:00.000-08:00

Similar to prior posts, this post talks about R&D on FXAA 4.0: algorithm is subject to change before release, and all shots are work in progress...

The Spatial-Only Algorithm

Most of the heavy R&D is over and the result is a new spatial-only algorithm for FXAA 4.0. Note, the only reason this algorithm exists is because it is also the base for the temporal algorithm. The spatial-only algorithm lives somewhere around FXAA 3.11 PC at a medium quality setting, but at the perf a little above FXAA 3.11 Console. Have not profiled yet, so do not actually know.

The aim here is performance for huge resolution rendering or super-sampling. The spatial-only FXAA can also resize just like the temporal version (but lower quality), and it will also support in-line super-sample down-sampling, taking 2 or more FXAA filtered results per output pixel to support ultra high quality in one pass. Super-sample stills will need to wait until another post.

However, !/$ the spatial+temporal solution is the way to go, easily 4x higher quality for nearly same cost.

Spatial-Only Algorithm Enlarge + Sharpen

Similar to the interjection mods, added a built-in classic "halo-ing" sharpen into FXAA 4.0. Below is an example enlarging and sharpening (set at a very subtle 50%) with the spatial-only algorithm. The sharpening can amplify edge noise, EDIT: edge noise and poor quality in the shot below is a result of me using the settings for native resolution instead of the correct settings for enlarging (oops it was 4 am),

Another example this time with a really fail case for bilinear resize, and sharpness turned up,

Spatial-Only Algorithm Sharpen Without Resize

With the early exit off and edge detection off, setting sharpening to 100% will maintain detail even when the algorithm filters highly detailed textures,

Spatial-Only Algorithm Sharpen Over MSAA

The algorithm plays nice over MSAA even when sharpening is set to 100%,

AA at Different Scales

As a side effect of the algorithm functioning when non-pixel aligned, apparently the algorithm can sometimes anti-alias in blurred content. For instance, the left below is the source shot, where the red arrow marks aliasing caused by bilinear upsample of downsampled DOF. The right is the FXAA 4.0 spatial-only algorithm with the early exit removed and sharpening turned off (note the notches got removed),

Here is another example,

GDC 2012: Rendering Technology of Prey 2

2011-12-21T17:07:00.000-08:00

Brian Karis of Human Head Studios will be covering Rendering Technology of Prey 2 at GDC 2012,

"This session presents the major graphical challenges and innovations behind the Prey 2 renderer. I will describe novel additions and optimizations to existing algorithms and the gritty details needed to implement them gained from production experience. I will also cover new approaches used to better approximate offline CG quality. Topics will include occlusion culling, virtual texturing, physically-based rendering, deferred shading, global illumination, screen space approximations, and post processing. Finally I will give a detailed look at the architecture of the renderer and how these many complex systems fit together to build the final rendered image."

FXAA 4.0 Will Have New Spatial-Only Algorithm

2011-12-12T02:49:00.000-08:00

FXAA 4.0 is still a work in progress. No stills this time, and much of the prior shots would have changed by now. Updates as of this weekend,

(1.) FXAA 4.0 will have a new spatial only algorithm for those who cannot integrate with the temporal filtering. There will be multiple quality/perf levels. The spatial algorithm is part of the temporal algorithm. It contains no searching, supports dynamic output resolution, and works really well mixed with hardware filtering (like MSAA).

(2.) I've come up with a new aliasing detection filter, one that completely eliminates filtering from areas that have no aliasing. As prior this only depends on color input, uses same texture fetches as prior, but now it can be set to filter edges only, and avoid texture detail. This will be on sliders for developers to hand tune. Bottom line, no more loss of detail, as was a common complaint with FXAA 3.11, and no need for a depth based edge detection pass.

(3.) I'm very close to managing to extend both the quality and edge length of the 9-fetch spatial algorithm (same number of texture fetches as FXAA 3.11 Console) to almost match the default medium FXAA 3.11 PC results. Managed this via some very smart limiting of the filter results.

Kotaku: What is FXAA

2011-12-09T17:52:00.000-08:00

My favorite gaming news site, Kotaku, just posted this post on FXAA.

Keloid Trailer - Short Film by BLR

2011-12-09T11:37:00.001-08:00

Keloid Trailer - A Short Film by BLR from BLR_VFX on Vimeo.

FXAA 4.0 Stills and Features Part 2

2011-12-06T18:57:00.000-08:00

All prior posts showed FXAA 4.0 with the sharpening knob turned off.

Sharpening

FXAA 4.0 has as a sharpness slider. FXAA 4.0 applies a special kind of sharping which can be applied in a single pass in conjunction with super-resolution. Here is a bilinear 2x2 up-sample,

Now FXAA 4.0 Super-Resolution set to sharp, note the sharpening is different than an un-sharp mask. So extra sharpening with a classic linear filter can be applied over this if desired.

Sharpening - Without MSAA Input

Sharpening without MSAA input reduces the AA quality slightly. Here is bilinear 2x,

Here is what this would look like at native resolution,

And FXAA 4.0 2x super-resolution with sharpening, note in this case the render only renders 25% of the pixels/frame as the image above. The sharpness of this image is limited by FXAA 4.0 only one sample from the current frame and one sample from the prior frame per a 2x2 group of output pixels, and needing to do AA also,

Another example, here is a native no-AA without resizing,

And the same with FXAA 4.0 at the sharpest setting,

FXAA 4.0 Stills and Features

2011-12-06T00:42:00.001-08:00

Source images ripped from the PC high resolution screen shots thread on neogaf. FXAA 4.0 is in prototyping now, so all output subject to change as development progresses. All FXAA 4.0 processed frames generated with the actual FXAA 4.0 shader which takes just 9 texture samples per output pixel. As FXAA is a temporal algorithm, all input shots were decimated by 2x without filtering as the best option to simulate expected FXAA input, taking the upper left sample of the 2x2 pixel quad as the previous frame, and the lower right sample of the 2x2 pixel quad as the current frame.

FXAA Goals

FXAA 4.0 is a combination of a demosaicing/deinterleaving, super-resolution, and anti-aliasing algorithm all in one tiny full screen pass. The goals of FXAA 4.0 are,

(1.) Aid in the support of film quality visuals for games.
(2.) Ultra high quality/cost anti-aliasing.
(3.) Decouple rendering resolution from display resolution.

Hyper-Sharpness

Using the term "hyper-sharpness" to describe the quality of typical GPU renderings where the image is usually box filtered and contains high frequency features which can only be reproduced when the features are perfectly aligned to the pixel grid. Hyper-sharpness is exactly why our font rendering algorithms snap features to pixel centers. Typical hyper-sharpness results in temporal aliasing under motion because the features fade in and out as the image moves.

Below is an example of a shot which was down-sampled using pure decimation without filtering, leaving a hyper-sharp image. The source was a larger perfect anti-aliased screen-shot of Rage,

Regardless as to how sharp this image looks, no film could ever be played in a theater with this amount of temporal aliasing that this would exhibit. Also a game post process cannot sharpen this image using a simple unsharp-mask without amplifying image aliasing. Notice the increase in "stair-step" or breakup of highlights on the railings towards the lower part of the image (don't confuse with the rope at the top of the image),

Here is the source image decimated without filtering, then running through FXAA 4.0 set to remove all possible temporal aliasing and simulate film,

Note the image can be sharpened in post without any addition of aliasing,

Super-Resolution

Taking the base image after decimation without filtering, then up-sampling 2x in width and height with standard bilinear texture filtering, looks as awful as expected,

FXAA 4.0 has ability to re-target at a larger-than-rendered resolution for free during its single full screen pass. Below is a shot of using FXAA 4.0 to up-sample 2x,

Note FXAA 4.0 uses temporal super-sampling, so every other frame renders at a slightly different sub-pixel offset. FXAA 4.0 reconstructs each frame from the current and prior frame, using the extra sample per pixel for a higher quality up-sampling. In the above shot, only one sample per a 2x2 group of pixels gets rendered per frame.

Fractional Super-Resolution Works

Here is standard bilinear with 3/2x in width and height,

Here is FXAA 4.0 with 3/2x in width and height,

Super-Resolution Filtering Quality

Here is a new source image, same style of non-filtered decimation. Notice the aliasing on the grate on the bottom,

Now the source processed by FXAA 4.0, notice how FXAA 4.0 will correctly reconstruct the grate on the bottom,

The same reconstruction holds up even when FXAA 4.0 is re-targeting to a much larger resolution. For example 3x in width and height,

Not sure why anyone would want 3x enlargement except maybe shadow filtering (which is still on my todo list). Anywhy, at these levels of enlargement, FXAA 4.0 will add some noise which can be seen on the white dots.

Classic Workloads

FXAA 4.0 still works on classic game style workloads. No AA,

FXAA 4.0,

Big FXAA Update Soon

2011-12-03T02:48:00.000-08:00

Have not had the time to seriously work on FXAA since 3.11 was released, but managed to make an incredible amount of progress today getting one of the temporal algorithms closer to release...

What's In the Next Release of FXAA

TEMPORAL ALGORITHM

The new algorithm makes use of 2x temporal super-sampling for much higher base quality with none of the disadvantages of the prior only spatial FXAA 3.11. Render even frames with a quarter-pixel shift to the upper left and odd frames with a quarter-pixel shift to the lower right.

NEW PIPELINE

But still a single full screen pass. The "Output Frame" and "Feedback Frame" are native resolution, while "Rendered Frame" and "Motion Vectors" can be a different size from native resolution.

     Rendered Frame -> o------o -> Output Frame --> HUD -> Swap
     Motion Vectors -> | FXAA |  
 o-> Feedback Frame -> o------o -> Feedback Frame >-o
 |                                                  |
 o--------------------------------------------------o

RUN-TIME VARIABLE RESOLUTION RENDERING

Rendered input frame can be either larger or smaller than native resolution. FXAA automatically does the re-sampling for free. Supports any fractional resolution (does not have to be a multiple of native size). Supporting up to ultra-high-quality 4x OGSSAA all the way down to 1/4x native area. Game can dynamically adapt rendering resolution to maintain desired frame rate.

MIX MSAA + OGSSAA + FXAA 2xTSSAA

This version of FXAA is designed to work really well with already filtered input. The new algorithm is based on a pure filtering approach, without any "searching" which would constrain the algorithm to a pixel grid or would cause problems when MSAA "hides edges". Use MSAA or CSAA during normal rendering and just pipe that into the new FXAA. FXAA enables free mixing of variable resolution hardware AA like MSAA, OGSSAA (ordered grid super-sampling), and TSSAA (temporal super-sampling). Game can scale to any quality level.

REALLY DAMN FAST

The current high quality setting uses only 9 texture fetches per pixel. This is the same number of texture fetches taken by FXAA 3.11 Console on PS3. Have not ported the new FXAA to PS3 or 360 yet, but there will be a direct console port!

Assuming texture bound on PC, a mid-range 560 Ti is likely able to process a 1080p frame in around 0.36 ms. Still modifying the algorithm, so don't have final numbers yet.

Random Demo Tubes

2011-11-27T16:49:00.000-08:00

Guerrilla Publications Link

2011-11-21T13:57:00.000-08:00

Guerrilla Publications

Interactive, Multiresolution Image-Space Rendering for Dynamic Area Lighting

2011-11-20T17:30:00.000-08:00

Interactive, Multiresolution Image-Space Rendering for Dynamic Area Lighting

Understanding WIN32 GetRawInputBuffer()

2011-11-20T14:14:00.000-08:00

First, Microsoft's UI update of their online WIN32 docs is awesome, and a lot of the online documentation has gotten much better over the years. However the docs for GetRawInputBuffer() are fail.

On XP at least (have not tested Vista/Win7/Win8 yet),

The entire RawInput system is built on-top of WM_INPUT events in the standard Window's message queue. When processing messages, WM_INPUT messages must not be removed in order to use GetRawInputBuffer() as GetRawInputBuffer() will use then remove those events.

In my usage case I'm doing this to drain out the message queue without disrupting GetRawInputBuffer(), and relying on the fact that for events like WM_ACTIVATEAPP (ALT+TAB), PeekMessage() automatically calls the WinProc() without returning the message to the application directly,

while(1) {
  if(PeekMessage(&msg, 0, 0, WM_INPUT-1, PM_NOYIELD | PM_REMOVE) == 0) break;
  DefWindowProc(msg.hwnd, msg.message, msg.wParam, msg.lParam); }
while(1) {
  if(PeekMessage(&msg, 0, WM_INPUT+1, 0xFFFFFFFF, PM_NOYIELD | PM_REMOVE) == 0) break;
  DefWindowProc(msg.hwnd, msg.message, msg.wParam, msg.lParam); }

GetRawInputBuffer()

Docs on GetRawInputBuffer() are more than confusing, and why do they use malloc() in the samples? Here is a less confusing example,

// Some temp global buffer, 1KB is overkill.
static uint64_t rawBuffer[1024/8];
...
// Then in some function,
const uint32_t bytes = sizeof(rawBuffer);
// Loop through reading raw input until no events are left,
while(1) {
  // Fill up buffer,
  int32_t count = GetRawInputBuffer((PRAWINPUT)rawBuffer, &bytes, sizeof(RAWINPUTHEADER));
  if(count <= 0) return;
  ...
  // Process all the events, 
  const RAWINPUT* restrict raw = (const RAWINPUT* restrict) rawBuffer;
  while(1) {
    // Process raw event.
    ...
    // Goto next raw event.
    count--;
    if(count <= 0) break;
    raw = NEXTRAWINPUTBLOCK(raw); } }

#AltDevBlogADay: Cold code - the last bastion of code optimization?

2011-11-03T08:33:00.000-07:00

Awesome post by Andy Thomason: #AltDevBlogADay: Cold code - the last bastion of code optimization?

Voxatron Debut

2011-10-31T11:40:00.000-07:00

The Humble Voxatron Debut

Exclusive CPU Core Access

2011-10-31T09:44:00.001-07:00

One of the top items on my OS feature wishlist: Exclusive CPU Core Access.

Concept

Enables a full screen application like a game to have exclusive access to CPU cores. Removes preemption on those cores. One important step towards the path of solving the latency problem on non-console devices.

How to Implement

The OS reuses it's CPU Hot-Plug functionality to migrate threads off the cores, then grants the application exclusive access. When the app is back-grounded or exits, the CPU cores are reclaimed.

Why Not?

The modern OS deserves a redesign, the concept of sharing a single processor core is outdated as all devices are going to be many-core from now on. Can certainly have both hard real-time and time-sharing in the same OS on the same hardware.

64-bit ARM

2011-10-28T15:48:00.000-07:00

64-bit ARM is going to be awesome. Tiny bit of info starting to get public...

ARMv8 Technology Preview

Few highlights,

(1.) 31 general purpose 64-bit integer registers.
(2.) 32 128-bit SIMD registers.
(3.) SIMD adds float min/max, and nice rounding control.
(4.) Conditionals now limited (branches, compares, and selects).

Random AA Thoughts Continued

2011-10-27T23:32:00.000-07:00

OGSSAA + FXAA

Some users have been figuring out how to mix OGSSAA with FXAA, and the results are really nice. Here is a crop from Alice showing 2x2 OGSSAA with FXAA. See more images on the Neogaf thread where this crop was taken from.

Would be great if games supported this natively as it is trivial to implement. Just allow the user to render the game larger than display resolution, apply FXAA, then down-sample to the display. Would be useful to provide 2x and 4x OGSSAA+FXAA (Nx = N times the screen area), and for the extreme up to 8x OGSSAA+FXAA.

Great AA really changes the visual quality of a game.

AMD Recommends FXAA on BF3?

This is entertaining,

AMD Recommends Using NVIDIA's FXAA in BF3

I'd use both MSAA and FXAA for BF3, as MSAA helps with sub-pixel details and temporal aliasing.

Texture or Object Space Shading

Super-sampling is a requirement for good AA when doing screen space shading. The more rough and specular a lit surface is, the more samples are required. As the surface moves even with small sub-pixel offsets, shading drastically changes, and for temporal coherence and to reduce temporal aliasing, many samples are required.

This is less so with texture or object space shading as the shading is done in a spatially coherent way on the surface of an object.

One of the primary arguments against object space shading is the feeling that it would require more shaded samples for the same quality as screen space shading. For example, likely some occluded surfaces would get shaded. One could envision shading directly into a virtual texture physical page cache (which is larger than the resolution of the screen).

However I have a feeling that object space shading would actually require fewer shaded samples because,

(1.) Shading could be done lower than the refresh rate. More distant surfaces don't change shading that fast, why recompute each frame?

(2.) Object space shading would be ideal in thread occupancy on GPUs, work in a perfect size tile all the time. Only tiles with visible or part visible texels need to be shaded.

(3.) As the prior post hinted, to remove temporal aliasing requires a low-pass filter which filters out somewhere between 1 to something smaller than 2 pixel sized features (depending on desired sharpness). This reduces the required number of shaded texels.

(4.) Not all surfaces need a uniform density of shading. For example dull smooth surfaces, or surfaces which need more shading density in one direction. For example, see how Rage as an extreme case has variable texel density.

There are some other advantages of object space shading,

(1.) It decouples shading rate from screen resolution. The object space shading cache could be dynamically resized to fit the frame budget.

(2.) It simplifies AA, as the frame is then forward rendered and could easily use traditional MSAA without the heavy memory burden of deferred rendering.

Sharpness vs Temporal Aliasing

2011-10-26T20:33:00.000-07:00

This will come as a shock to most people, but for a given resolution, CG movie quality graphics is not obtainable without a loss of sharpness compared to our current game graphics practices.

Should be Obvious

There is a direct trade off between sharpness and temporal aliasing. This should be clear, if not, here is an example image,

The first two lines show the same single pixel on/off pattern, just the second line starts at a half pixel offset. As the pattern moves it horizontally it will flicker between being visible and invisible, a perfect example of temporal aliasing.

The second two lines show a pattern of a two pixel on/off pattern, with the second line again starting at a half pixel offset. As the pattern moves horizontally the pattern will remain visible.

CG Movie vs Game Graphics, the Double Standard

"The anti-aliased image does not look as sharp!"

Of course, because if the anti-aliased image does not filter out pixel sized data then some amount of temporal aliasing will remain in a moving picture.

Temporal aliasing is not acceptable for CG in movies, as viewers would complain that their eyes are bleeding. However for game graphics for anything with side-by-side comparisons, some percentage of people will complain of a lack of sharpness if any anti-aliasing technique removes the temporal aliasing.

GPU Hardware AA

For edges and MSAA, a 1 pixel box filter (the default resolve operation) will never filter out all temporal aliasing. The filter needs to be larger than one pixel. Even for single textured un-shaded surfaces with one sample, the default texture filtering can also leave temporal aliasing.

Another Visual Example

Rage with MSAA and virtual 4x4 OGSSAA (I resized an image grabbed from a Neogaf thread). This first image shows no filter before down-sampling via decimation. Limit in terms of sharpness and most aliased.

This second image uses a smaller than destination pixel size filter before down-sampling. This is sharper than a default MSAA resolve, and would still show extreme temporal aliasing.

This is roughly similar to 4x4 OGSSAA, and this would still display temporal aliasing, and is already less sharp.

Same image but with a large enough filter to remove temporal aliasing,

The same as above but sharpened with a pixel sized unsharpmask set at 50%,

Rage Jackal Canyon Tube

2011-09-29T09:03:00.000-07:00

Akiko by Flopi 4K Tube

2011-09-07T23:49:00.000-07:00

Parallel Programming With Clones

2011-09-07T01:24:00.001-07:00

This is a very simple way to do CPU parallel programming which automatically handles load ballancing without requiring atomic operations. Start the same program running on N threads in parallel sharing the same memory space. Write the program such that,

(1.) The program can skip over work which it knows has been completed by another clone.

(2.) That clones will always write the same results to memory for data.

Under these rules, any thread can be preempted and the program continues, when the thread returns it quickly skips ahead to the front of computation. Threads trade sometimes redoing work for avoiding expensive communication and synchronization. A simple example,

// frameCounter advances by one each frame
uint32_t frame; 
...
// check if should start task
volatile uint32_t taskStart; 
volatile uint32_t taskEnd;
if(taskStart != frame) 
{ taskStart = frame; DoTask(); MemoryBarrier(); taskEnd = frame; }
...
// do some other work
...
// verify task was completed
// before using results of task
if(taskEnd != frame) 
{ DoTask(); MemoryBarrier(); taskEnd = frame; }
...
// use results of DoTask()

Keep data double buffered and setup a simple barrier construct so that threads cannot fall over two frames behind. Also insure that threads are running enough out of phase so that they do not all always attempt to do all the same work.

For exceptions to the rule, keep track of a threadIdx for each thread (perhaps in thread local storage). External interfaces need to be predicated to only run in one thread, for example OpenGL access requires that calls be predicated to only the thread which owns the active GL context.

Hawken Tube

2011-08-31T00:28:00.000-07:00

Android Wish List

2011-08-24T21:22:00.000-07:00

Just in case this isn't clear, these are my personal opinions only!

(1.) SUPPORT OPENGL

Yeah GL without the ES. ES is even limiting on the current generation of tiny GPUs designed for low power, what about future rounds from all the major GPU vendors.

(2.) SUPPORT FULL SCREEN MODE

Low-power, high-efficiency, access to run full screen. Game aware device orientation. Game renders into back buffer, OS scans out from the game's back buffer without the normal expensive copy and UI composite step. Support hardware scaling scanout, for efficient up-sampling. Game provides user an option for low-power mode where the game can run at a lower resolution taking advantage of the scaling scanout to save battery life.

What about touch screen button bar?

Setup the full-screen API such that the application can resize the display, and provide a way for the application to query the size/position of the required UI bar. The application draws as normal leaving the UI bar region alone. And during the swap call, have a way for the Android UI to redraw just the bar.

(3.) SUPPORT VARIABLE FRAME RATE

Along with full screen mode, only scan out the display at the rate at which the application calls swap. Smart games supporting user controlled "power saving" mode can adapt frame rate to save battery life.

(4.) SUPPORT FULL NATIVE DEVELOPMENT

Complete the API layer such that Java is not needed for AAA style games. So native input, native sound, native full screen, native graphics, and native support for Android Market in-app purchases and license verification requests. The entire Java development layer is a nightmare of complexity. Want only C/C++ with inline assembly. Also inefficient CPU overhead through the OS, libraries, UI, etc is fail for battery life, and fail on mobile devices which share power between the CPU and GPU.

(5.) SUPPORT NATIVE DEVELOPMENT ON THE DEVICE

How funny is it when you can use a Linux device that is slower and less capable than an Android Tablet to build your Android app, and you still cannot simply use your Android Tablet to build your app or develop? Or when someone at a coffee house has an Eee Pad Transformer (Android Tablet which has a keyboard and mouse pad) and needs to bring a 2nd notebook simply to develop. I want my Android computer which can be used to produce content, not my Android brick that can only consume content.

CLOSING

With the top players like Google, Microsoft, and Apple with a range of 150 to near 350 billion market cap, there is no lack of capital to fund greatness. Yeah, the race is on. All these companies have loads of talented individuals, but who has the balls to take the mobile platform to the next level?

PixelJunk SideScroller Tube

2011-08-19T08:28:00.000-07:00

Advances In Realtime Rendering in 3D Graphics and Games

2011-08-18T08:53:00.000-07:00

Advances In Realtime Rendering in 3D Graphics and Games

Prey 2 Stills

2011-08-17T10:12:00.000-07:00

Prey 2 Stills on Gamersyde

Evoke 2011 Tubes

2011-08-14T22:09:00.000-07:00

Mikkel Gjoel's Brink OpenGL Talk

2011-08-12T18:01:00.000-07:00

Brink Preferred Rendering With OpenGL

Great talk. Definitely when going to bindless vertex attributes (was 5%-10% reduction in frame time on CPU from slides) need to transform per-frame temporaries from allocate every frame to manual allocate once then roll your own dynamic allocation inside that buffer.

FXAA 3.11 Bug Fixes For 360

2011-08-12T17:43:00.000-07:00

FXAA 3.11 Source Updated

Bug fixes for 360 version starting on line 1425, here is what the new correct source is,


    float4 rgbyB = rgbyN2 + rgbyP2 + rgbyA * 0.5;

/*--------------------------------------------------------------------------*/

    float4 rgbyR = ((FxaaLuma(rgbyB) - lumaMax) > 0.0) ? rgbyA : rgbyB; 

    rgbyR = ((FxaaLuma(rgbyB) - lumaMin) > 0.0) ? rgbyR : rgbyA; 

    return rgbyR; }

Assembly Tubes

2011-08-07T18:50:00.000-07:00

Rage Uprising Tube

2011-08-04T13:35:00.001-07:00

Note, FXAA Does Not Use Blending

2011-08-02T17:57:00.000-07:00

I should have been more proactive about documenting FXAA integration for people new to the shader. FXAA runs in two ways,

(1.) DEFAULT. FXAA takes a texture (the screen image prior to drawing HUD or text) as input, then outputs WITHOUT blending to another render target. FXAA writes all pixels.

(2.) DISCARD. Using "#define FXAA_DISCARD 1". This only works on OpenGL for the PC taking advantage of extensions which enable the ability to bind the current frame buffer object as a readable texture. This option technically should not work with "#define FXAA_GREEN_AS_LUMA 1", but might work. This option also does not use blending, and also does not write all pixels.

Fixed a few typos in FXAA 3.11

2011-07-26T16:12:00.000-07:00

Changes reflected in the online source,

(1.) Fixed 360 typos where float4 should have been float.
(2.) Made "#define FXAA_PS3 1" work without also "#define FXAA_HLSL_3 1".
(3.) Set "#pragma regcount 7" on PS3 shaders for better texture cache perf.

Latest feedback,

(1.) PS3 with early exit path is 1.2 ms (with regcount 7).
(2.) However with FXAA_GREEN_AS_LUMA is 1.4 ms (adds 1 clk cycle to shader).

Will attempt to address the FXAA_GREEN_AS_LUMA perf for PS3 in FXAA 3.12 and attempt to take advantage of TextureRemap.

Voxel Pipe

2011-07-26T10:03:00.000-07:00

Voxel Pipe Paper

Including a teaser pair of images for a future voxel based real-time GI paper,

"Finally, in Figure 5 we show a proof of concept of a real-time global illumination system that we plan to disclose in the near future. The system re-lies on our voxelization pipeline to create a proxy of the scene geometry that is used to trace incoherent rays. The proxy is rebuilt every frame, allowing support of fully dynamic geometry, materials and lighting."

Specular Showdown in the Wild West

2011-07-22T09:38:00.000-07:00

Awesome post by Stephen Hill: Specular Showdown in the Wild West

FXAA 3.11 Released

2011-07-21T22:55:00.000-07:00

Download FXAA 3.11 Source

"... as for the next release, not sure when, probably not for a while, I'm really busy right now on other projects ..."

That comment was just yesterday, and actually I'm serious this time, likely not going to update for a while, but had to at least incorporate all the great suggestions I've been getting from developers, hence the FXAA 3.11 release.

FXAA Now Supports Using Green Instead of Luma

#define FXAA_GREEN_AS_LUMA 1

And massively lower the following inputs towards zero (or set to zero),

FxaaFloat fxaaQualityEdgeThresholdMin,
FxaaFloat fxaaConsoleEdgeThresholdMin,

This define enables developers to avoid packing luma into the alpha channel when processing non-linear RGB data. The compromise here is that colors which lack any green (like pure blues, reds, or just a mix of blues and reds) will not get any AA. This is likely ok, because often these blues and reds border something which has a non-zero green channel so at least one side will get AA, and pure hues are often rare in practice.

A second compromise is that the early exit path will be less effective since the min threshold will have to be lowered in order to insure AA is not turned off on colors which contain only a very minor amount of green.

Cleaned up FxaaPixelShader() Inputs and Tuning Knobs

Unified the inputs to all version of the shaders so that integration is easier. For those not using all the shaders, simply use zeros when inputs are not needed. Look on line 721 in the source for all the input docs (or see below).

All the major tuning knobs, with the exception of 2 PS3 settings and FXAA_QUALITY__PRESET, are now shader inputs instead of defines as they were in the last version.

Note, the constants used for FXAA Console are now different (they now match the needs of the optimized 360 version) so some C code will need to change when integrating FXAA 3.11.

Quick conversion from FXAA 3.10 (with old defines in comments),

FxaaFloat4 FxaaPixelShader(
    pos,
    posPos,
    tex,
    tex,                        // these will be different for 360
    tex,                        // these will be different for 360
    rcpFrame,
    fxaaConsoleRcpFrameOpt,     // this is new
    fxaaConsoleRcpFrameOpt2,    // this is new 
    fxaaConsole360RcpFrameOpt2, // this is new
    0.75,                       // FXAA_QUALITY__SUBPIX
    0.166,                      // FXAA_QUALITY__EDGE_THRESHOLD
    0.0833,                     // FXAA_QUALITY__EDGE_THRESHOLD_MIN
    8.0,                        // FXAA_CONSOLE__EDGE_SHARPNESS
    0.125,                      // FXAA_CONSOLE__EDGE_THRESHOLD
    0.05,                       // FXAA_CONSOLE__EDGE_THRESHOLD_MIN
    fxaaConsole360ConstDir      // this is new
)

New input docs (also in source code),

FxaaFloat4 FxaaPixelShader(
    //
    // Use noperspective interpolation here (turn off perspective interpolation).
    // {xy} = center of pixel
    FxaaFloat2 pos,
    //
    // Used only for FXAA Console, and not used on the 360 version.
    // Use noperspective interpolation here (turn off perspective interpolation).
    // {xy__} = upper left of pixel
    // {__zw} = lower right of pixel
    FxaaFloat4 fxaaConsolePosPos,
    //
    // Input color texture.
    // {rgb_} = color in linear or perceptual color space
    // if (FXAA_GREEN_AS_LUMA == 0)
    //     {___a} = luma in perceptual color space (not linear)
    FxaaTex tex,
    //
    // Only used on the optimized 360 version of FXAA Console.
    // For everything but 360, just use the same input here as for "tex".
    // For 360, same texture, just alias with a 2nd sampler.
    // This sampler needs to have an exponent bias of -1.
    FxaaTex fxaaConsole360TexExpBiasNegOne,
    //
    // Only used on the optimized 360 version of FXAA Console.
    // For everything but 360, just use the same input here as for "tex".
    // For 360, same texture, just alias with a 3nd sampler.
    // This sampler needs to have an exponent bias of -2.
    FxaaTex fxaaConsole360TexExpBiasNegTwo,
    //
    // Only used on FXAA Quality.
    // This must be from a constant/uniform.
    // {x_} = 1.0/screenWidthInPixels
    // {_y} = 1.0/screenHeightInPixels
    FxaaFloat2 fxaaQualityRcpFrame,
    //
    // Only used on FXAA Console.
    // This must be from a constant/uniform.
    // This effects sub-pixel AA quality and inversely sharpness.
    //   Where N ranges between,
    //     N = 0.50 (default)
    //     N = 0.33 (sharper)
    // {x___} = -N/screenWidthInPixels  
    // {_y__} = -N/screenHeightInPixels
    // {__z_} =  N/screenWidthInPixels  
    // {___w} =  N/screenHeightInPixels 
    FxaaFloat4 fxaaConsoleRcpFrameOpt,
    //
    // Only used on FXAA Console.
    // Not used on 360, but used on PS3 and PC.
    // This must be from a constant/uniform.
    // {x___} = -2.0/screenWidthInPixels  
    // {_y__} = -2.0/screenHeightInPixels
    // {__z_} =  2.0/screenWidthInPixels  
    // {___w} =  2.0/screenHeightInPixels 
    FxaaFloat4 fxaaConsoleRcpFrameOpt2,
    //
    // Only used on FXAA Console.
    // Only used on 360 in place of fxaaConsoleRcpFrameOpt2.
    // This must be from a constant/uniform.
    // {x___} =  8.0/screenWidthInPixels  
    // {_y__} =  8.0/screenHeightInPixels
    // {__z_} = -4.0/screenWidthInPixels  
    // {___w} = -4.0/screenHeightInPixels 
    FxaaFloat4 fxaaConsole360RcpFrameOpt2,
    //
    // Only used on FXAA Quality.
    // This used to be the FXAA_QUALITY__SUBPIX define.
    // It is here now to allow easier tuning.
    // Choose the amount of sub-pixel aliasing removal.
    // This can effect sharpness.
    //   1.00 - upper limit (softer)
    //   0.75 - default amount of filtering
    //   0.50 - lower limit (sharper, less sub-pixel aliasing removal)
    //   0.25 - almost off
    //   0.00 - completely off
    FxaaFloat fxaaQualitySubpix,
    //
    // Only used on FXAA Quality.
    // This used to be the FXAA_QUALITY__EDGE_THRESHOLD define.
    // It is here now to allow easier tuning.
    // The minimum amount of local contrast required to apply algorithm.
    //   0.333 - too little (faster)
    //   0.250 - low quality
    //   0.166 - default
    //   0.125 - high quality 
    //   0.063 - overkill (slower)
    FxaaFloat fxaaQualityEdgeThreshold,
    //
    // Only used on FXAA Quality.
    // This used to be the FXAA_QUALITY__EDGE_THRESHOLD_MIN define.
    // It is here now to allow easier tuning.
    // Trims the algorithm from processing darks.
    //   0.0833 - upper limit (default, the start of visible unfiltered edges)
    //   0.0625 - high quality (faster)
    //   0.0312 - visible limit (slower)
    // Special notes when using FXAA_GREEN_AS_LUMA,
    //   Likely want to set this to zero.
    //   As colors that are mostly not-green
    //   will appear very dark in the green channel!
    //   Tune by looking at mostly non-green content,
    //   then start at zero and increase until aliasing is a problem.
    FxaaFloat fxaaQualityEdgeThresholdMin,
    // 
    // Only used on FXAA Console.
    // This used to be the FXAA_CONSOLE__EDGE_SHARPNESS define.
    // It is here now to allow easier tuning.
    // This does not effect PS3, as this needs to be compiled in.
    //   Use FXAA_CONSOLE__PS3_EDGE_SHARPNESS for PS3.
    //   Due to the PS3 being ALU bound,
    //   there are only three safe values here: 2 and 4 and 8.
    //   These options use the shaders ability to a free *|/ by 2|4|8.
    // For all other platforms can be a non-power of two.
    //   8.0 is sharper (default!!!)
    //   4.0 is softer
    //   2.0 is really soft (good only for vector graphics inputs)
    FxaaFloat fxaaConsoleEdgeSharpness,
    //
    // Only used on FXAA Console.
    // This used to be the FXAA_CONSOLE__EDGE_THRESHOLD define.
    // It is here now to allow easier tuning.
    // This does not effect PS3, as this needs to be compiled in.
    //   Use FXAA_CONSOLE__PS3_EDGE_THRESHOLD for PS3.
    //   Due to the PS3 being ALU bound,
    //   there are only two safe values here: 1/4 and 1/8.
    //   These options use the shaders ability to a free *|/ by 2|4|8.
    // The console setting has a different mapping than the quality setting.
    // Other platforms can use other values.
    //   0.125 leaves less aliasing, but is softer (default!!!)
    //   0.25 leaves more aliasing, and is sharper
    FxaaFloat fxaaConsoleEdgeThreshold,
    //
    // Only used on FXAA Console.
    // This used to be the FXAA_CONSOLE__EDGE_THRESHOLD_MIN define.
    // It is here now to allow easier tuning.
    // Trims the algorithm from processing darks.
    // The console setting has a different mapping than the quality setting.
    // This only applies when FXAA_EARLY_EXIT is 1.
    // This does not apply to PS3, 
    // PS3 was simplified to avoid more shader instructions.
    //   0.06 - faster but more aliasing in darks
    //   0.05 - default
    //   0.04 - slower and less aliasing in darks
    // Special notes when using FXAA_GREEN_AS_LUMA,
    //   Likely want to set this to zero.
    //   As colors that are mostly not-green
    //   will appear very dark in the green channel!
    //   Tune by looking at mostly non-green content,
    //   then start at zero and increase until aliasing is a problem.
    FxaaFloat fxaaConsoleEdgeThresholdMin,
    //    
    // Extra constants for 360 FXAA Console only.
    // Use zeros or anything else for other platforms.
    // These must be in physical constant registers and NOT immedates.
    // Immedates will result in compiler un-optimizing.
    // {xyzw} = float4(1.0, -1.0, 0.25, -0.25)
    FxaaFloat4 fxaaConsole360ConstDir
)

FXAA Console 360 Code

The optimized 360 version is now the only 360 version. The 3.9 and 3.10 optimized 360 versions had a bad typo/bug which I found and fixed with 3.11. Not sure if anyone got far enough to try it yet. Hopefully it will work now, please let me know if output for some reason does not look right.

Note the 360 version needs 3 samplers which alias to the same texture. Each sampler has a different exponential bias to factor out multiplies which would otherwise be done in the shader.

Fixed Type Aliasing Problem for OpenGL Developers

Switched all types to "Fxaa*" so that FXAA does not alias defines with other developers who are already redefining GL's vecN types to floatN.

FXAA Console for PC Is Tiny Bit Faster on NVIDIA Cards

Actually at this point if you are running FXAA Console on PC, you might want to try just running the FXAA Quality algorithm with the lowest presets,

#define FXAA_PC 1
#define FXAA_QUALITY__PRESET 10

Or,

#define FXAA_PC 1
#define FXAA_QUALITY__PRESET 20

FXAA 3.10 Released

2011-07-20T20:38:00.000-07:00

Download FXAA 3.10 Shader Source

Images

FXAA 3.10 Preset 13 (about same cost as FXAA 3.9)

FXAA 3.9

No AA

360 Optimized Shader Updates

Fixed a missing ";" typo. Made some changes here to make it easier to integrate in engines like Unreal.

Added FXAA Quality Presets

Added presets to enable developers to tune performance/quality trade off. Change with define,

#define FXAA_QUALITY__PRESET 12

The 2N presets are like FXAA 3.9 but with slightly better long edge quality. If you have problems with dithering (or ringing) by thin objects then use a 2N preset. The 1N presets trade better long edge quality for more dithering artifacts. For a given cost, 1N presets will have better long edge quality than 2N presets. Notes from the code,

OPTIONS
-----------------------------------------------------------------------
10 to 15 - default medium dither (10=fastest, 15=highest quality)
20 to 29 - less dither, more expensive (20=fastest, 29=highest quality)
39       - no dither, very expensive 

NOTES
-----------------------------------------------------------------------
12 = slightly faster then FXAA 3.9 and higher edge quality (default)
13 = about same speed as FXAA 3.9 and better than 12
23 = closest to FXAA 3.9 visually and performance wise
 _ = the lowest digit is directly related to performance
_  = the highest digit is directly related to style

Performance is determined by the last digit. Example runtime for GTX 480 for a 720p frame,

_0 = 0.171 ms _1 = 0.188 ms _2 = 0.202 ms (default for FXAA 3.10) _3 = 0.214 ms (default for FXAA 3.9) _4 = 0.226 ms _5 = 0.238 ms _6 = 0.247 ms _7 = 0.256 ms _8 = 0.265 ms _9 = 0.274 ms

FXAA 3.10 Soon

2011-07-20T01:55:00.001-07:00

Just finished up the FXAA 3.10 update. Still need to do a little performance and image testing before I can release it (only tested thus far on my notebook). Teaser,

(0.) Re-introduced presets to FXAA Quality.
(1.) So better user control on quality/performance.
(2.) Have much better long edge support!
(3.) Default preset should be faster than FXAA 3.9.
(4.) Default preset will have higher long edge quality.
(5.) Possible faster new early exit.

HardOCP Article on FXAA in FEAR3

2011-07-18T13:28:00.001-07:00

HardOCP Article on FXAA in FEAR3

Anyone want FXAA 3 with better long edge quality for PC?

2011-07-17T20:08:00.001-07:00

Digital Foundry : Future of AA

Should I add "FXAA Long-Edge-Quality" to FXAA 3?

I might have gone too aggressive in the optimization vs long-edge-quality trade off with FXAA 3 Quality aiming for lower end PC GPUs. Easy to fix if anyone wants it. Let me know.

FXAA 3.9 Typo Fixed

2011-07-13T16:28:00.000-07:00

Was a typo which is fixed now in the prior linked source: line 1347 had an extra ";", should be,

#define lumaM rgbyM.w

Not,

#define lumaM rgbyM.w;

NVIDIA FXAA 3.9 Released

2011-07-13T13:40:00.000-07:00

(1.) Fixed FXAA Quality artifact problems.
(2.) Some FXAA Quaility optimizations targeting NVIDIA cards.
(3.) New prototype optimized 360 version of FXAA Console.

Download FXAA 3.9 Shader Source Code

FXAA Quality Algorithm Quality Improvements

There are a few image quality improvements in FXAA 3.9 for specific cases where FXAA 3.8 had problems. Here is one subtle example, FXAA 3.9 is better on already AA'ed thin features which transition between vertical to diagonal to horizontal. FXAA 3.8 has more distortion in these cases. See the area around the ring below.

Image crop taken from Civ, which already has great AA...

FXAA 3.8 (left), FXAA 3.9 (right)

FXAA 3.9 Performance

With FXAA 3.9's new quality improvements, performance should be similar to FXAA 3.8, with the exception of increased performance on some NVIDIA cards. For example, FXAA 3.9 on GTX 480 is about 24% faster than FXAA 3.8. A 720p frame on GTX480 can now be processed in 0.21 ms (actual timing depends slightly on the image). More importantly for lower end cards:

$113 GTX 550ti 1920x1200 @ 1.3 ms
 $88 GTS 450   1680x1050 @ 1.1 ms

The Quality version of the algorithm now runs on low-end NVIDIA cards at higher resolutions faster than the Console version of the algorithm on PS3 at 1280x720.

FXAA on DX11 and GL is slightly faster than DX9 (due to gather4), numbers above represent GL.

Prototype Fully Optimized FXAA for 360

Latest build contains an untested fully optimized version of FXAA Console for the 360. This code was generated thanks to suggestions from Andy Luedke. Should be fully texture bound now. Let me know if there are problems (don't have a 360 to test on right now).

Why Pixel and Sub-Pixel Contrast Reduction?

The "No AA" image taken from a crop of a BBC2 screen shot. Actually this image has some AA (see the iron-sight in the lower right), however aliasing remains due to alpha test and problems associated with tone-mapping after MSAA resolve...

FXAA from version 1, was designed differently than prior AA filters in that FXAA specifically targets reducing contrast on features which are pixel in size. Single-pixel sized features alias by snapping from pixel to pixel, and sub-pixel sized features alias by flickering on and off as they overlap a pixel's center. To me, this is more visually distracting than edge crawling on multi-pixel features.

No AA (left), No Contrast Reduction (middle), FXAA (right)

Without FXAA's Pixel and Sub-Pixel Contrast Reduction, visually aliasing still remains. With FXAA, aliasing is much reduced and consistent across image features. No post-process AA filter is going to fix everything, there is always a compromise between sharpness and lack of aliasing, but I feel as if FXAA does a good job and has a great quality/cost ratio.

Certainly some developers and users will have different preferences on the aliasing vs softness trade-off. For this FXAA 3.9 Quality algorithm has a tuning knob: FXAA_QUALITY__SUBPIX. The "No Contrast Reduction" images were produced by zeroing this tuning knob.

No AA

FXAA_QUALITY__SUBPIX = 0.0

FXAA_QUALITY__SUBPIX = 0.75 (default)

Crysis 2 DX11 Tube

2011-06-27T15:40:00.001-07:00

Sharper FXAA3 Console

2011-06-27T14:22:00.000-07:00

This is was NOT documented prior, but FXAA3 Console can be made sharper by changing the "rcpFrameOpt" input. To sharpen change,

// This must be from a constant/uniform.
// {x___} = 2.0/screenWidthInPixels
// {_y__} = 2.0/screenHeightInPixels
// {__z_} = 0.5/screenWidthInPixels
// {___w} = 0.5/screenHeightInPixels
float4 rcpFrameOpt

To,

// This must be from a constant/uniform.
// {x___} = 2.0/screenWidthInPixels
// {_y__} = 2.0/screenHeightInPixels
// {__z_} = 0.333/screenWidthInPixels
// {___w} = 0.333/screenHeightInPixels
float4 rcpFrameOpt

Lowing the constant from "0.5/" to "0.333/" will decrease the amount of sub-pixel AA, allowing more aliasing and less blur on single pixel features.

Voxel Cone Tracing

2011-06-24T08:44:00.000-07:00

See Cyril Crassin's Interactive Indirect Illumination Post

FXAA3 Without RGBL, or With Wrong Luma

2011-06-22T08:39:00.001-07:00

FXAA3 Without RGBL

An alternative for when getting Luma in Alpha is a problem and FXAA can run on non-linear RGB data: (1.) use the Green channel in place of Luma in the Alpha channel, (2.) adjust the early exit thresholds. Note this will not work for linear RGB, and this will result in no AA on pure red and blue edges.

If anyone goes this route for PS3, let me know, I will make an updated PS3 shader, as working with the Y channel changes around all the shader scheduling...

FXAA3 With Incorrect Luma

If there is a need to run FXAA right after blending, and the blending passes do not store or use destination Alpha: have the opaque writes before blending write out Luma into the Alpha channel, then blend, then run FXAA like normal.

In this case, Luma will not match the RGB in areas with blending. But likely areas with blended content either maintain the edges of the opaque surfaces (windows), or blend over content which does not need AA (particles).

FXAA should work in this case. Would be interested in getting feedback on this. If there are problems, let me know, I can likely fix them...

FXAA3 Source Released

2011-06-21T15:46:00.000-07:00

Releasing FXAA3, this is the 8th update to the shader source (Fxaa3_8.h).

FXAA 3 Source Shader

Results With FXAA3 Quality (PC) Algorithm

Note this image was processed at one time instead of individual crops of images processed then stitched together.

No AA (left), then FXAA3 Quality on linear RGB (right)

Linear RGB vs Non-Linear RGB

Linear RGB (left), then non-linear RGB (right)

FXAA3 Console vs FXAA3 Quality

Both non-linear, Console (left), and Quality (right),

FXAA3 Console Tuning

2011-06-20T16:29:00.000-07:00

This is for all those with the 2rd release candidate (Fxaa3_7.h). Also sorry in advance, some of the test swatches were sourced from JPEGs.

Still exploring getting the best quality of out of the latest FXAA3 Console. The settings in the last file might not be optimal (defaults were slightly blurry again), here is the latest that I have found. Note in all these screenshots I have,

#define FXAA_CONSOLE__EDGE_THRESHOLD_MIN 0.0

to turn off this feature so that output looks the same between PS3 and 360. Best to tune everything else first with this set to zero. Then afterwards tune the threshold min for 360.

IMO The New Best Settings

This is using non-linear RGB, and non-linear Luma, with the following settings,

#define FXAA_CONSOLE__EDGE_SHARPNESS 8.0 #define FXAA_CONSOLE__EDGE_THRESHOLD 0.125

Original then these settings with FXAA3 Console,

Why Does FXAA3 Console Add A Little Noise?

After gradient detection and early exit (5 samples), the algorithm only uses 4 or 2 samples to "blur" to remove aliasing. Hard to do any better in the very limited performance window...

Going Sharper

This is a little sharper and faster on 360 due to a more aggressive early exit, but starts to turn off AA on some edges. Same non-linear RGB, and non-linear Luma, with the following settings,

#define FXAA_CONSOLE__EDGE_SHARPNESS 8.0 #define FXAA_CONSOLE__EDGE_THRESHOLD 0.25

Comparing edge threshold 0.125 (left) then 0.25 (right),

Turning Off Early Exit On PS3

This saves 2 shader clock cycles on PS3, but will decrease non-edge sharpness as the algorithm starts acting like a directionally adaptive noise reduction filter!,

#define FXAA_CONSOLE__EDGE_SHARPNESS 8.0 #define FXAA_EARLY_EXIT 0 #define FXAA_PS3 1

Early exit on (left), and off (right).

Linear RGB vs Non-Linear RGB

First off Linear only looks good with,

#define FXAA_CONSOLE__EDGE_SHARPNESS 8.0

Linear RGB input tends to "bloom" bright areas outward where as non-linear RGB tends to maintain the dark thin features (this is a function of how the algorithm works). Yes technically linear is more correct, however given how hacky FXAA3 Console is, technically correct does not always yield the visually best results.

Non-linear (left), and linear (right)

Going Softer On Edges

Believe these were the last defaults. When using non-linear RGB inputs can use,

#define FXAA_CONSOLE__EDGE_SHARPNESS 4.0 #define FXAA_CONSOLE__EDGE_THRESHOLD 0.25

Edge sharpness 8.0 (left), and 4.0 (right),

Going Softer Everywhere

Perhaps if looking for a more film like look (even digital film is never sharp like a GPU rendered image),

#define FXAA_CONSOLE__EDGE_SHARPNESS 4.0 #define FXAA_CONSOLE__EDGE_THRESHOLD 0.125

Edge sharpness 8.0 (left), and 4.0 (right),

All Images

Picasa

Xenoblade Tube

2011-06-20T08:17:00.000-07:00

Lighting Tree Leaves With Modified Normals

2011-06-18T18:29:00.000-07:00

Lighting 3D Trees

Optimizing the Linear Play-Through Game

2011-06-11T09:09:00.000-07:00

I would like a AAA studio targeting PC and PS3 (requires Blue Ray) to optimize for the interactive movie experience. Want a consistently awesome 10 hour CG quality experience instead of game quality experience where the player controls the micro events in-between fixed big budget macro events: similar 3rd person gameplay as current linear titles like Uncharted, but from a director's camera.

When, with many AAA titles, it is just as much fun to watch someone play through your game as it is to play through the game, this is hinting at something important. How about enable the player to be able to release the controls and have the fully scripted path take over, or where if the player sucks, the game takes over and makes the player not suck enough to keep the experience awesome. Why force frustrating replays? Other games already have the frustrating replay game mechanic nailed, and how many players of those games actually finish the game? Why have more choice in a linear game, how about less choice and more awesome. Want replay-ability? Who honestly replays linear games? Rather the studio reuse technology and asses and build another awesome new play-through from the perspective of another character in the same universe.

Linear games should look substantially better than non-linear games.

Factor Out Rendering of Static Content

Camera runs through a fixed linear play through, however local interactive elements are any range between fully dynamic and fully scripted. Camera is on a rail. Opaque static geometry, background non-interactive visuals, and assets such as static shadow maps, light probes, environment maps, etc, are fully CG pre-rendered. The local GPU (+SPUs on PS3) are tasked only with rendering characters and dynamic objects, and compositing them with the pre-rendered streamed content.

Blue Ray capacity, at the low end is 25GB, which is good for over 10 hours of video at 4 Mbits/sec.

Highly risky, but highly likely to be technically possible on current generation hardware...

FXAA3 Console Status Update 2

2011-06-10T08:46:00.000-07:00

Thanks again for everyone who has provided feedback for the FXAA3 preview!

Adjusted the Console Algorithm Again

FXAA2 and FXAA3 Console applied to a wire frame image resulted in some artifacts at the center of intersecting lines. Also really wanted FXAA3 to not have a performance hit when using the sub-pixel AA contrast reduction.

The solution to both problems was to make an adjustment to the 4 samples used for filtering. Prior solutions kept all 4 samples on the same line, picking either the first 2 close samples (for short edges) or all 4 samples (for longer edges) for the resulting box filter. The new solution uses different and simpler logic for choosing the positions of the first 2 close samples, and the last 2 far samples. Assuming this is the last adjustment, I'll be covering this in detail at the SIGGRAPH presentation. Note the slides which are on the DVD are now outdated.

Tuning on PS3

Verified the limiting "speed of light" of the first version of the new algorithm as 13 shader clock cycles for NV4x/RSX (ALU bound). On a 7900 GT this limit is 1.11ms for a 720P frame. Was able to achieve 1.16ms in a sample GL app. NVIDIA's GLSL compiler path managed to get optimal instructions even before I verified with hand tuning. This early version did not contain the early exit code which resulted in some blurring on areas which needed no AA filtering.

Then revised the algorithm adding back "early exit" in the form of a conditional mov. This added one cycle for a limit of 14 shader cycles. Estimated limiting speed of light on PS3 of around 1ms for 720p frame. To approach this limit, PS3 likely requires manual forcing a higher register count to get better texture cache performance though lower shader occupancy.

Still have to verify PS3 compiler is getting correct instruction generation, but PS3 tuning is nearing its logical conclusion.

Tuning on 360

The limiting speed of light for the non-early exit version is 8 texture fetches or 0.92ms. The early exit version adds an extra texture fetch, but has early exit after 5 texture fetches. Early profiling results before this latest algorithm were presenting with a measured 1.1ms for 720p (maybe ALU bound). Goal for 360 is under 1ms total.

Note I'm doing at least 2 versions of the FXAA Console algorithm now: one for PS3 and one for 360, because of the differences in vector pipelines and branch performance of these 2 consoles.

The End Result of All This

FXAA3 should be around 1ms for 720p on either console, and using the same interface, has a higher quality PC version. FXAA3 Console on PS3 does not have the same long edge quality as Sony's MLAA, but overall FXAA3 costs significantly less, freeing up SPU time for things like tile deferred lighting, etc!

Ideas for Unique Texturing After Rage

2011-06-09T22:48:00.000-07:00

Loss of Chroma

Maybe just an artifact of double compression (compressed video of a game with highly compressed unique texturing)? Seems like the trade off between unique texturing and limited storage media is resulting in a loss of chroma. Low bit rate in the darks tends colors to push towards primary colors (red/green/blue), or secondaries (magenta/yellow/cyan). Might also be amplified by quick transcoding not being able to get ideal dithering when recompressing to DXT, or lack of ability to have a scale and bias for each physical page to maximize bit utilization of DXT?

Would be awesome for another unique texturing title to go monochrome.

Maybe practical, is there a way to do a parallel friendly error diffusion dither during transcode, maybe using a more expensive two pass DXT compression? First pass normal compression, second pass doing a second compression, where each pixel adds in weighted error of first pass gathered from an eye-pleasing diffuse pattern?

Also maybe practical, seems like the low-bit rate dark problem can be improved retroactively by artist managed color grading (seems like this is currently being used?), or proactively by allowing pages to have unique dequantization look up tables?

Vector Quantization

VQ is easily impractical for CPU based decoders because of the memory latency problem (look up tables are larger than cache sizes). Given that the GPU does not have this problem, would be very interested to take another look at using some form of vector quantization based GPU image decompression for virtual texturing. Would be ideal to be able to do simple VQ based transcoding from a pixel shader running in parallel with other normal graphics pixel shaders.

Prey2 E3 In-Game Footage

2011-06-09T19:07:00.001-07:00

PixelJunk Sidescroller E3 Tube

2011-06-09T16:30:00.000-07:00

Least Squares Vertex Baking

2011-06-09T07:49:00.001-07:00

Least Squares Vertex Baking

Prey2 E3 2011 Trailer

2011-06-03T08:54:00.001-07:00

FXAA Console Status

2011-05-30T16:00:00.001-07:00

Quick update on the FXAA Console Algorithm,

Got console feedback for the FXAA2 and FXAA3 Console algorithm, working some serious re-optimization for consoles. PC: FXAA3 is faster than FXAA2, unfortunately not the case on consoles. On 360: FXAA2=1.3ms (TEX bound), FXAA3=1.5ms (ALU bound). On PS3: FXAA2=1.7ms and FXAA3=2.5ms, both massively ALU bound. Was expecting trouble on PS3 because the code was never vectorized, and PS3 compiler won't find all the transforms a human will.

Plugged in a 7900 GT (an optimization proxy for PS3) and went to town on optimization for FXAA2. Timing progression with various code transforms. Numbers on a 720p frame.

1.76 ms = FXAA2 baseline (from original post)
1.50 ms = factoring out texture coordinate math to vertex shader
1.24 ms = factoring + RGBL optimization
1.23 ms = factoring + RGBL + transform ending "if" to conditional move
1.17 ms = hand instruction vectorization, switching to dot products, etc

Possible that PS3 ends up faster than 360 for FXAA2 with these new optimizations.

FXAA3

FXAA3 is getting a new round of optimizations too. For PS3 expect FXAA3 will be a little slower (so far 0.16 ms slower) than FXAA2 because the FXAA3 early-exit or early-discard is a net loss on this platform. Still haven't fully optimized FXAA3 Console shader. For 360 not sure, need to double check if early-exit is a win after the hand vectorization.

FXAA3 in final release will be including both the FXAA2 and FXAA3 optimized shaders.

Tegra 3 Glow Ball Demo

2011-05-29T21:11:00.001-07:00

Good friends from the NVIDIA demo team built this demo,

BF3 Pure Awesomeness

2011-05-27T11:10:00.000-07:00

Lit particles and dynamic lighting,

FXAA 3 Status Update

2011-05-27T01:01:00.000-07:00

Linearly Correct

Finally have FXAA working correctly in a linear color space: sRGB typed textures work amazingly well. Here is FXAA 3 Quality using linear color space processing one of my favorite hard-to-anti-alias sources: the beautiful pixel art from the Metal Slug series.

No FXAA

FXAA

Console Perf

Console perf with FXAA III is still an unknown, waiting on some more feedback and then another round of tuning before FXAA III is released. FXAA II performance has been reported by a few developers, with PS3 taking about 1.3x more time than Xbox360, which is to be expected. While PS3 has more texture performance, PS3's lack of unified shaders hurts when doing GPU post processing (pixel pipe only): FXAA II Console is likely ALU bound on PS3.

FXAA III has an early exit path which FXAA II didn't have, this should improve performance, but FXAA III, as it is in the prototype stage, requires more ALU for pixels which don't early exit. Very curious about performance.

4/3 Super-sample FXAA

Really like the results I'm getting with 4/3 Super-sample FXAA. The technique is simple, mocked up below on a crop from Mirror's Edge. Render 1.7x the number of total pixels as normal and without any anti-aliasing,

Then apply FXAA Quality, and down-sample by 3/4 in each direction to return to the target resolution,

Image quality clearly increases with increased amount of super-sampling. With FXAA, the super-sampling can be fractional. Compared to deferred shading with super-sample lighting, I believe this fractional super-sampling + FXAA method will provide better quality results at a lower cost.

This is ideal for me, since my preferred PC outputs are a small 720p HDTV and another 1080p 3D HDTV: want higher quality/pixel rather than crazy high resolutions.

HDR24 Encoding

2011-05-13T10:34:00.000-07:00

In case this is useful for someone, I haven't had the time to explore this in practical applications yet...

Goal

ALU is scaling faster than bandwidth: want a fast effectively linear 24-bit per pixel color framebuffer/texture encoding which has 8-bits free in the alpha channel for other usage (in my case a proxy for depth to avoid an extra depth buffer read). Encoding needs to not add artifacts during texture interpolation.

Enter HDR24

Using hardware sRGB support for linear2sRGB encode/decode, HDR24 encoding maintains data in 32-bit/pixel sRGB, but changes the meaning of some upper region of the displayable range of sRGB (for example, the top 1.0/8.0).

Tonemapping

Games use tonemapping in order to map a high dynamic range (HDR) into the the low dynamic range (LDR) of displayable sRGB values. However these mappings distort saturation and contrast across the entire displayable sRGB range. Here are a few resources,

A Closer Look At Tone Mapping
Filmic Tonemapping Operators
Why Reinhard Desaturates Your Blacks
Why Filmic Curve Saturates Your Blacks

Often tonemapping is done after lighting right before post processing and this causes a mix of problems,

(1.) Tonemapping alone is hard for an artist to control when tuning saturation and contrast.

(2.) Tonemapping followed by Color Grading is hard to control because the Color Grading fights against the change done in Tonemapping.

(3.) Post tonemapping, sRGB-to-linear fetched texels are no longer linear, post processing is not working in a linear color space.

(4.) Game tonemapping clips to {1.0,1.0,1.0} white which removes any hints of clipped tint via white point color temp.

HDR24 Highlight Compression as Tonemapping

HDR24 takes a different approach. The idea is to leave a majority of the displayable range unadjusted, and to provide reversible highlight compression at the heel of the displayable range only. This way,

(1.) Engine would do exposure adjustment on lit HDR data prior to conversion to HDR24. This insures colors are nearly at un-post-processed final output luminance.

(2.) Post processing works effectively in a linear color space. With the exception of texture interpolation between texels in the compressed highlight area. Error which likely is not noticable anyway.

(3.) After post processing is finished effectively in linear space, Color Grading can be applied. Effectively non-visible noise (+/- around one sRGB value) is added during HDR24 encoding, which should provide enough quality for Color Grading from 8-bit data.

(4.) Noise is acceptable in the clipped HDR range. Clipped range always gets returned to small compressed area in highlights only. Noise will get filtered out, because post processing operators are effectively low-pass filters.

Visual Result

Grabbed a linear 14-bit/channel Canon 5D Mark II photograph RAW file from dpreview.com then used dcraw to extract a 16-bit/channel raw image. Applied some contrast,

Then added some HDR over-exposure in the color channels. Here is non-tone-mapped and clipped LDR output, notice how clipped areas loose any hint of color temperature,

Same image encoded with HDR24. This example is using the upper 1.0/8.0 of the displayable linear range to compress the HDR range of 1.0-16.0. Note the highlight compression maintains a hint of the hue of a clipped color as can be see in white color temp. Highlight compression also softens the transition to overexposure, more like a real film exposure responce curve.

Clearly after conversion to 8-bit LDR, all HDR information is normally lost. Here is the result of scaling the first clipped output by 1/16,

However with HDR24 encoding, the mapping is semi-reversible. After readback from an 8-bit per channel framebuffer, the upper 1.0-16.0 linear range is noisy, but quite usable for post processing operations, as the saturation and luminance is maintained.

On the left is a simple 9-tap motion blur from LDR 8-bit/channel input (notice the typical desaturation caused by highlight clipping). On the right is the same using HDR24.

Technical

HDR24 is configurable via two inputs:

(1.) Supported HDR linear range (0-1 = LDR, 1-N = HDR).
(2.) Amount of top displayable linear range to use for compression.

The tonemapping operator is a Log2() curve starting where the derivative matches the 45deg slope of the unadjusted region (1/ln(2)), and compressed to fit in the user configurable compression region.

Converting configurable options to constants required for operator,

// EXAMPLE INPUTS
float paramRange = 16.0f; // Supported HDR linear range.
float paramTop = 1.0f/8.0f; // Amount of top displayable linear range to use for compression.
 
// INTERMEDIATES
float log2deg45 = 1.442695f; // 1/ln(2)
float begin = log2(log2deg45);
 
// OUTPUTS
float constSplit = 1.0f - paramTop;
float constStart = log2deg45 - constSplit;
float constMulC = (1.0f - constSplit) / (log2(paramRange + log2deg45 - constSplit) - begin);
float constAddC = constSplit - (constMul * begin)
float constMulD = 1.0f / constMulC;
float constAddD = begin - (constMulD * constSplit)

Highlight compression,

if(color.r > constSplit) color.r = (log2(color.r + constStart) * constMulC) + constAddC;
if(color.g > constSplit) color.g = (log2(color.g + constStart) * constMulC) + constAddC;
if(color.b > constSplit) color.b = (log2(color.b + constStart) * constMulC) + constAddC;

Highlight decompression,

if(color.r > constSplit) color.r = exp2((color.r * constMulD) + constAddD) - constStart;
if(color.g > constSplit) color.g = exp2((color.g * constMulD) + constAddD) - constStart;
if(color.b > constSplit) color.b = exp2((color.b * constMulD) + constAddD) - constStart;

Adding noise to the encoding before final output,

Out of time, saving this for another post (fast noise without a texture lookup!)...

ARM PC on USB/HDMI Stick

2011-05-05T17:31:00.000-07:00

ARM PC on USB/HDMI stick!

GPU Point Based Rendering

2011-04-27T01:54:00.000-07:00

Rendering points?

Odd-Even Merge Sort

Up until Fermi, triangle setup was limited by a serial path tied to the GPU clock rate. Recently, there has been an explosion of great demos, see Matt Swoboda's work taking advantage of Odd-Even Merge Sort to incrementally sort points on the GPU. There is also a bounded method to pre-calculate the sortings for a static point cloud.

A2C

Another option to sorting is alpha to coverage (A2C). Hardware A2C masks leave a lot to be desired (and yes, worse on AMD, guessing their pattern is a trick to increase utilization of bandwidth during stores?). With newer GPUs, software coverage mask can be generated in the shader to provide better dithering. Or alternatively the alpha value could be modified instead (works on older GPUs). CSAA on NVIDIA hardware can greatly increase the quality for a given bandwidth. Temporal dithering could be placed under control with temporal feedback. Also might want to give Stochastic Transparency a read for ideas.

OIT

A non-option (too expensive) is AMD's Linked List OIT, or the next logical extension, Adaptive Transparency.

Certainly these OIT ideas can be optimized (and likely will still be too expensive) by removing the linked list (removes bandwidth, memory, and atomic overhead) and instead maintaining a single counter per pixel and a fixed size array per pixel. Overflow problem can be dealt with stochastically. For instance, before incrementing the atomic counter to find array position, non-atomically read the counter, then based on the counter's position and the sample's alpha value, start tossing out low alpha samples. As the array fills up, start pruning at higher alpha values. Can add in a stochastic randomness to this. Could have objects or groups of points, frame incrementally sorted using odd-even merge sort to reduce the likely-hood of filling the array in the worst case back to front ordering.

Or could go a step further, by removing the atomic, and just selecting a random position in the per pixel array to write to. The challenge of this method is that the ideal random position offset should be the same for a given SIMD grouping (or warp in CUDA) to avoid randomly hitting different cache lines, and also keeping point locality in a SIMD grouping. Note if doing this manually, clearly the arrays need to be interleaved in groups for cache coherency. Yes there will be collisions and dithering.

FreePipe

FreePipe aims for correctness and still has good perf for tiny triangles. Global 32-bit minimum atomics used to simulate a z-buffer. Global atomics are much faster in Fermi compared to prior generation (GT200), however access patterns can have large effects on performance. Ultimately I suspect that global atomic performance could be a bottleneck if the amount of ALU and TEX/MEM per sample is low enough.

Probabilistic Methods

The aim here is to remove the usage of atomics, and instead take advantage the performance of direct global memory loads and stores, by engineering the algorithm such that there is a high probability that a read-modify-write cycle address collision is rare, or there is some inherent ability to converge to the correct results even in the presence of errors (perhaps temporally for real-time graphics).

"When the particle has been updated (or maybe respawned) it is rasterized into a 32 bit combined z and color buffer (11+7+7+7 bit). No hard sync on the zbuffer (ie it's just if (z < zbuffer) zbuffer=z) , so it may be incorrect some places, but not very likely." - Beyond3D post on the tech for the 4KB demo Michigan by Loonies.

Given a larger address range for random accesses (and ideally block random accesses), the probability of access collision decreases (probability of needing atomics decreases). Certainly GPU architecture has a large effect on this with regards to access latency. The probability of a collision during a read-modify-write cycle decreases if the "modify" ALU instruction(s) are scheduled with only enough multitasking to hide any ALU latency. Clearly the "modify" code should not do any other memory transactions.

Estimated ideal pattern for this kind of algorithm,

(1.) Output array has a 2D surface style address layout instead of a linear layout. Use surface stores (or address calculation), or manually convert into a block linear address mapping where a block fits into a single cache line.

(2.) SIMD groups (warps) randomly grab a group of neighboring points to draw. Note in this case collisions between items in this group have an unspecified ordering.

(3.) Render into an image tree? Choose a level of the image hierarchy based on the projected radius of the point group distribution (SIMD group). This has two aims, first this is ideal for push-pull type hole filling, and second, this reduces the number of cache lines touched in a memory transaction.

Bright Future

Only scratching the surface of what can be done with parallel computing and point based methods on GPUs.

Api Trace GUI

2011-04-27T00:57:00.000-07:00

Api Trace GUI Post

More Revision 2011 Tubes

2011-04-25T13:45:00.000-07:00

Michigan a 4K intro by Loonies Tube

2011-04-24T15:57:00.000-07:00

CNCD & Fairlight - Numb Res Tube

2011-04-24T11:34:00.001-07:00

Tri-Ace Tech Tube

2011-04-22T08:49:00.000-07:00

Tri-Ace Research

Voxatron

2011-04-19T12:07:00.000-07:00

Blog

WebGL Path Tracing

2011-04-19T12:06:00.001-07:00

WebGL Path Tracing by Evan Wallace

Inemeri IBL Tube

2011-04-14T13:04:00.000-07:00

Simplifying My Workflow Post

The Hobbit Start of Production Tube

2011-04-14T12:32:00.001-07:00

Speculative Contacts

2011-04-11T09:27:00.000-07:00

Speculative Contacts

Prey2 Concept Art + Screen Shot

2011-04-07T09:49:00.000-07:00

TSSAA prevPos Notes

2011-04-04T12:44:00.000-07:00

A few options exist for generating the required "prevPos" input.

NO REPROJECTION : Just set "prevPos.xy = posPos.xy". Result will be blurry in motion, and converge to high quality when motion stops. Not the best option, but could be a first step for integration.

CAMERA BASED REPROJECTION ONLY : For each pixel, sample depth, compute the view space position, then transform to the projected view space position in the prior frame. Result will be blurry on objects which move at a different rate than the camera change. Might work ok.

MOTION VECTOR BASED REPROJECTION : Best option. Compute reprojection position using the motion vector field which is used for motion blur, "prevPos.xy = posPos.xy - mv.xy". Motion vector field can easily be 1/2 or 1/4 screen resolution (better for texture cache). Motion vectors can be computed per fragment or per object, the TSSAA algorithm should be highly tolerant of incorrect or slightly off motion vectors.

An Unfinished AA Idea

2011-04-04T10:48:00.000-07:00

Posting another AA idea here which I didn't see through to completion, perhaps someone will find this useful for something...

Using a similar sample pattern to DLAA, grab the following 13 samples around the pixel center MM, where the spacing is {-7.5, -3.5, -1, 0, 1, 3.5, 7.5} pixels,


             N3



             N2 

             N1
W3    W2  W1 MM E1  E2    E3 
             S1

             S2



             S3

Then compute horizontal and vertical blurs for 3 increasing larger widths in the vertical and horizontal direction,

H1 = (W1 + MM*2.0 + E1) * 0.25 V1 = (N1 + MM*2.0 + S1) * 0.25 H2 = (W2 + H1*2.0 + E2) * 0.25 V2 = (N2 + V1*2.0 + S2) * 0.25 H3 = (W3 + H2*2.0 + E3) * 0.25 V3 = (N3 + V2*2.0 + S3) * 0.25

Then for each {horizontal, vertical} pair of blurs for each width, choose the blur which results in the lower change from the center pixel (MM). For the given sets of edge lengths, this chooses correctly the blur direction. Note that each Bn might choose a different direction.

B1 = (abs(H1.g - MM.g) < abs(V1.g - MM.g)) ? H1 : V1 B2 = (abs(H2.g - MM.g) < abs(V2.g - MM.g)) ? H2 : V2 B3 = (abs(H3.g - MM.g) < abs(V3.g - MM.g)) ? H3 : V3

Then the unfinished part, compute weights for each Bn (without taking any more samples), and return a weighted sum,

W1 = ? W2 = ? W3 = ? OUTPUT = (W1 * B1) + (W2 * B2) + (W3 * B3)

TSSAA (Temporal Super-Sampling AA)

2011-04-03T13:36:00.000-07:00

Here is a prototype of Temporal Super-Sampling Anti-Aliasing which solves prior problems of ghosting and reprojection, while supporting quality around 8x SSAA.

Performance

This requires 11 32-bit texture fetches per pixel. Estimate cost around 1.7ms for a 1280x720 frame on Xbox360.

Screen Shots

Images are ripped from a simple GL test app which uses a modified Unigine screenshot to texture. Images are presented 2x size. Second shot shows the extreme motion case (where TSSAA slightly blurs the image),

TSSAA without motion (16x sub-pixel jitter from below)

TSSAA with fast motion (16x sub-pixel jitter from below)

No AA

How it Works

Yakiimo3D's DX11 Perspective Matrix Jittering Temporal AA is a good place to get some background on the Halo/Crysis method for temporal 2xSSAA, which is a similar but slightly different temporal AA method (combining samples instead of accumulating samples).

This TSSAA algorithm reprojects the results from the prior TSSAA output frame. Then limits this reprojection by a smooth minimum and maximum color computed from the neighborhood of the pixel in the current frame. This limiter removes ghosting and other temporal artifacts caused by incorrect reprojection. The current pixel is accumulated with the limited reprojection using a simple falloff.

Artifacts

(1.) Motion will tend to reduce sharpness a little.
(2.) Fast exposure change exhibits light image erosion/dilation.
(3.) Very thin sub-pixel details will still exhibit some temporal dithering.

Advantages Besides Super-Sampling

Haven't had the chance to test in a real game, so would really like to get some feedback if anyone tries this!

TSSAA should remove shader jitter and noise (so should likely apply any film grain effects after TSSAA perhaps during write to back-buffer). In theory things like jittered shadowing, dithered DOF, dithered motion blur, and SSAO without blur will be partly cleaned up by TSSAA, but only if the noise is single pixel noise. Lower frequency noise likely will not get cleaned up as well.

In theory TSSAA should work on transparent objects even if the object's motion vectors are not rendered into the motion vector field and used for reprojection offset. Might even work better when the typically sharper background motion vectors are used.

Temporal Sub-pixel Shift

When using this method, frames need to be drawn with a sub-pixel shift on the projection matrix (different offset for each frame). This is the same type of shift one would use when accumulating frames for a super-sampled AA "press-release" screen shot. Not sure what the best pattern is right now (haven't had the time to try all options). I have been using the following pattern for 4x,

{-0.75,-0.25}, {-0.75,0.25}, {0.25,-0.75}, {-0.25,0.75}

Then for 16x adding in a different offset each 4 frames,

{-0.375,-0.125}, {0.375,0.125}, {0.125,-0.375}, {-0.125,0.375}

Likely best to use a rotated grid pattern (like hardware AA) but distribute the samples temporally such that the distribution fills a pixel evenly temporally.

When integrating with AFR Rendering

Have not tried this with AFR, so this is only a best guess at how to integrate!

For AFR rendering with two GPUs, integration will need to be adjusted. Instead of depending on the previous N-1 frame output, use the previous N-2 frame output. Then for computing the previous position, offset by the N frame motion vector * 2.0. The algorithm should work fine even if the reprojection vector isn't fully correct.

Source

Note, I've only tested the TSSAA_GLSL_130 path, everything else should work however...

This source is designed to work with the DX10/DX11/GL pixel position instead of the half offset DX9 position. Setting TSSAA_FALLOFF at 1/8 seems to have the best trade-off between sharpness during motion and reduction of temporal dithering. Setting TSSAA_SMOOTHER to zero will make the shader sharper during motion, but the algorithm will be less effective on edge AA on the borders of objects in motion on a non-moving background.

Note this should be applied using non-HDR and non-sRGB inputs (towards the end of the post processing pipeline)!

/*============================================================================

                 TSSAA PROTOTYPE by TIMOTHY LOTTES @ NVIDIA

============================================================================*/

/*============================================================================
                               API PORTING
============================================================================*/
#ifndef     TSSAA_GLSL_120
    #define TSSAA_GLSL_120 0
#endif
#ifndef     TSSAA_GLSL_130
    #define TSSAA_GLSL_130 0
#endif
#ifndef     TSSAA_HLSL_3
    #define TSSAA_HLSL_3 0
#endif
#ifndef     TSSAA_HLSL_4
    #define TSSAA_HLSL_4 0
#endif    
/*--------------------------------------------------------------------------*/
#if TSSAA_GLSL_120
    // Requires,
    //  #version 120
    //  #extension GL_EXT_gpu_shader4 : enable
    #define int2 ivec2
    #define float2 vec2
    #define float3 vec3
    #define float4 vec4
    #define TssaaInt2 ivec2
    #define TssaaFloat2 vec2
    #define TssaaTex sampler2D
    #define TssaaTexLod0(t, p) texture2DLod(t, p, 0.0)
    #define TssaaTexOff(t, p, o, r) texture2DLodOffset(t, p, 0.0, o)
#endif
/*--------------------------------------------------------------------------*/
#if TSSAA_GLSL_130
    // Requires "#version 130" or better
    #define int2 ivec2
    #define float2 vec2
    #define float3 vec3
    #define float4 vec4
    #define TssaaInt2 ivec2
    #define TssaaFloat2 vec2
    #define TssaaTex sampler2D
    #define TssaaTexLod0(t, p) textureLod(t, p, 0.0)
    #define TssaaTexOff(t, p, o, r) textureLodOffset(t, p, 0.0, o)
#endif
/*--------------------------------------------------------------------------*/
#if TSSAA_HLSL_3
    #define int2 float2
    #define TssaaInt2 float2
    #define TssaaFloat2 float2
    #define TssaaTex sampler2D
    #define TssaaTexLod0(t, p) tex2Dlod(t, float4(p, 0.0, 0.0))
    #define TssaaTexOff(t, p, o, r) tex2Dlod(t, float4(p + (o * r), 0, 0))
#endif
/*--------------------------------------------------------------------------*/
#if TSSAA_HLSL_4
    #define TssaaInt2 int2
    #define TssaaFloat2 float2
    struct TssaaTex { SamplerState smpl; Texture2D tex; };
    #define TssaaTexLod0(t, p) t.tex.SampleLevel(t.smpl, p, 0.0) 
    #define TssaaTexOff(t, p, o, r) t.tex.SampleLevel(t.smpl, p, 0.0, o)
#endif

/*============================================================================

                                VERTEX SHADER
                                
============================================================================*/
float4 TssaaVertexShader(
float2 pos,                 // Both x and y range {-1.0 to 1.0 across screen}.
float2 rcpFrame) {          // {1.0/frameWidth, 1.0/frameHeight}
/*--------------------------------------------------------------------------*/
    float4 posPos;
    posPos.xy = (pos.xy * 0.5) + 0.5;
    posPos.zw = posPos.xy - (rcpFrame * 0.5);
    return posPos; }

/*============================================================================
 
                                PIXEL SHADER
                                
============================================================================*/
float3 TssaaPixelShader(
float4 posPos,     // TssaaVertexShader() output interpolated across screen.
TssaaTex texNext,  // Current frame before TSSAA.
TssaaTex texPrev,  // Previous TSSAA output.
float2 posPrev,    // Position of pixel in previous TSSAA output frame.   
float2 rcpFrame) { // Constant {1.0/frameWidth, 1.0/frameHeight}.
/*--------------------------------------------------------------------------*/
    #define TSSAA_FALLOFF (1.0/8.0)
    #define TSSAA_SMOOTHER 1
/*--------------------------------------------------------------------------*/
    #if TSSAA_SMOOTHER
        float3 nextNW = TssaaTexOff(texNext, posPos.zw, TssaaInt2(-1,-1), rcpFrame).xyz;
        float3 nextNE = TssaaTexOff(texNext, posPos.zw, TssaaInt2( 2,-1), rcpFrame).xyz;
        float3 nextSW = TssaaTexOff(texNext, posPos.zw, TssaaInt2(-1, 2), rcpFrame).xyz;
        float3 nextSE = TssaaTexOff(texNext, posPos.zw, TssaaInt2( 2, 2), rcpFrame).xyz;
    #else
        float3 nextNW = TssaaTexOff(texNext, posPos.zw, TssaaInt2( 0, 0), rcpFrame).xyz;
        float3 nextNE = TssaaTexOff(texNext, posPos.zw, TssaaInt2( 1, 0), rcpFrame).xyz;
        float3 nextSW = TssaaTexOff(texNext, posPos.zw, TssaaInt2( 0, 1), rcpFrame).xyz;
        float3 nextSE = TssaaTexOff(texNext, posPos.zw, TssaaInt2( 1, 1), rcpFrame).xyz;
    #endif
/*--------------------------------------------------------------------------*/
    float3 nextN = TssaaTexOff(texNext, posPos.xy, TssaaInt2( 0,-1), rcpFrame).xyz;
    float3 nextW = TssaaTexOff(texNext, posPos.xy, TssaaInt2(-1, 0), rcpFrame).xyz;
    float3 nextM = TssaaTexLod0(texNext, posPos.xy).xyz;
    float3 nextE = TssaaTexOff(texNext, posPos.xy, TssaaInt2( 1, 0), rcpFrame).xyz;
    float3 nextS = TssaaTexOff(texNext, posPos.xy, TssaaInt2( 0, 1), rcpFrame).xyz;
/*--------------------------------------------------------------------------*/
    float3 prevM = TssaaTexLod0(texPrev, posPrev.xy).xyz;
/*--------------------------------------------------------------------------*/
    float3 nextNN = nextNW + nextNE;
    float3 nextSS = nextSW + nextSE;
    float3 nextMinH = min(nextNN, nextSS);
    float3 nextMaxH = max(nextNN, nextSS);
    float3 nextWW = nextNW + nextSW;
    float3 nextEE = nextNE + nextSE;
    float3 nextMinV = min(nextWW, nextEE);
    float3 nextMaxV = max(nextWW, nextEE);
    float3 nextMinHV = min(nextMinH, nextMinV) * 0.5;
    float3 nextMaxHV = max(nextMaxH, nextMaxV) * 0.5;
/*--------------------------------------------------------------------------*/
    float3 nextMinHVN = min(nextMinHV, nextN);
    float3 nextMaxHVN = max(nextMaxHV, nextN);
    float3 nextMinWM = min(nextW, nextM);
    float3 nextMaxWM = max(nextW, nextM);
    float3 nextMFalloff = nextM * TSSAA_FALLOFF;
    float3 nextMinSE = min(nextE, nextS); 
    float3 nextMaxSE = max(nextE, nextS);
    float3 nextMinHVNWM = min(nextMinHVN, nextMinWM);
    float3 nextMaxHVNWM = max(nextMaxHVN, nextMaxWM);
    float3 nextMin = min(nextMinSE, nextMinHVNWM);
    float3 nextMax = max(nextMaxSE, nextMaxHVNWM);
/*--------------------------------------------------------------------------*/
    return nextMFalloff + (min(nextMax, max(nextMin, prevM)) * (1.0 - TSSAA_FALLOFF)); }

Inemeri IBL

2011-04-01T23:54:00.000-07:00

Inemeri's IBL prototype is looking really good, a painterly lighting model complements the art style well. Not sure if they are using a down-sampled rendering result of the prior frame, if so, would be tempted on mobile devices to try rendering an even more low-poly version of the scene into a tiny render target, blurring, then using this as the IBL lighting source (instead of using the prior frame).