Thanks Yann Yeo for Voxel cone tracing link, thanks Redox for several earlier links.

• OpenCL 1.2 was released, news behind link (no paradigm shifts today).
  
  
• OpenACC is a new API trying to find implementers.
  The problem with OpenMP is that current specification does not manage memory, therefore implementations don't know what regions of memory need to be available in case of running code on GPU. And they need to know, accessing CPU memory from GPU over PCIe is too slow, even with 2x faster PCIe 3.0 devices coming in a month or two.
  Solution: members of OpenMP Working Group on Accelerators did theirs job and extended OpenMP for use on GPUs, by adding necessary pragmas, e.g. for explicit copying of data between CPU and GPU memory. To be accurate, this initial release did not extend OpenMP. Instead, only GPU specific pragmas were specified in a new separated API called OpenACC. If it works well, we might see these pragmas merged into future version of OpenMP.
  Still, there will be fight with Microsoft. As the most influential compiler developer, Microsoft uses its near monopoly position to promote proprietary APIs as a way to lock customers in. Right now, Microsoft pushes competing API, C++ AMP, and tries to help it by not supporting OpenMP (ok, by supporting 9 years old version 2.0).
  Although both OpenACC or C++ AMP APIs look well and I wish them support from GPU vendors, I did not see any implementations yet. What we have and what works best today is OpenCL for GPUs, OpenMP for CPUs and CUDA for confused developers.

> Scenes are made of roughly 10 spheres. It makes me wonder in good sense,
> we coders love limits and intros show we can do lots in 256 bytes...
> can we make good gameplay with renderer limited to 10 spheres?
> I guess building games on Thierry's commented source code would be piece
> of cake. Any demo organizer here?

Guido van Rossum started it

> But if you want to be taken seriously as a researcher, you should
> publish your code! Without publication of your *code* research in your
> area cannot be reproduced by others, so it is not science.

> > Indeed, scientifically, it's *preferable* that the results be validated from *scratch*,
> > to avoid risk of "contaminating" recreations with any mistakes made in the original experiment.
>
> Ideally, perhaps, but we do not live in a world of ideals. In the world of reality researchers
> often do not have the time to reproduce the code, in which case the publication goes without
> challenge (this is the most frequent outcome in the computing field). Other times the researcers
> will want to reproduce the code but cannot due to lack of documentation. A really ambitious
> researcher may spend a ton of time figuring out assumptions and discovering major flaws
> in the previous work, none of which can be published (in the computing field) because
> no conference will accept papers that do not show "improvement."
>
> In the computing field, that last point is the Charybdis to the Scylla of unstated assumptions
> and privately-held code. Either one is grounds to seriously question the validity
> of our research methods, and therefore results, in computing.
>
> I once talked to a few higher-ups at IBM research. They flat out stated that they will
> not accept any computing publication as true until they verify it themselves and they find that
> 99% of it is worthless either because it cannot be reproduced or the papers are flat-out lies.

• OpenGL 4.2 brings bunch of new features, even for older hardware, with nearly immediate support from AMD and Nvidia. Highly appreciated.
  
  
• OpenMP 3.1 also brings bunch of new features, with support in upcoming gcc 4.7. Appreciated, but complicated to use until adoption improves, Microsoft still supports only 2.0. There's risk Microsoft will never go to 3.x as OpenMP slowly loses importance - it was designed for systems with single address space, it does not know what memory regions are accessed, therefore it can't move code execution to GPU (it does not know what memory regions and when to copy between CPU and GPU). It's great for systems with single address space, and although research projects exist for extending it to GPUs with separated address space in future versions, it's probably too much hassle, I don't expect success.
  
  
• Cilk Plus is Intel's new parallelization tech (Intel acquired it by purchasing Cilk Arts), ported to upcoming gcc 4.7. What they didn't say in presentation is that it's similar to OpenMP in capabilities, but less powerfull and less supported than OpenMP, it would be crazy to start using it.
  
  
• C++ AMP is Microsoft's new parallelization tech, with only promised support so far, but with potential, as it targets GPUs from the beginning. Like OpenCL, it explicitly manages memory and transfers between CPU and GPU address spaces, and it restricts parallel code to not use recusrsion, library functions etc. With C++ lambdas, it's even more nice and compact than OpenCL and CUDA code, it's win in design. The only problem - it's designed to lock you into Microsoft platform, like CUDA is designed to lock you into Nvidia platform, so it can't replace multiplatform OpenCL.

> I just wanted to know if you know of some realtime, or even 
> semi-realtime solution that would work better than the current 
> non-realtime model, that exists or will exist soon? From the demos on 
> your page (lightsmark, Interactive illumination using voxel cone 
> tracing, etc), the technology looks fairly mature- it just confuses me 
> why it doesn't seem to be readily offered on the market.

There are many excellent ideas and papers, but only one in ten techniques makes it into executable demo so that people can actually see it, and then only one in ten demos makes it into product, so that people can actually use it.

Turning idea or paper into demo requires significant effort, there is not enough space in paper for all important details, implementer needs to be as clever as researcher who wrote the paper. In the end, implementer can discover problems that were not evident from the paper and throw whole work away. Still, it is fun, so many try it and we see at least few new demos every year. Of course, paper authors have working 'demos', but existing journal and grant system does not require them, so no one releases them. (It's difficult to setup efficient grant system, I can't say I would do it better, researchers already do miracles for minimal wages. Still, demos would increase chances of new tech getting into production, which is one of motivations behind grants.)

Turning demo into product needs huge effort too. First demos tend to show single data set that works on single GPU. Product needs all corner cases working on diverse hardware, reaching such maturity can take years. Secondary problem is that demo written only to demonstrate something, or research code written for testing new ideas, can't be integrated into product. One has to write new code from scratch to make it as flexible and easy to use as pieces of lego, to adapt it for production. Such rewrite is not terribly difficult for experienced skilled programmer, but writing something again, from scratch, without adding new features is no longer fun, not many do it.

Then there are slightly inflated expectations, as several companies already use realtime GI for marketing, although closer inspection shows that actual products still use static lightmaps.

Although realtime GI spreads slowly due to obstacles I just described, some visualization products already made it to end users. Realtime pathtracing is the most often demonstrated tech, I think some plugins already exist, it's only too slow/noisy for presenting architecture to end users. Then there is RenderLights, probably first noise-free realtime GI product on market (video of RL 1 year ago, it's quite more advanced now). I did not mention RL earlier because it's based on my code. Several other visualization companies licensed my code, as far as I know they did not release yet. There may be other products I don't know of, let me know anyone if you know. So it's not much, but it's getting better.

As a proper pathtracing paper, it starts motivation by poking into rasterization: "Code complexity increases tremendously for systems that simulate global illumination effects using rasterization because multiple approximations need to be combined to achieve a sufficient visual quality." Hmm, I use rasterization and I don't filter indirect illumination across geometry edges since 2005. But I won't poke back into pathtracing, guys already suffer enough by waiting for their renders to complete ;)

Still, I would like to see this demonstrated in Sponza (i.e. no caustics, guided filter destroys them), it could produce noiseless interactive pathtracing even faster than in Brigade ERPT video. For interactive use, this could soon become practical.

Interactive renderer works well, but as other plain pathtracing renderers, it is slow/noisy. It is slower than comparable GPU pathtracers on comparably expensive hardware.

Intel claims that Intel Compiler improves performance by approx 10% (compared to Visual Studio or gcc). Although I did not test Intel Compiler, 10% is believable for included trivial scenes; it is hard to believe for complex scenes typically bottlenecked by memory access, something that can't be solved by compiler. Maybe that's why included scenes are so small, Cornell box the most complex one.

For testing complex scenes, package includes .obj importer, however it is bit difficult to work with, noise is sometimes too much. (Also, what I did not find documented: Emittance is ignored, emissive triangles have to be specified from commandline, with full coordinates. Textures have to be in .ppm. "Error: file format not supported" sometimes means "your filename contains space, lol".)

Lee Salzman published his open source LPV implementation (source and linux+windows executables) probably in 2009-09 (according to date attribute of files). I'm not adding it to demos, because I don't see it working, demo runs strangely lit, without expected GI effects when spheres move.

Andreas Kirsch published his open source LPV implementation (page, source and windows executable) in 2010-07. It seems to work in fixed region, majority of scene stays completely black, and light and objects can't be moved, so it's not easy to test it, but those few colums in center of scene have expected indirect illumination.

Aside from OpenGL, I just bought several new GPUs for tests, so next time some confused developer uses CUDA instead of OpenCL, I'll come with better screenshots, I promise.

Motivation

API

This single member function makes class C refcounted, without breaking old apps, without breaking simple general rule that every object has its owner who creates and deletes it by plain old "delete". Users may look at createReference() as if it creates full copy, but in fact, it increments internal refcount and returns "this". Trick is in destructors and operator delete. We can't stop compiler from calling them, but implementation is ours, we can check refcount and destruct member variables only if it is 0. Some class psychologists might talk about destructor abuse, so keep implementation in closet to avoid arrest, expose only nice createReference(), users will love you and your middleware will dominate.

Implementation

static void* g_classHeader;

class Ninja
{
public:
	//! First in instance lifetime.
	void* operator new(size_t n)
	{
		return malloc(n);
	};

	//! Second in instance lifetime.
	Ninja()
	{
		refCount = 1;
	}

	//! When deleting an instance, this is called first.
	virtual ~Ninja()
	{
		if (--refCount)
		{
			// backup class header, we are going to die temporarily
			g_classHeader = *(void**)this;
			return;
		}
		// destruct members as in your usual destructor
	}

	//! When deleting an instance, this is called last.
	void operator delete(void* p, size_t n)
	{
		if (p)
		{
			Ninja* n = (Ninja*)p;
			if (n->refCount)
			{
				// resurrect instance after destructor
				*(void**)n = g_classHeader;
			}
			else
			{
				// delete instance after destructor
				free(p);
			}
		}
	};

	//! Creates new reference to this. Both pointers must be deleted (in any order).
	//
	//! It is not thread safe, must not be called concurrently for one buffer.
	//! It may be called concurrently for different buffers.
	virtual Ninja* createReference()
	{
		if (this)
			refCount++;
		return this;
	}


private:
	volatile unsigned refCount;
};

Example use case

Pros

• More efficient than smart pointers (no additional allocations, no additional indirection, in this case also no locks, simply no overhead).
• It does not affect work with objects (new, delete), old code runs with refcounted objects unmodified. I.e. refcounting is invisible, ideal for ninjas.
• It does not create any new types, old types become refcounted.

Cons

• You must be able to not destruct member variables in destructor, types with automatic destruction can't be used.
• Very intrusive, the same code must be implemented in every refcounted class, it can't be implemented once and reused.

Years passed and 2d compression gurus did not sleep. And although fractal compression did not make revolution we expected, wavelets went nearly mainstream in jpeg2000. Nearly mainstream, because despite wide support in software and 1.2x better compression, people still store data in jpeg, using libjpeg 6.

This summer, Guido Vollbeding released libjpeg 7. 11 years after v6. First thing I noticed in readme was shocking broncs-like FILE FORMAT WARS: The ISO JPEG standards committee actually promotes different formats like JPEG-2000 or JPEG-XR which are incompatible with original DCT-based JPEG and which are based on faulty technologies. IJG therefore does not and will not support such momentary mistakes and then v7 is basically just a necessary interim release, paving the way for a major breakthrough in image coding technology with the next v8 package which is scheduled for release in the year 2010. Ha! I want to know more, how will that v8 work? Friend google quickly found forum post from 2004 that shows Guido badmouthing jpeg2000 and promising revolution in jpg. cpoc commented I like how the guy always raises the unknown-secret mega property of DCT that trumps everything... so no one can argument. Few years later, Guido revealed his secret (pdf). If you need more proof the guy's crazy, see the last page, figure B-1. However, if you really read the pdf, he does good work and v8 may greatly improve progressive jpg, improve compression in low quality settings and lossless compresion. Let's wait for v8.

Or not wait? Guido did not provide jpg compression improvement numbers and I think it won't be enough to catch jpeg2000. Jpeg2000 is already here and it's 1.2x better than jpg. Current state of the art, DLI, is 1.6x better than jpg. It's very slow, but few months back, Dennis Lee added "fast" mode that's fast enough for practical use, with only slightly worse compression, still much better than jpeg2000. I think dli could do what jpeg2000 did not, seriously kick jpg ass. But Dennis Lee would have to open source it.

• Light Propagation Volumes in CryEngine 3 (video) - Excellent, looks like practical for hard-boiled game devs. It's mix of several techniques, so devs may be sad it's complicated to implement, but it's reality of extreme optimizations. Note that it's optimized for good look, correctness is sacrificed for speed (only one bounce etc).
• Hardware-Accelerated Global Illumination by Image Space Photon Mapping - Faster photon mapping, but not fast enough for game devs.
• An Efficient GPU-based Approach for Interactive Global Illumination (video) - Faster raytracing, but not fast enough for game devs.
  Update 2009-08-28:
• Micro-Rendering for Scalable, Parallel Final Gathering - Very interesting for novelty, not yet fast enough for realtime or good enough for offline production. Static micro-rendering has similar asymptotical time complexity to static KD tree raytracing, however micro-rendering works with point hierarchy that is easier to update in dynamic scenes (in O(n)). Micro-rendering seems to be faster also in absolute numbers.

Remaining problem: No vendor supports OpenCL-OpenGL interoperability. Without fast switch to OpenGL, it's useless for hybrid rendering. DX11 + compute shaders will probably come sooner than working OpenCL+OpenGL. Until vendors fix the problem, OpenCL and DX11 won't compete, they will cover different markets, OpenCL for GPGPU, DX11 for hybrid rendering.

Update: OpenCL GPU driver developed by Apple should be present in just released OSX 10.6. I didn't have time to test it yet, but if it's true, it's first publicly available OpenCL GPU driver. Others report it doesn't support any extensions. It would mean no OpenGL interoperability.

On unrelated note, this is truly web 2.0, now with all post titles linking to itself. I considered going even further, but this single static html file design already scales fantastically. See how bandwidth grows exponentially (go internets go) while page size grows linearly (I'm not robots).

Update: AMD drivers are also already available to selected developers. AMD expects public release in "second half of 2009".

Update 2009-05-28: Another closed beta release from Nvidia. It's good, one of two features I miss was added.

Update: Must.. resist.. temptation.. rewrite.. everything.. for.. 3.1.
I'm weak, already rewriting.

First, there was smallPT, GI in 99 lines of C++, fantastic piece of code that renders Cornell box like scene made only of spheres. It's very slow, but Thierry Berger-Perrin ported it to CUDA and added new scenes and controls for realtime manipulation, The Once Known as SmallPT was born. Magnificent. Beware, requires Nvidia card. Source code included.

Scenes are made of roughly 10 spheres. It makes me wonder in good sense, we coders love limits and intros show we can do lots in 256 bytes... can we make good gameplay with renderer limited to 10 spheres? I guess building games on Thierry's commented source code would be piece of cake. Any demo organizer here?

Update: this slide at hardware.fr shows Nvidia plan: final runtime/driver is expected in the middle of 2009.

I reported it to MS, but before fix is available (and before OpenOffice releases 3.0 with .pptx support), users will google for solution. Let's attract them. Viewer says PowerPoint Viewer cannot open the file "Shedding Indirect Light on Global Illumination for Games.pptx".. Error in event log is Faulting application PPCNVCOM.EXE, version 12.0.6211.1000, time stamp 0x46ce621f, faulting module oartconv.dll, version 12.0.6214.1000, time stamp 0x47029c72, exception code 0xc0000005, fault offset 0x0022d94b, process id 0x3b78, application start time 0x01c912bcf5abd300.

BTW, every child knows that when writing new GI article, it's important to keep first two sentences in standard form
1. "GI is extremely critical stuff"
2. explain why
Gameplayer levels up the scheme by adding strong third sentence
3. "If you don't believe us, go back and fire up the original Doom."

For me, no surprises inside.
Design looks like it could work on Cell too, although with terribly expensive random access to big data structures in global memory. Still, OpenCL would be good for Cell in PS3 development, because one would be able to test on PC and have roaring PS3 jet-engines turned off.

As far as I'm concerned, I welcome even this smaller step, because driver bugs are no longer problem, I became pretty skilled in avoiding them.
Thinking more about Khronos, I hope OpenCL will start soon. Because when we have generic compute shader platform with simple memory management, OpenGL and Direct3D are just unimpressive high level APIs built around hw rasterization, sentenced to slow death. AMD dropped CTM in favour of OpenCL. Nvidia will drop CUDA for the same reason 10 years later. The only two questions are, will Intel manage to add serious OpenGL support before everyone else drops it (in 25 years)? Of course, serious = runs Lightsmark. And will Microsoft manage to prevent OpenCL implementations on next Xbox?

Update:
- That thing about CAD companies, repeated on all forums, originates from Carmack.
- I don't think that death of 'new API' is sooo big disaster for driver developers. Completely new API in addition to the old one, maintained for compatibility is a lots of additional work, while current deprecation model forces them only to add some #ifdefs and compile twice. If they want, they can create separated 'clean' GL3 codebase at any time.

Update2: AMD is not dropping CTM, sorry. Plans are to layer OpenCL on top of CAL that evolved from CTM. However effect is the same, mainstream app developers won't use proprietary API if they get portable one.

Update: users report scores over 700 fps.

Finally, id gets it too. Carmack explained that he found some of the complaints of Doom 3's brand of horror to be "completely valid," saying that the "contrived nature of monsters hiding in a closet" and the extreme darkness were two things that caused the company to cancel its game Darkness and begin production of Rage.

I really like this effort, it's 100x more important for realtime raytracing than Intel's marketing dollars. On the other hand, people should not confuse this with realistic lighting. With zero or constant ambient, it looks less realistically than an average game. Of course there are ways how to improve lighting and even without looking into Arauna, I'm sure Jacco has something in development.

Update: Gotchas solved, but still no luck. With disabled Adblock, download page finally opens. Download link shot down Firefox, right click to save is necessary. Game started, but image was distorted, then it stopped responding and silently disappeared. Demo displayed single static image. Maybe Vista x64 still too exotic?

Update: Demo starts after hitting space, thanks Dhatz. Very nice scene! Scene and lights are static, but it's still the best realtime raytracing demo ever. Looking for improved realism, majority of pixels have only direct lighting + const ambient, so it resembles late 20th century. Exceptions are pixels with specular reflection, refraction and some strange noise.
One extremely important point I forgot to mention: source code of Outbound engine, Arauna, is available too.

Update: problem disappeared after several changes in rendering code. I haven't tried to track it down to one-line change, but changes were related to display lists.

Update at the end of summer: It seems Aaron reached rougly 50% of his goals. Still I think it's an excellent result.

Somehow I think there's not enough realism yet, even with truecolor added to VESA BIOS some time ago. Realtime global illumination is not pixel perfect. How do I quantize in 500000-dimensional space, quickly?
What, NP hard? Starting assembler...

Direct3D defines approx 10 fixed levels of shader length/complexity/whatever (e.g. SM4.0), you must always pick one of them. If you can fit your shaders into these levels, everything is fine, simply look at level supported by graphics card and use appropriate shader from your collection.

OpenGL is simpler and more general, it doesn't specify length/complexity/whatever, you simply compile your shader and driver says OK. Or Sorry, too many interpolators/whatever used, we'll screw you by running it in software. Now you have several shaders and you want to pick the most complex one supported. The only question is, do you test bottom up or top down?

I started with top down: when most complex shader failed to compile/link, I went to second most complex and so on, until I found first supported one. It worked fine except that every card behaved differently, I had strange things reported. I had to buy 5 graphics cards for testing and write several workarounds, based on vendor and model string (ouch!) to prevent broken rendering and crashes inside driver when requested to compile valid GLSL shader. Behaviour often changed with graphics card driver version. It wasn't getting better.

I switched to bottom up: when the simplest shader compiled/linked, I went to second most simple and so on, until next one failed. Number of revealed driver bugs fell down significantly, only one card family continued to crash in driver or render incorrectly.

Conclusion? OpenGL is superior in design, but inferior in practical use, due to insufficient driver testing. Search for the most complex shader won't make you happy.

Try search for the most complex 5-state Turing machine instead. It's like writing really good intro in 5 instructions. Ready?...go! After 45 years of pure research without drivers, problem is still open. Nice intro article. It's about beavers too.

Back when only video was released, I speculated that it's copy of our technique from 2000 (used in RealtimeRadiosity 2 demo) where per-vertex layers of lighting are precomputed for sampled positions of light / objects. When light / objects move, closest samples and their layers of precomputed lighting are selected and per-vertex lighting is reconstructed as weighted average.

Back to reality. CryTek's technique is much simpler and less correct. Only scalar ambient occlusion value per texel is precomputed. The rest is magic expression in shader (not specified precisely) that combines distance attenuation, normal divergence and ambient occlusion. It doesn't simulate light paths, so it would probably look bad in scenes with mostly indirect light, but it probably looks great in mostly open scenes / no big blockers. They added support for portals... I think it was absolutely necessary to avoid light freely leaking through walls.

Conclusion: In mainstream realtime rendering, indoors, it will be first big improvement over aging constant ambient. It's rough estimate rather than light simulation, so it needs lots of additional infrastructure (portals) and work from artists to look good/avoid artifacts. It also means it's viable only for gaming industry and even there, adoption won't be fast. However it's very inexpensive for end user (gamer), so chances are good that it will be adopted by several other engines before fully automated physically based RTGI techniques take lead.

It's still nice to use gcc. It has the best warnings of all compilers, it always finds some error that was completely ignored by other compilers.

It seems only remotely related to rendering, however it has some importance for me as my code is heavily templated and new language features may simplify it.

Ok, it's still too hot area to set one standard now. Land changes quickly with AMD's Fusion plan to integrate GPU and CPU, optimal APIs for 3d graphics evolve as never before. So let's live with several APIs for next few years.

But that reminds me, hacking chaotic VGA registers was fun, because it was simple and nearly bug free; on the other hand using well designed VESA BIOS Extensions (VBE) was pain, because there were bugs in ROM and no chance to fix them without buying new graphics card, so we had to live with bugs. I wrote this list of approx 20 bugs and workarounds for VESA BIOS implementations. With this list, I was able to run on buggy cards, while others displayed random pixels or crashed.
Using GPGPU APIs could be fun again, because they are not set in stone (in ROM), and users can fix bugs by simple driver/library update, we don't have to gather workarounds.
The only thing that makes me worry is closed nature of those APIs. In comparison of APIs created by one company and multiple companies, those multi- look clearly better; see eg. how well specified is OpenGL in comparison to DirectX. DirectX must be rewritten each 2 years to catch up with OpenGL ;)

You can see many papers with precomputed global illumination. They offer very fast rendering of precomputed data and it's not always clearly visible that it must be precomputed, it could be hiden deep inside the article. So some of them are misinterpreted by massmedia as a final solution of the old problem with global illumination. However those hours of precomputations and those limitations coming from precomputed data (eg. you can't move any object) should not be forgotten.

Imagine architectural modeling. Architect or user interactively manipulates geometry of the building. He wants to see it nicely lit. Existing papers offer him for example relighting (Direct-to-Indirect Transfer for Cinematic Relighting by M.Hasan) - once building is modeled, you can run expensive precomputation and when it's done, manipulate light in realtime. But don't touch the geometry. And don't touch the camera. It's both fixed. Well.. can we have more interaction?

Now let's go for games. Games create strong pressure for technological progress, together with sex, however I'll stick to games. While previous generation had static buildings and precomputed illumination fits for such scenarios, current trend is to make everything destructible, even walls. Soon game developers will start looking around for fast global illumination in dynamic scenes.