Matlab Offscreen Rendering Toolbox

Project 3D mesh onto image using off screen OpenGL rendering, determine face color, visibilty, etc.

Description

This toolbox provides support to use OpenGL's rendering capability to project 3D triangular meshes onto an image plane. This has a few applications including extract the color of each face on the mesh given an observed image, determine the shadow area on the mesh given a light source direction and rendering a mesh.

Directly using OpenGL calls has the advantage of using more advanced graphics hardware capabilities, which results in faster rendering, and better control of anti-alliasing and other GL options. Version 2 also added the support of rendering a depth image.

This toolbox is used heavily to for the research of reflectance and illumination modeling published in Eurographics Symposium on Rendering 2006 and Siggraph 2007 Technical Sketch

1. Tianli Yu, Hongcheng Wang, Narendra Ahuja, Wei-Chao Chen, Sparse Lumigraph Relight by Illumination and Reflectance Estimation from Multi-View Images, Eurographics Symposium on Rendering (EGSR), 2006.
2.Tianli Yu, Hongcheng Wang, Narendra Ahuja, Wei-Chao Chen, Sparse Lumigraph Relight by Illumination and Reflectance Estimation from Multi-View Images, Technical Sketch, SIGGRAPH, 2006.

watch a youtube video of the result at:
http://tianliresearch.blogspot.com/2006_06_01_archive.html

• MATLAB release: MATLAB 7.6 (R2008a)

[Ben]

Thank you Lucas! Your notes on how to compile the code is very helpful. I will follow your notes.

[Lucas Chavez]

Very good submission and exactly what I needed. I had a few headaches installing and also I have a couple questions.

First, lessons learned. I have a Windows 7 64 bit processor machine. I had to make the following changes.

1. Had to uninstall my 64 bit Matlab and install a 32 bit. This is because I was getting the error "this is not a valid 32 bit application" when trying to use the mex file
2. Downloaded freeglut and glew from http://www.transmissionzero.co.uk/software/freeglut-devel/ and http://glew.sourceforge.net/
3. Extracted the folders from the zip file. Named the glew folder simply "glew" (it had a version number on it)
4. Changed "CompileOffscreen.m" at lines 28 and 29 to be:
GlewPath = '-I./glew/include -L./glew/lib -lglew32';
GlutPath = '-I./freeglut/include -L./freeglut/lib -lfreeglut ';
5. Find the "bin" file in both the freeglut and glew files. In glut get "freeglut.dll" , in glew get "glew32.dll" and put them in the root directory of Offscreen.
7. Look for (#include "GL/glut.h") and replace with (#include "GL/freeglut.h") in any .cpp or .h that has it.
6. It should successfully compile. Run OffScreenTest.m to check

Ok. So now a couple questions. Any time I run a script which calls the .mex files more than once it shuts down Matlab and Matlab doesn't produce a dump file. Why could this be? Something with OpenGL not initializing/exiting gracefully?

In the readme.txt you show how to go from 3d coordinates to pixel coordinates on the depth image. How do you go the other way?

Thanks :)

[Tianli Yu]

Re: Jason
Please try the latest version. I fixed a few issues with freeglut. Hopefully there is no crash any more.

Thanks!

[Jason Heym]

I can compile and run the MEX functions. However, the offscreen window is not dismissed. When I click to close the window, Matlab crashes hard.

My setup:

Windows 7 x64
ATI FirePro V8800
Matlab R2011b (x64)
Visual Studio 2010 express
Microsoft Windows SDK 7.1
Freeglut 2.6.0 (compiled to x64 library)
GLEW 1.7.0 (static / -DGLEW_STATIC)

Anyone with a similar configuration?

I used Freeglut because it seemed more recent and had a visual C project file compatible with VC Express 2010.

[Tianli Yu]

Re: Cihan Ula
If you are using 64 bit version of matlab, you will need 64 bit version of glut and glew, glut32 and GLEW32 won't work in this case.

[Cihan Ula?]

It seems it does not work in x64 Win7.
For x86, I used the following conf. and it worked propoerly.
1. Download glut and glew and extract them under C:\ path. They should be located as C:\glew and C:\glut.
2.Use the following paths which are found in CompileOffscreen file, which is located in offscreen folder.

GlewPath = '-IC:/glew/include -LC:/glew/lib -lGLEW32';
-LC:/glew/lib -lglut32';
GlutPath = '-IC:/glut -LC:/glut -lglut32';

[Michael Hödlmoser]

 

[Diederick]

People interested in Using OpenGL from MATLAB should have a look at the PsychToolBox (PTB), a library with extensive abilities that is used predominantly by vision researchers to program up their stimuli in MATLAB and display them through OpenGL. It includes MoGL, a Matlab wrapper around just about all OpenGL functionality, although PTB it also contains higher level functions, e.g. for easy drawing of dots, or procedural creation of Gabor patches.

[Gayathri]

The code compiles successfully after providing a couple of dlls in the same directory as the code. But MATLAB crashes while trying to execute the code with the sample data provided. Need a more stabilized version that is independent of the platform.

[Al]

In a Windows machine, I located glew and glud in C:\ and my setting is:

GlewPath = '-IC:/glew/include -LC:/glew/lib -lglew32';
GlutPath = '-IC:/glew/include -LC:/glew/lib -lglut32';

It compiles successfully.

[Tianli Yu]

I cannot reproduce the crash on leopard, and I don't have a snow leopard to test with. Judging from the error message, you could try add a dummy display callback, by calling

void glutDisplayFunc(void (*func)(void));

after glutCreateWindow in OffscreenGL's constructor.

see glut's spec for more details
http://www.opengl.org/resources/libraries/glut/spec3/node46.html#SECTION00081000000000000000

[Malcolm Reynolds]

I got this compiled on Snow Leopard 10.6.2 (had to change the #include to <GLUT/glut.h> to stop error messages) but now I'm having the following problem when running:

EDU>> OffScreenTest
camnum =
27
2010-02-01 18:36:48.265 MATLAB[33555:3503] GLUT Warning: glutInit being called a second time.
2010-02-01 18:36:48.311 MATLAB[33555:60f] GLUT Warning: The following is a new check for GLUT 3.0; update your code.
2010-02-01 18:36:48.312 MATLAB[33555:60f] GLUT Fatal Error: redisplay needed for window 1, but no display callback.

I get this if I run matlab with -nodesktop, and after this it crashes to the command line. If I try from within the full matlab GUI, it just disappears silently when it crashes.

If you could provide any guidance it would be most appreciated, I am looking to do something similar in terms of render to off-screen FBOs calling out from matlab and so getting your code working would be a big help.

Cheers,

Malcolm

[Tianli Yu]

The spaces in your GlewPath & GlutPath are causing the issue. Try to use "C:\DOCUME~1" in the path.

[joli woo]

mex -I.\C:\Documents and Settings\Administrator\My Documents\MATLAB\Offscreen\glew\include\GL L.\C:\Documents and Settings\Administrator\My Documents\MATLAB\Offscreen\glew\lib -lGLEW -I.\C:\Documents and Settings\Administrator\My Documents\MATLAB\Offscreen\glew\include\GL L.\C:\Documents and Settings\Administrator\My Documents\MATLAB\Offscreen\glew\lib -lglut BatchFaceColorGradImpl.cpp

C:\PROGRA~1\MATLAB\R2007A\BIN\MEX.PL: Error: 'and' not found.

??? Error using ==> mex at 206
Unable to complete successfully.

Error in ==> CompileOffscreen at 17
eval(['mex ' GlewPath ' ' GlutPath ' BatchFaceColorGradImpl.cpp']);

my glutpath is set to GlewPath = '-I.\C:\Documents and Settings\Administrator\My Documents\MATLAB\Offscreen\glew\include\GL L.\C:\Documents and Settings\Administrator\My Documents\MATLAB\Offscreen\glew\lib -lGLEW';
GlutPath = '-I.\C:\Documents and Settings\Administrator\My Documents\MATLAB\Offscreen\glew\include\GL L.\C:\Documents and Settings\Administrator\My Documents\MATLAB\Offscreen\glew\lib -lglut';

Is there anything wrong there?

[Srimal Jayawardena]

Ok so it works this way when I add

-lGLEW -lGL -lGLU

to the include path as follows

GlewPath = '-I/usr/include/GL -L /usr/lib/ -lGLEW -lGL -lGLU';

I've had this issue issue before as well and ended up replying to my own post :)
http://www.mathworks.com/matlabcentral/newsreader/view_original/677788

[Tianli Yu]

The package are tested in mac, but it should also work in windows once you set the correct include and lib path. Unfortunately I don't have Windows version of matlab at hand to test it out.

For windows, you will need to download the GLUT and GLEW binaries:
GLUT: http://www.xmission.com/~nate/glut.html
GLEW: http://glew.sourceforge.net/

Extract both zip files, GLEW will extract to a set of bin/, include/, doc/, lib/ folders. You could copy glut.h into glew/include/GL, glut32.lib into glew/lib/, so that they share the same path. And copy glew32.dll and glut32.dll into the offscreen root folder.

Then you could use modify the path in CompileOffscreen.m to something like this:

GlewPath = '-I./glew/include -L./glew/lib -lGLEW';
GlutPath = '-lglut';

You should be able to compile it now. Let me know if you have further problems.

[Luigi Giaccari]

Warning: MEX could not find the library "glut"
specified with -l option on the path specified
with the -L option.
BatchFaceColorGradImpl.cpp
OffscreenGL.h(12) : fatal error C1083: Cannot open include file: 'GL/glew.h': No such file or directory

C:\PROGRA~1\MATLAB\R2009A~1\BIN\MEX.PL: Error: Compile of 'BatchFaceColorGradImpl.cpp' failed.

Never used open GL, any help?