The Promised Meat
I thought I would give you a day in the life to track the progress of the prototype as it was being built. This part of the blog can get highly fluffy and very techie, so I advise readers to skip this part and jump past all the date stamps. If you absolutely must drill down into the basement, read on:
01:48 Dug Out A DBP Module and Cuppa
I have decided for the sake of expedience to 'graft' the PerC stuff onto the Basic3D module, the central 3D command set of DBP. I know it's hacky and not very responsible, but there is method to the madness.
I will be able to publicly share my modifications through Google Code when I have finished my PerC stuff so other DBP users can benefit from it. It also means the contaminating code does not affect the new VS2010 build of the modules which I intend to overhaul before starting into the guts of DBP for another development.
02:44 Two New Commands and a Blob
I've created the first of what may be many DBP apps for this project, and added two new hacky commands called MAKE OBJECT PERCBLOB and UPDATE OBJECT PERCBLOB. I have added code into the module to create and modify the vertices of a basic sphere so I can see this on screen.
Now I know everything is running fine and I can see the 3D, and manipulate it, I just have to replace the mesh form with something that represents the depth data in some reasonable way. It means if anything goes wrong, it's nothing to do with my 3D, just the data and the code that finds the data. Clever huh!
02:58 Ambitious Way
I was just about to create a basic vertices only grid then realised later on I would have to change four vertices for every coordinate in the depth buffer that changed. It is times like this that you realise taking the slightly longer route during early development will save headaches later. Going to use an index buffer and keep it to one vertex per depth reading, which will make a smaller footprint for the 3D object and make changing it MUCH easier.
03:16 Who'd Be A Programmer
It turns out 320x240 grid and six indices per face works out at over 400K in size, and a 16-bit indices buffer has a maximum size of 65535. The current DBP uses WORD index values (designed many moons ago when WORDs where faster and more memory efficient than gorging on 32-bit index buffers). The natural step is to go back to vertex only where I can have a very large mesh. That said, I don't think I will need (or want) to use the entire 320x240 depth area for the final 3D object (unless the subject is extremely fat and wide). As I am both, if I can squeeze myself into a single indice buffer it should be good for most users. I can have 10,922 faces, which works out at roughly 45x40 capture area which is okay to start with. If I had more time, I would just bite the bullet and step up to DirectX 11 and get mucky with tessellation and geometry shading tricks but I don't have the luxury of time here. I will proceed with my little 45x40 grid and basically detect the best place to grab the depth data from as it's probably not far from the actual area I am interested in.
03:29 Smug Mode Times Two
I adjusted the size and my entire vertex+index construction code compiled and ran first time perfectly (almost). When I ran the app there was no 3D object. Thanks to the fact I added a CONTROL CAMERA WITH ARROWKEYS to my DBP program, I was able to go for a short walk to view the other side of my object. And presto, there it was. The face culling winding order was reversed, that's all. I could have been hammering at that for ages wondering where my 3D went, but after over ten years of messing with 3D graphics I know that old chestnut!
03:44 Make Short Video
I've just made a short video of the current state of the DBP app, with the wibbly wobbly 3D grid, ready to have depth data added onto it.
Adding the Depth Data is a tense moment as I will be adding lots of new headers and dependencies from the SDK sample code, and many wonderfully bad things can happen. Fingers crossed, except mine as I need them for typing.
04:24 HOT TIP : Warning For PerC SDK C++ Users
Been banging my head against a wall wondering why my cut and paste code is not linking properly, and decided to trace the decorated name it was creating with that used by the Perc SDK Util Library. Turns out you cannot use a project that switches off wchar_t as a unique type as this confuses the linker. This option can be changed under "Project Properties>C/C++/Language/Treat WChar_t as Builtin type", it can also be changed via the "/Zc" option.
07:13 Way Too Much Fun
I just want to report that I became absorbed with the 3D version of myself. Once I got the basic representation of depth working in 3D, I just kept going. Adding normalisation, averaging vertices, tweaking depth scope and scale, all sorts of tweaks. Have to stop though as I needed to produce a blog video.
The Final DBP Code So Far
Rem Project: PercA.exe
Rem Created: Saturday, February 23, 2013
`
rem App Init
sync on : sync rate 0 : color backdrop rgb(255,128,0)
`
rem Make 3D floor
make matrix 1,1000,1000,100,100
position matrix 1,-500,-55,-500
`
rem Make 3D Object
load image "brick.png",1
make object percblob 1,50,50,50
set object cull 1,0
`
rem Place camera
position camera 0,0,75
point camera 0,0,0
`
rem Add lights
make light 1 : set directional light 1,0,0.9,-0.1 : color light 1,512,255,0
make light 2 : set directional light 2,0,-0.9,-0.1 : color light 2,0,255,512
make light 3 : set directional light 3,-0.5,0.0,-0.2 : color light 3,-100,255,-100
`
rem Main loop
do
`
rem Move camera around
control camera using arrowkeys 0,1,2
set point light 0,camera position x(),camera position y(),camera position z()
set light range 0,200
`
rem Each cycle refresh 3D data in object
update object percblob 1,0,0,0
`
rem Prompts
set cursor 0,0
print screen fps();"fps"
if spacekey()=1 then texture object 1,1
if returnkey()=1 then texture object 1,0
`
rem Update screen
sync
`
rem End loop
loop
The Progress In Brief
So what you have seen is the depth data from the Perceptual Camera used to generate a 3D construct in a prototype that allows me to move around the object and view it from different angles.
I discovered that there is enough fidelity in the depth data to create a good face shape and with further work I can produce more striking 3D elements. I am also happy with the speed of everything so far, and will probably stick to detecting my own gestures and intents using this raw data too.
The technical side of getting the SDK into the DBP module was painless, and apart from a sticky moment with the wchat_t type, quite easy. I now have a good foundation on which I can add DBP BASIC and C++ willy nilly to solve the various challenges ahead.
The Prototype Binary
If you have a Perceptual Camera all set up, you are welcome to try the prototype yourself. Find it at the end of this link:
I will offer the disclaimer this has only been tested on one machine and is not guaranteed to work on any other. If it does, please comment and let me know as I need all the testers I can get.
Signing Off
The next step will be to round off the 3D object so it's a real head instead of a rubber wall with protrusions. I also want to get it textured and get it transmitted to another client sooner rather than later. It's always a good idea to get the main chunks of your functionality in early so you know what general shape your app is going to be. I am happy with the shape of the app at week two, and looking forward to see my Perceptual 3D gubbins evolve.
This is very cool! Worked great on my desktop PC (Win 7 64-bit) but I couldn't get it to run on my Ultrabook. First it threw the d3dx9_31.dll error, then after putting that DLL in its directory it said "Failed to load DLL (3: DBProCameraDebug.dll)". But I do love being able to try the prototypes as you create them so thanks for sharing :)
ReplyDeleteYeah, it rather needs some quite old legacy DLLs, but the final app would install those at some point (probably a good idea to do that sooner rather than later). Thanks for helping me test!
ReplyDelete