« High-performance httpd components | Main | Water, water, everywhere »
February 27, 2005
OpenGL -> Image snapshots
I got code working last night that hits an OpenGL context and grabs the image from the frame buffer. I learned a few things, such as:
- It takes 5ms to read the pixels on my powerbook, 4ms on my powermac. That's a non-negotiable cost.
- Threads in OS X aren't as difficult as I first thought.
- The image in the frame buffer is upside down, so you have to flip it when reading it back out. I had learned this before, but forgotten about it. I got quite a surprise after the first run of grabbing images!
- Pasting together OpenGL contexts and renderers by hand is pretty darn hard. It will get easier with experience, but I was pulling my hair out last night.
I've got some homework I need to work on today, but if I have any spare time tonight, I'm going to hook in some sample code I found that creates a Qui cktime movie from a succession of images. Another thing I need to investigate is whether or not I stream the raw packed pixel snapshots to disk during capture and postprocess them once capture is done, or if I try to do realtime conversion to an image and then insertion into the movie. The realtime implementation would eventually let me turn an OpenGL view into a Quicktime broadcast stream, which has interesting implications. I think I'm going to do offline rendering first, since it's much more straightforward. Realtime *is* doable, though, since even at 60fps, I get 16ms per frame, with only 25% of that slice gobbled up by grabbing the image.
I bought O'Reilly's new Quicktime for Java developer's notebook, so I've been using that for high-level guidance on how to create the movie. It's a good book so far, I loaded the samples on my powerbook and the audio/video capture stuff worked on the first try. It's much easier to read the book for QT api guidance versus reading the Apple QT docs, which are in need of an update and polishing, IMO.
I found the Qucktime docs on my latest build, the old version I was working with didn't have up-to-date docs. QT7 is pretty cool, I'm going to try to port some Sequence Grabber code I have for capturing movies and see how it works. There were a couple issues with grabbing in QT6.5 that made it difficult to do live time-shifting. Turns out that while the SG is grabbing video, it's putting the video in a QT mov file, but doesn't finish writing the resource data until capture is stopped. The old API has options for not writing the resource data when doing a capture, and there were a couple other interesting things I was going to try (write movies into N second clips that are stored in a ring buffer and then flushed to disk), but I wanted to try fixing it with QT7 since CoreVideo makes it easier to do things like effect transitions or overlaying text. I was doing a NSView overlay on top of my video windows and using that to render text, but with CoreVideo, I should be able to draw the text inside the movie and get similar levels of performance. The QT6 way of rendering text in a video was a little slow, the simple demos I tried soaked up a lot of cpu on my powerbook.
Posted by djb at February 27, 2005 12:09 PM