Main | March 2005 »

February 27, 2005

New Tiger builds

I upgraded Tiger today to the newest build, 8A393. I was running a build that was a couple months old, so it's nice to pick up all the new bug fixes. The OS is getting more and more stable, it feels like Apple could have it ready by WWDC.

I have a theory re: CoreImage and the new Quartz features in Tiger. I think Apple has had these features available for their internal application groups for quite some time. Perhaps they weren't as pluggable/reusable as they are now, but thinking about Final Cut Pro, iPhoto, Keynote, Motion, etc, Apple has had similar capability for their apps for quite a while. I think they are releasing CoreImage with Tiger to see what third-party developers will do with all that graphics power. They've had the advantage for a few OS releases (I'm guessing they've had ImageUnit-like components since 10.1), so now they can make it available to others.

We've now got pluggable components for transforming audio, graphics, and video, and APIs to create pdf documents programmatically. Good printer support, zero conf networking, and a complete network stack. Streaming video/audio frameworks, a zero conf embedded sql engine and app data framework, etc, etc. The sense of power you get when sitting down at a Tiger workstation is incredible, you can write software to do ANYTHING. There are times when I'm working on code and just start laughing out loud. Working with OS X is just too much fun. I'm starting to think that I would very much enjoy writing OS X apps for a living.

Posted by djb at 10:16 PM | Comments (0)

Water, water, everywhere

I realized last night that there's a lot of stuff I'm working on, so I'm going to spend a few hours on Tuesday looking at all my current software projects. The office whiteboard just isn't cutting it, so I'm going to stick a list of everything in a big text file, sort it by priority, and pick two or three things I want to focus on. That doesn't mean I can't spend time on anything else, just that I'll be spending the bulk of my time on a handful of projects.

I've been thinking about taking some of my work and possibly turning it into a software product I could base a company on, so that will probably take a priority. The toy project stuff is fun, but it doesn't pay the mortgage. I've been talking to some ex-colleagues about available contract work, so I might do a part-time job to pay for school and living expenses, freeing me up to spend more time polishing the potential products without being totally freaked out by lack of funds.

Posted by djb at 12:35 PM | Comments (0)

OpenGL -> Image snapshots

I got code working last night that hits an OpenGL context and grabs the image from the frame buffer. I learned a few things, such as:

I've got some homework I need to work on today, but if I have any spare time tonight, I'm going to hook in some sample code I found that creates a Qui cktime movie from a succession of images. Another thing I need to investigate is whether or not I stream the raw packed pixel snapshots to disk during capture and postprocess them once capture is done, or if I try to do realtime conversion to an image and then insertion into the movie. The realtime implementation would eventually let me turn an OpenGL view into a Quicktime broadcast stream, which has interesting implications. I think I'm going to do offline rendering first, since it's much more straightforward. Realtime *is* doable, though, since even at 60fps, I get 16ms per frame, with only 25% of that slice gobbled up by grabbing the image.

I bought O'Reilly's new Quicktime for Java developer's notebook, so I've been using that for high-level guidance on how to create the movie. It's a good book so far, I loaded the samples on my powerbook and the audio/video capture stuff worked on the first try. It's much easier to read the book for QT api guidance versus reading the Apple QT docs, which are in need of an update and polishing, IMO.

I found the Qucktime docs on my latest build, the old version I was working with didn't have up-to-date docs. QT7 is pretty cool, I'm going to try to port some Sequence Grabber code I have for capturing movies and see how it works. There were a couple issues with grabbing in QT6.5 that made it difficult to do live time-shifting. Turns out that while the SG is grabbing video, it's putting the video in a QT mov file, but doesn't finish writing the resource data until capture is stopped. The old API has options for not writing the resource data when doing a capture, and there were a couple other interesting things I was going to try (write movies into N second clips that are stored in a ring buffer and then flushed to disk), but I wanted to try fixing it with QT7 since CoreVideo makes it easier to do things like effect transitions or overlaying text. I was doing a NSView overlay on top of my video windows and using that to render text, but with CoreVideo, I should be able to draw the text inside the movie and get similar levels of performance. The QT6 way of rendering text in a video was a little slow, the simple demos I tried soaked up a lot of cpu on my powerbook.

Posted by djb at 12:09 PM | Comments (0)

February 26, 2005

High-performance httpd components

One last note, I spent an evening last week doing some exploratory work on a foundation for an accelerated http server framework. I'm using a single-threaded select loop server based on libevent (which scales much higher than apache and other http implementations), and am working on the http layer now. My test platform (2 x 2.0Ghz Powermac) will run static apache requests at a max of 8k rps, but the raw libevent code will scale much higher than that (20k-25k) because its per-request overhead is much, much lower than apache. I'm slowly working on an efficient http core, taking hints from BSD-licensed http server code on the net.

Originally, I wanted to use servlets for this (because Java gives you a wealth of implementations for monitoring, configuration, middleware, etc), but they are just too much overhead. I've thought a little bit about getting one of the open servlet containers to use libevent through JNI, but it still feels like the servlet call chain for each request would be pretty slow.

I've been interested in http caches/accelerators for a while, so that's the arena I'm aiming for with the libevent httpd work I'm doing. A nice side-effect is when you're doing caches, your http layer doesn't need to be as complex as a bona-fide http server. There is still a lot of stuff to keep track of, but anything to reduce complexity is good.

Posted by djb at 04:30 PM | Comments (0)

OpenGL -> Quicktime

After the Ruby meeting, I resumed work on a project I've been working on. I have an OpenGL window which I'm rendering text and graphics into, but wanted the capability to create a Quicktime movie from the contents of the window. This is more difficult than you'd think, there are a few things to keep track of.

First of all, there doesn't seem to be a simple Cocoa framework call that renders a movie from an OpenGL window or view. This means I have to come up with a scheme of grabbing images from the OpenGL frame buffer and stick them into a Quicktime object.

Quicktime uses a graphics handle called a GWorld for most of its rendering calls. So, I've been working on a way to grab the pixels from the Open GL frame buffer (glGetPixels()) with a fixed clock to enforce frame rate (since simple OpenGL calls can run much faster than 30 or 60fps), and then turn the pixels into a frame of video which is then added to my GWorld, rendering it as a new frame in the movie.

I've been working on this on and off for a couple days, it's the hardest OS X coding task I've tried so far. I know I can get it to work, but I won't know how well it performs until it's done. It feels like this should't be a huge cpu drain, and if there are problems, I can always use my new friends Shark and OpenGL Profiler to help me out. Hopefully, I'll have this figured out and polished by next week.

Once the movie creation is working, then I'm going to hook in embedding of audio tracks with the video. This lets me add background music to the movies. I'm working on a prototype that lets me move the audio offsets around and draw the audio clip on a movie timeline window (think non-linear editing like FCP), so that will get polished and integrated.

The end result will be a framework that lets me render graphics/video/text to my OpenGL view, select tracks to play in the background, and render a completed Quicktime movie.

Posted by djb at 04:14 PM | Comments (0)

RubyCocoa slides

The talk on Thursday night went well! I went through the slides and gave an overview of RubyCocoa, and then we spent an hour hacking on the sample apps I wrote. The AudioGrapher app is pretty simple, it opens an audio file and graphs out its samples in two views. The top view shows the original samples, and the bottom view graphs transformations made to the original sample set. It has a slider to move around the audio file since the sample window is much smaller than the size of an average audio file.

It graphed the samples as dark gray lines on a light gray background in an interleaved format. One of the first changes we made was to draw left channel samples with red bars, and right channel samples with blue bars. This was interesting, but we were loading stereo audio files and wanted to see the fade graphically. I changed the app to render the left/right channel values on top of each, using 50% alpha blending on each channel to give a nice layered effect. We made a few other cosmetic changes to the view and changed its background color, the nice was that every change took at most a few minutes to make.

One neat non-gui feature we added was a check inside the ruby view class to detect changes to its source code. This let us edit the source while the app was running, and the app then automatically picked up the changes. Even though the build/startup time with RubyCocoa is pretty quick (~15 seconds), this reduces it to 0. Heh, heh.

I'm getting together with some of the seattle.rb hackers on Thursday night, we're going to clean up the changes I made to RubyCocoa to enable self-contained apps, and will be looking at ways to bundle third-party ruby code easily with RubyCocoa apps, and options for code obfuscation.

I've posted the slides from my talk online.

Posted by djb at 03:51 PM | Comments (0)

February 22, 2005

Short article on OS X Developer Tools

I mentioned a few days ago how I wanted to talk more about OS X's developer tools, so today I wrote a summary explaining my favorite tools and what tasks I find them useful for. I've been doing a fair amount of video and graphics programming the past several months, so my favorite tools are ones related to profiling/performance and graphics code debugging. Go take a look.

Posted by djb at 06:29 PM | Comments (0)

February 21, 2005

RubyCocoa self-contained app working!

I was able to tweak RubyCocoa's source and got a fully self-contained RubyCocoa app developed. The changes are ugly and not reusable by others yet (the path to RubyCocoa.framework's ruby libs is hardcoded into the RC runtime), but the darn thing works. I'm pretty excited. The bundled app is about 15MB in size, which is big, not the end of the world.

In order to get this working, I had to:

Now I need to work on making these changes usable by others and perhaps imported into the RubyCocoa project itself.

Posted by djb at 03:42 PM | Comments (0)

February 20, 2005

Analysis of RubyCocoa's internals

I spent some time today studying how RubyCocoa is implemented, since I like the concept of glue layers between languages and wanted to compare RubyCocoa with other layers I've seen before. After learning how it works, I'm very impressed with how RubyCocoa is implemented.

It has three main components. First, there is an ObjC header parsing script that finds all official ObjC headers on your system and generates ruby extension glue code that creates ruby wrapper objects for them. The parser is like SWIG but specialized to make the generated APIs more idiomatic to ruby developers. It's a little under 380 lines of code, which is made possible because Ruby's C extension API and interpreter data structures are pretty simple and well-designed.

The second component is a script that parses the major framework headers to find their class names, creating a small set of ruby libraries that just contain a generated set of class import commands. For example, the generated AppKit.rb looks at AppKit.h and notes that it has to load NSView, NSControl, etc, and creates import statements in ruby that register those classes for wrapping by the bridge.

The third component is a runtime bridge/glue layer that implements the imports that the second component calls. The imports create thin ObjC wrapper objects that only hold what class they wrap plus a small amount of metadata. When you create a OSX::NSArray object in ruby, the bridge code tells the ObjC runtime to create a NSArray object and store it its reference, which is then passed by the bridge back to the ruby OSX::NSArray wrapper class and stored away.

When you later make a method call on your wrapped array, the glue code simply parses your method to determine what selector was used, sending that selector to the object referenced by the id it stored earlier. A small library of type mapping code is used to massage arguments and return values when they are passed across the bridge. If the wrapped object doesn't support the selector, then an error is returned back to the ruby layer.

This is very elegant, since you're not updating hand-maintained glue code every time Apple releases software updates, and the glue is small and straightforward. This wouldn't have been possible with C++, but ObjC is dynamic enough to let you specify method names and receivers at runtime. A very cool hack that would have been a lot more LOC with other implementations and much harder to maintain as well. I love to see dynamic stuff like this.

And a note re: performance, the implementation of RubyCocoa makes it much easier to implement performance tuning and tune things like roundtrip latency across the bridge for method calls. Perhaps I will do some timings on the bridge to find hot spot candidates for inlining the Ruby code with C. There are much larger and more uniform gains to be made in performance with this kind of wrapper implementation versus a traditional fully hand maintained glue layer (writing C code utilizing Ruby's extension API) or a partially hand maintained glue layer like SWIG. I'm confident that RubyCocoa can be sped up.

Speaking of dynamic generation of glue code, I learned yesterday that JOGL uses something similar, and that one of JOGL's developers half-jokingly said their glue generation code was more valuable than the JOGL bindings themselves, since it could apply to creating JNI wrappers for other C libraries. I would argue they didn't have to say that half-jokingly; I'm learning more and more that a well-designed glue system is worth its weight in gold.

Posted by djb at 02:47 AM | Comments (0)

February 19, 2005

Quick ports with RubyCocoa

I ported my Objective-C audio app to RubyCocoa this afternoon, so now I've got a better feel for its pros/cons.

The method calls from the Ruby side are fairly similar to the native Objective-C calls, so in most cases, you can read the Apple docs for different classes and write corresponding Ruby calls on the first try.

Let's say you started developing your app with a NSView object that displays dvd video, and wanted to add a transparent view on top of it so you can show subtitles without a lot of fuss. In order to add the subview to your view, the Objective-C call would look like:

[dvdView addSubView:subview positioned:above relativeTo:otherView];

Calling the same method from ruby looks like this:

dvdView.addSubView(subview, :positioned, above, :relativeTo, otherView);

RubyCocoa provides other ways to mimic ObjC's selectors with Ruby method calls, but this technique is closest to the Smalltalk selector syntax that Cocoa uses. It makes porting the calls from the Apple docs easier IMO.

One of the big things I learned today is that you want to keep any NSView-like classes in Objective-C if possible. For example, NSView's drawRect method is called whenever you need to redisplay a view. The app I wrote has two 512x201 views that display audio data, with a slider attached to the pair of them to let users move around the sample set quickly. The redraw is snappy with a native Cocoa app, but RubyCocoa ends up being slow because the AppKit calls are going through the Ruby <-> Objective-C bridge.

The bridge delay is usually okay because in most cases, you're loading data periodically and setting controls and views in your window without doing any kind of continuous processing. When you're drawing a view though, your drawRect method could be called dozens of times a second. My Objective-C drawRect completes in a 3-5 milliseconds, but the Ruby one takes about 150 milliseconds. That's quite a performance difference!

The performance differential isn't a big deal for me since my major use of RubyCocoa is going to be for prototyping applications and deciding how their structure and business logic will work. RubyCocoa co-exists well with Objective-C controller/model/view classes, so it's not hard to implement time-critical classes in Objective-C.

My slides are about half-done, and I've got the audio app plus a skeleton framework ready for the live hacking on Thursday night. I was even able to add some features that the Objective-C app doesn't have yet! Ryan and I were chatting tonight about packing/unpacking of binary values, and we talked about RC and the preparations I'm making for the talk. The more progress I make, the more excited I get to give it, I think it's going to go well.

Once I finish the slides tomorrow morning, I'm going to spend some time on figuring out how to bundle ruby's extensions and libraries into RubyCocoa apps. I think a completely self-contained app would be cool, and bundling a ruby interpreter with full libs only adds 10MB to the app size, and nothing to the memory footprint. A fully contained app would also mean you had full control over what ruby libraries and versions were being used, so it would make the apps more reliable and remove the dependency on user installation and tweaking of external software and libraries.

Posted by djb at 09:21 PM | Comments (0)

February 18, 2005

RubyCocoa progress

I got the latest CVS version of RubyCocoa working well with Ruby 1.8.2 on my mac last night. There had been a few problems using the standard tarball, and I was encountering interesting doubly-defined symbol errors since RubyCocoa was statically linking in libruby, as was the ruby interpreter itself. I fixed the linking errors and ran across a new one related to NSApplication, so I did some googling and found people had better luck with the CVS version. It worked the first time I tried it, it was great to get the HelloWorld app up and running.

When RubyCocoa installs itself, it installs the RubyCocoa framework into the standard frameworks location on your host. The framework is linked to the version of ruby that was used to setup RubyCocoa, so you don't have to set environment variables, or play with executables, it just works. An added benefit is that you can then create honest-to-god OS X .app files that link against RubyCocoa's frameworks, and make clickable applications. Distribution of the application is a little dicey, since while ruby's interpreter is linked into RubyCocoa, the ruby files and extensions that implement non-core libraries are not.

I did some searching and found out that there has been some progress made on being able to bundle RubyCocoa apps that are entirely self-contained (which would be great, because then you don't have to ask users to download anything, they just download your .app file and start using it, never knowing that the core logic is in Ruby versus Obj-C or Java). It sounds like they are 80% there, but nobody has posted how to link in the libs/extensions properly. I'm going to take a shot at that tonight, I'm betting some TLC with ktrace will point the way.

I started writing up the slides for my talk, and will be writing a couple small apps this weekend for practice. I'd like to figure out the self-contained Ruby app problem before the talk, but if it doesn't work out, you can still run apps okay on a developer's host, it just makes redistribution difficult for users who want to try out the software.

I'll post screenshots and some sample code later this weekend to give you a taste of what working with RubyCocoa is like, and will also post the slides the day of my talk.

Posted by djb at 03:02 PM | Comments (0)

OpenGL is turtles all the way down

I'm really starting to like OpenGL. It does a lot more than I expected, I was expecting graphics primitives at a lower level. It's great to be able to specify an interpolation method, and then just start drawing points on the canvas, knowing that OpenGL will connect them, fill in the shapes, add dotted lines, or whatever other decorations I need. And it feels a bit like LOGO too!

I mention LOGO because before you place points on the canvas, you tell OpenGL what decoration to apply (take my points, and turn them into a filled polygon, or take my points, and draw colored lines between them with a width of 2 pixels). LOGO comes to mind because you're setting the color manually before each graphics operation, and there is a begin and end for every series of points. If you want to draw a red line parallel to a blue line, you have to set your color to red and say you're going to draw a line, then add two points to the canvas, then end the draw operation, repeating the series of steps to draw the blue line. It ends up making the code very easy to understand, although a little verbose.

The pseudocode looks like this:

set color red

prepare to draw line
place point(10, 0)
place point(20, 0)
stop drawing line

set color blue

prepare to draw line
place point(10, 10)
place point(20, 10)
stop drawing line

So I should modify what I said above about OpenGL being higher level than I expected. It lets me perform complicated tasks easily, but the way you tell it to draw shapes is about the level I expected. I don't mind having some structure there, it makes it much easier to think about scene rendering and what operations to do in what order. Too much syntactic sugar at the operation ordering level would end up making the graphics code undecipherable, IMO. It has a small set of rules that are easy to remember, and if you want syntactic sugar, I imagine it would be fairly easy to write a compiler that takes higher level primitives and generates the OpenGL commands to implement them, taking care of setting colors, ordering draw methods, etc, etc.

I bought a nice OpenGL reference this afternoon, so I'm looking forward to working my way through it. I think in a week or two, I'll be able to start tackling the 3d time-series graph I want to implement.

BTW, the JOGL book I bought is a bit of a disappointment. I've found a few bugs in the sample code and the formatting is pretty bad, even for a technical book. The visual noise and formatting inconsistencies make it more difficult to absorb the lessons the author is trying to teach me. I'm not slamming the author, I know that writing a book is difficult and the editing/publishing cycle can have surprises in it. I guess the book was good from the perspective of understanding how JOGL wraps native OpenGL libraries on different platforms, and to learn some tips/tricks for gotchas you wouldn't know otherwise, but I feel the real jumps in knowledge and confidence are going to come from the OpenGL bible I bought today, even though all its examples are in C.

I've got JOGL added to my eclipse tree, so I don't need a reference for JOGL itself. I love how eclipse will load a jar, parse out javadoc if it's there, and then auto-complete method names and arguments for you when you're writing code. The pop-ups for method signatures (and the side window showing the doc for that method) make exploring unfamiliar frameworks very quick and painless. I shudder to think how frustrated and slow I'd be if I were trying to do java work with makefiles and vi or emacs. Did that before, and will never do that again.

p.s. Kudos if you caught the connection between my post title and LOGO. I have a soft spot for LOGO since I spent a lot of time playing with it when I was first getting into computers and software.

Posted by djb at 02:25 PM | Comments (0)

February 17, 2005

Shifting gears to Ruby

I'm giving a talk on RubyCocoa to the Seattle Ruby user's group next week. I was the second member of the group, which was founded by Ryan Davis, a software developer I met while working at Amazon, who now works for the Omni Group.

If you haven't heard of Ruby before, it's an OO scripting language that takes inspiration from Perl, Python and Smalltalk, and is hugely popular in Japan. I very much prefer it to Perl, it was designed for OO development from the start and isn't as idiosyncratic as Perl and much more readable to boot.

RubyCocoa is a set of Ruby bindings for OS X's Cocoa and AppleScript frameworks, it lets you write OS X apps in ruby. Since OS X implements the V in MVC for you, apps need only implement a model and a controller. A stock install of OS X lets developers write Cocoa apps in Objective-C and Java, and while these are fine languages for developing applications, they can get in the way when you're creating prototypes or trying to do rapid development.

Enter RubyCocoa, which lets you write models 10x faster than Java or Objective-C. It is an interpreted language, so you don't have to wait for builds, and Ruby itself is more extensible than Java or Objective-C, which lets you focus on sketching out features and working on your prototypes. I think RubyCocoa is a strong technology for prototyping OS X apps, so I'm going to talk about its pros and cons to the group.

The format of my talk will be fun, since it's a live hacking talk. I've done this a couple times for past meetings, and it was always well-received. Basically, I do an interactive session where I ask people for requirements for an application, and then I develop it in front of the group. It makes for a fun environment, since there are always a couple good-natured hecklers who will ask why I'm adding arguments to a method, or refactoring a class, etc, etc. It's also a good way to instruct the novice developers in the audience how to reliably build apps from the ground-up.

The RubyCocoa app I'll be writing for the group is going to be some sort of content browser that makes soap calls to an online resource, I'm not sure who yet. Suggestions have included Amazon and Google, and I bet there are a couple others we could try. I'll be spending some time over the next couple days making sure my RubyCocoa install is up to date, that all the tools and debuggers are working, and do a little refresher work to make sure I remember how to glue all the pieces together. I'm going to write the core controller callbacks and instantiate a blank window and canvas before the presentation, but that's it, the adding of UI elements and connecting them to the model will all be done live. It should be a lot of fun.

I'm planning to do a future talk on using Apple's Accelerate framework from Ruby. Accelerate framework contains Altivec-accelerated routines for math operations, matrix calculations, equation solving, and different types of FFTs. I'm planning to write a ruby extension that wraps Accelerate framework, my calculations show that Ruby plus wrapped Altivec will be much faster than straight C code using FFT-W. The benefit of having fast FFTs in ruby is that it's much easier to shove data around and transform it with ruby than Obj-C or Java, so it will make it easier to prototype DSP algorithms and enable hard-core number crunching with Ruby on OS X.

Posted by djb at 03:38 PM | Comments (0)

OpenGL explorations

OpenGL is turning out to be quite fun! I haven't tried any of GLUT's features out yet, but I've got a simple java app now that draws a cartesian coordinate system on a canvas, along with a light grey background grid. I've found a few bugs/typos in the example code from the book I'm reading, so those were fun to find and fix. I think I'm going to need to get an additional reference book that covers OpenGL in general, since there are a lot of shapes and drawing methods to keep track of.

Skimming forward, I can see I'm going to be learning about defining wireframes, laying textures on them, and positioning and configuring lights to apply shadows. One big thing I've been itching to do with OpenGL is 3d rendering of audio samples to show how the frequency spectrum of an audio clip varies over time. When thinking about the x/y/z rendering, I realized that I have another dimension of data available to me, namely the color of pixels in the Z axis. I could apply a color gradient from dark to light that rises from the floor of the graph to the peaks, making it easier to spot outliers from the average data.

I've downloaded some JOGL examples and tried them out, they were doing some tough number crunching. One of them was a large textured logo with realtime waves and shadows applied to its surface. Just to make sure JOGL was working okay, I booted up OS X's Open GL Profiler app. This app is going to be my bread and butter over the coming weeks. It lets you attach to a process and sample its OpenGL calls to see what frame rate it's using, and you can even grab a trace of calls over a period of time, and then see the raw api calls in a window, kind of like how a debugger can show you disassembled asm code for a C function. The freaky thing is that you can attach the OpenGL Profiler to any app running, so you can peek at games running on your mac and see how they implement their 3d rendering.

I think I'm going to need to make a future entry discussing OS X's terrific developer tools. Most people are aware of Xcode and Interface Builder, but there's much more stuff available. Now, developing apps on OS X isn't painless yet, and it's still a little difficult to port unix apps over, but when you look at the platform and include all of the great tools for developers, it just might be the ultimate platform for software development, beating out Windows, Linux, BSD, BeOS, etc, etc. I'll share my thoughts on what tools I find useful in a future entry.

Posted by djb at 02:15 PM | Comments (0)

February 16, 2005

JOGL -> Java OpenGL

In tandem with the development work I've been doing with CoreImage, I've also been learning more about the non-shader bits of OpenGL. I bought a book on the java bindings for OpenGL (JOGL) and have started working through the examples.

It's quite topical for me, since I recently wrote a Cocoa application that takes audio samples and applies transformations to them, graphing the time-series data before and after transformation. (I'm using the app to help me develop audio DSP algorithms.)

It turns out that OpenGL has primitives for doing coordinate transforms, graphing equations, connecting vertices, and all sorts of stuff I was doing by hand inside an Objective-C NSView subclass. I had been considering rewriting the app to use ObjC for the controller and java for the models and subviews, but JOGL seems like it could give me a quantum leap in "lickability" of my sample graphs.

I don't want to jump right in and start refactoring without a clear path for where I'm going, but I'm going to write a couple standalone JOGL apps and see if it can do what I'm looking for. I'm keeping an open mind as well, trying out samples that do 3d pie charts, ray tracing, etc. I've always wanted a good excuse to learn OpenGL and now I have it. I figured it would be easier to learn OpenGL using java bindings for it versus using Apple's Cocoa OpenGL wrappers or writing straight C/C++ code.

I've been working with Java on a near-daily basis for the past six months, but my positive experiences there will have to wait for a future entry. Suffice to say that I'm excited to be using JOGL and learning more about industrial-strength graphics rendering, and I'm looking forward to putting this knowledge to good use on future software projects I work on. There are several graphing and visualization features that I've had on the drawing board for a while, but now I have the tools that will let me implement them. Pretty exciting stuff.

Posted by djb at 10:27 PM | Comments (0)

<Media>Units

When I bought my G5 workstation this fall, I also signed up for a membership with Apple's Developer Connection that gives me access to pre-release versions of their next major OS release, Tiger. One of Tiger's most interesting new features is a framework called CoreImage, which allows developers to write image transform components that work in a similar way as Apple's AudioUnits work with audio. They are generic media transform components/plugins that are designed to easily plug in to OS X applications.

For example, Apple's Garage Band (an app that lets musicians play/record/loop songs and samples) has support for AudioUnit plugins built in. When you're playing a virtual instrument, you can add a chain of AudioUnits that add echoes or reverb, adjust the EQ levels of the audio, make the tone sound watery, just about anything you desire. The genius part is that AUs aren't an Apple-only thing, any developer can write AUs for use with OS X. This means I can write an AU that reads samples and graphs them in a pop-up Cocoa window, and then use that same AU when I'm using any product that supports AU filtering of audio streams.

Now that I've given you a taste of AUs, let me get back to CoreImage and ImageUnits. My developer token includes a pretty strict NDA, but I can talk about things that are public knowledge. One of the interesting aspects of CoreImage components is that the transformation kernels (the chunks of code that do the raw pixel manipulation) use a subset of the OpenGL Shading Language. Shaders have been used for a good while in the game industry, but only recently did a cross-platform language spec get finalized for writing generic shaders that work across multiple GPU platforms and operating systems.

A shader is basically a shim into the OpenGL rendering pipeline that lets application developers specify how to transform a canvas of OpenGL pixels. Shaders can do things like apply texture maps, generate shadows based on the defined light sources, transform the geometry of the image, and many other actions. Different video card drivers and rendering programs had their own shader implementations in the past, but using OGLSL (OpenGL Shading Language) makes the IUs more pluggable than if they used proprietary shading languages.

Another great thing about CoreImage is that ImageUnits that you write for transforming images can also be applied to transforming video using the CoreVideo framework. And because the shaders and the pixel processing pipelines that feed data into and out of the units are all based on OpenGL and optimized to move pixels around quickly, they are blazing fast.

I just started working my way through a book on OGLSL that will help me get bootstrapped with IUs more quickly. I've built some sample code and seen what it can do, but now I need to get dirty and implement the transforms that I require. BTW, CoreImage itself comes with 95+ ImageUnits that can be used to perform common image transforms, so unless you have some twisted requirements, they will suffice for most developers of image/video apps. Unfortunately, I have a few IUs I need to write that haven't been implemented by Apple, but this should be a fun learning experience.

Posted by djb at 09:46 PM | Comments (0)

My Blog

This is my new blog to help me disseminate random musings on software development and theory. I've been doing a lot of software development since I left my job six months ago, and I figured it was time to start putting my ideas online to share with others. I hope you enjoy it.

Posted by djb at 08:45 PM | Comments (0)

Copyright 2005 © djb. All Rights Reserved.