Fun vertical sync hack

I was just playing with some code to scroll an image horizontally (am running on Windows XP). This code keeps an offscreen and periodically calls bitblt() to copy changed regions to the display.

As you probably know, most computer displays read from your video card's memory and "refresh" their display periodically. Of course, if you update the video memory while this scanning is happening, you get "tearing" (you see one part of one image and part of the next), and it looks ugly.

My app was looking ugly.

Good OSes prevent you from doing this, but GDI on Windows XP is not such a beast. (Vista and Win7 might do this right). For 3-d, lots of video cards are disabling vertical refresh waiting by default (probably for the benchmarks).

It turns out to be sort of painful to get a regular GDI window to do this right, but I found a pretty interesting optimization.

DirectDraw provides a GetScanLine method, so you can just call DirectDrawCreate, and then GetScanLine on the returned pointer. (It's a little more code for multiple monitors.) But simple vertical sync means you just busy-wait on this method to return a value less than the previous one. I've heard about people using multimedia timers to do this, without busy-waiting.

Of course, by waiting like this, you end up spinning a CPU for about 10ms everytime you want to redraw. It's not very fast and not very efficient.

But, it turns out that most applications don't redraw the whole screen on every draw. So because the GetScanLine API returns an actual value, you can wait for the scanline being drawn to be outside the region you want to draw. My application knows these numbers, and so it was easy to code up.

Since I'm redrawing only about 1/4 the screen every frame, this method requires ~1ms per frame, so it's almost not worth using multimedia timers at all.

Knowing how fast the monitor refreshes, you can also predict how long it might take to get the monitor to a "safe" region and use timers when you have to wait more than a millisecond.

But hey, if you know interesting and more power-efficient busy-wait methods, leave a comment.

What Native Code Really Means

I was working with Sree on a project about 12 years ago, and he insisted on using Win98 for development.

All of us kept asking, "Why won't you use NT4.0? It doesn't crash or lock up. You're rebooting 3 times per day." It made no sense.

Sree said, "80% of my customers have Win98. That's why."

Right now, people seem outraged that the new iPhone SDK bans use of the non-C-like languages or huge compatibility/portability frameworks: http://daringfireball.net/2010/04/iphone_agreement_bans_flash_compiler

And yes, it means more work for developers. But I'm cheering them on, and there are two reasons.

Reason #1: Speed

First, I tried some of the Maps apps on the iPad. You haven't used a computer until you've dragged a photo or a map around at full screen on this kind of device.

Maps draws at 60FPS all the time, no joke. It changes your life. It is immersive and responsive and real.

If you launch Safari on iPad and drag a map around in Google Maps, it's 3FPS, all the time. Like 20x slower. I don't really care why, and they've worked pretty hard to make Safari fast, but it's not even close to the native code.

Flash maps mostly run a bunch slower than Google Maps does. And across the board, interpreted stuff is slower than native. If you consider RAM and CPU together, it is hard to find an app with "portability" glue or an interpreted language that can run in <3x the speed of native code. And 3x really matters.

So Apple's got this amazing device, but mostly because they thought about all this and made the software sing. It is fast, it is a balanced machine with a GPU and CPU that work together to draw UIs really fast.

Even Google's nexus one can't seem to scroll a list or redraw a map at more than 15fps. And they wrote all the code (yeah, lots of it in Java). It is jerky and awkward in comparison to an iPhone 3gs. And that's all one team, not some engineer at another company doing the third-tier port of a portability framework in a hurry.

But now, Apple will be supporting multi-tasking and supporting background apps.

And honestly, they deserve to set the rules. You very simply shouldn't be writing a background app in Python that is 20x slower than the native one. You shouldn't be draining the battery to make a UI that has 1/5 the framerate of the native code. If using the GPU is better, use the GPU.

I love it that they're saying, do it the right way.

Reason #2: Get the UI right

So back to Sree using Win98 when everyone thought he was crazy. In 1998, WinNT 4.0 looked pretty much exactly like Win98. You used the same screen, the same mouse, and even the visual styling was pixel-identical. But the internals weren't the same. You made threads and Win98 was more jerky than NT. Memory wasn't cleared for you. DirectX was totally different.

Now people really believe they can make apps that port "effortlessly" from the desktop to a device with 10% the CPU power, 1/5 the screen resolution (multiple resolutions really), and they say with a straight face that this will be a good user experience?

That is simply not true. When you're designing for a mobile device, you should be thinking of every pixel, or how the slight differences in the screen size matter, of how the native controls work, of what the consistent way is to do animation (UI elements behave differently on different phones).

Buttons on Android are different sizes than iPhone. Tip of the iceberg.

You should be using win98, because your users are.

And of course, Apple is in a great position. People will bend over backwards to port code to their platform. They have no reason to make it easier, especially when that makes the user experience suck more. They can lop off the bottom 50 percentile of apps and it hurts them not one bit.

Really, forcing people who don't want to use their native, fast tools to use AJAX and HTML5 isn't such a bad punishment. It's a sure way to make portable apps, and it advances the idea of open and functional standards. It's a fine second tier.

There has been a trend for many years: the guys who write "scalable" apps for servers are happy to say they can buy 10x the machines, and this is usually cheaper than taking more time to get to market or using more engineers to do a task. They are smug about this, and it always drives people who care about using resources efficiently just a little bit nuts.

But most client developers have never been this way. Game engine guys don't think this way, and neither do native client app developers.

This is an incredibly exciting time, and this move will make some new possibilities based on people writing code that changes the world, that is massively better than what's been done before. It won't be the same old stuff, smaller.

Saving a few developer hours is not an excuse for making battery life awful and making apps that run at 5FPS, and forcing people to actually write an app for the platform they're deploying to is totally what I'd want on my platform.

Bravo Apple.

The Big Reveal

I think I'm seeing people who understand that computer UIs are part of a continuum, and the other people who still see stuff in a "big reveal" sort of way.

The ones who understand motion and calm....and the ones who understand big damn guitar solos.

Somehow the Microsoft and Adobe UIs have that big, guitars-screeching, moment where everything violently comes together, the moment you say, Ah, where you say, oh I'm glad I spent all this money. It's the TV channel ID animation 2% of the time, and the rest of the time it's kinda clunky.

And Apple has that moment of calm in between all the other smooth responsive intuitive stuff, they believe the reveal has to be all the freaking time, that making something continuous and animated and smart is better than the big progress bar with a clunk at the end. Or the other thing, that when you want it to be still is showing you tickers and motion, and when you want it to be responsive is totally locked up.

I think the truth is that it takes a lot more work to be calm. It takes a whole bunch of people saying, what if we sand this edge, and what if we make this a curve instead of a hard corner, and what if it felt like this? It's much easier to have a couple big important people say, we're going to spend our juice here...the rest can be old and boring, but we're going to make you really happy right now. It will look great on TV.

The crescendo, end of the world moment, where something distinctly unnatural happens, isn't how I want to interact with my world. The place with all the motion curves thought out, that leaves me calm is much better.

And I think people don't understand why this is better. But it is very easy to interrupt the human brain. You surprise me or make me feel grateful or create that emotional moment? You make me forget my goals, what I was doing, what the next step is....you interrupt flow.

If you make a simple machine that works perfectly....a door that closes so it doesn't rattle, even a bit, just clicks, then I go on with my life and do the next thing with my own focus intact and a continuity that matters. If I close a Microsoft Door, it wiggles a little bit to get my attention, and then it kinda sticks at the end because nobody really figured that this part of the animation would matter. If I close an Adobe Photoshop Door, it is kind of like closing my garage...it takes a while, and you have to watch it before you go through it. The Javascript door could work fine, except it's jerky, like 5 frames per second. There are even worse doors than this, ones that get stuck and ones that squeak so you think, wow I have to fix that, so compared to these old doors Microsoft and Adobe have done something interesting.

But yeah, Apple's doors are getting better every month. Sometimes you don't even have to say you want to close them, they just do. And other times they have the perfect motion curve so that if you're carrying something they close right behind you without touching. And that little click at the end is delightful.

And amazingly, they keep working on their doors because it seems to matter to them. They don't use the same one as last time.

But yeah, no more big reveals. Life isn't a slideshow, it's a continuous experience, and the little things matter, and it takes years and years to get this stuff right. Don't allow your user experience people to be disconnected from using and tweaking the software people actually use. If you don't work this way, you won't, um, know what door hit you in the butt on the way out.