Shared publicly  - 
This weekend's dose of interesting facts.
Dianne Hackborn originally shared:
How about some Android graphics true facts?

I get tired of seeing so much misinformation posted and repeated all over the place about how graphics rendering works on Android. Here is some truth:

• Android has always used some hardware accelerated drawing. Since before 1.0 all window compositing to the display has been done with hardware.

• This means that many of the animations you see have always been hardware accelerated: menus being shown, sliding the notification shade, transitions between activities, pop-ups and dialogs showing and hiding, etc.

• Android did historically use software to render the contents of each window. For example in a UI like there are four windows: the status bar, the wallpaper, the launcher on top of the wallpaper, and the menu. If one of the windows updates its contents, such as highlighting a menu item, then (prior to 3.0) software is used to draw the new contents of that window; however none of the other windows are redrawn at all, and the re-composition of the windows is done in hardware. Likewise, any movement of the windows such as the menu going up and down is all hardware rendering.

• Looking at drawing inside of a window, you don’t necessarily need to do this in hardware to achieve full 60fps rendering. This depends very much on the number of pixels in your display and the speed of your CPU. For example, Nexus S has no trouble doing 60fps rendering of all the normal stuff you see in the Android UI like scrolling lists on its 800x480 screen. The original Droid however struggled with a similar screen resolution.

• "Full" hardware accelerated drawing within a window was added in Android 3.0. The implementation in Android 4.0 is not any more full than in 3.0. Starting with 3.0, if you set the flag in your app saying that hardware accelerated drawing is allowed, then all drawing to the application’s windows will be done with the GPU. The main change in this regard in Android 4.0 is that now apps that are explicitly targeting 4.0 or higher will have acceleration enabled by default rather than having to put android:handwareAccelerated="true" in their manifest. (And the reason this isn’t just turned on for all existing applications is that some types of drawing operations can’t be supported well in hardware and it also impacts the behavior when an application asks to have a part of its UI updated. Forcing hardware accelerated drawing upon existing apps will break a significant number of them, from subtly to significantly.)

• Hardware accelerated drawing is not all full of win. For example on the PVR drivers of devices like the Nexus S and Galaxy Nexus, simply starting to use OpenGL in a process eats about 8MB of RAM. Given that our process overhead is about 2MB, this is pretty huge. That RAM takes away from other things, such as the number of background processes that can be kept running, potentially slowing down things like app switching.

• Because of the overhead of OpenGL, one may very well not want to use it for drawing. For example some of the work we are doing to make Android 4.0 run well on the Nexus S has involved turning off hardware accelerated drawing in parts of the UI so we don’t lose 8MB of RAM in the system process, another 8MB in the phone process, another 8MB in the system UI process, etc. Trust me, you won’t notice -- there is just no benefit on that device in using OpenGL to draw something like the status bar, even with fancy animations going on in there.

• Hardware accelerated drawing is not a magical silver bullet to butter-smooth UI. There are many different efforts that have been going on towards this, such as improved scheduling of foreground vs. background threads in 1.6, rewriting the input system in 2.3, strict mode, concurrent garbage collection, loaders, etc. If you want to achieve 60fps, you have 20 milliseconds to handle each frame. This is not a lot of time. Just touching the flash storage system in the thread that is running the UI can in some cases introduce a delay that puts you out of that timing window, especially if you are writing to storage.

• A recent example of the kinds of interesting things that impact UI smoothness: we noticed that ICS on Nexus S was actually less smooth when scrolling through lists than it was on Gingerbread. It turned out that the reason for this was due to subtle changes in timing, so that sometimes in ICS as the app was retrieving touch events and drawing the screen, it would go to get the next event slightly before it was ready, causing it to visibly miss a frame while tracking the finger even though it was drawing the screen at a solid 60fps.

• When people have historically compared web browser scrolling between Android and iOS, most of the differences they are seeing are not due to hardware accelerated drawing. Originally Android went a different route for its web page rendering and made different compromises: the web page is turned in to a display list, which is continually rendered to the screen, instead of using tiles. This has the benefit that scrolling and zooming never have artifacts of tiles that haven’t yet been drawn. Its downside is that as the graphics on the web page get more complicated to draw the frame rate goes down. As of Android 3.0, the browser now uses tiles, so it can maintain a consistent frame rate as you scroll or zoom, with the negative of having artifacts when newly needed tiles can’t be rendered quickly enough. The tiles themselves are rendered in software, which I believe is the case for iOS as well. (And this tile-based approach could be used prior to 3.0 without hardware accelerated drawing; as mentioned previously, the Nexus S CPU can easily draw the tiles to the window at 60fps.)

• Hardware accleration does not magically make drawing performance problems disappear. There is still a limit to how much the GPU can do. A recent interesting example of this is tablets built with Tegra 2 -- that GPU can touch every pixel of a 1024x800 screen about 2.5 times at 60fps. Now consider the Android 3.0 tablet home screen where you are switching to the all apps list: you need to draw the background (1x all pixels), then the layer of shortcuts and widgets (let’s be nice and say this is .5x all pixels), then the black background of all apps (1x all pixels), and the icons and labels of all apps (.5x all pixels). We’ve already blown our per-pixel budget, and we haven’t even composited the separate windows to the final display yet. To get 60fps animation, Android 3.0 and later use a number of tricks. A big one is that it tries to put all windows into overlays instead of having to copy them to the framebuffer with the GPU. In the case here even with that we are still over-budget, but we have another trick: because the wallpaper on Android is in a separate window, we can make this window larger than the screen to hold the entire bitmap. Now, as you scroll, the movement of the background doesn’t require any drawing, just moving its window... and because this window is in an overlay, it doesn’t even need to be composited to the screen with the GPU.

• As device screen resolution goes up, achieving a 60fps UI is closely related to GPU speed and especially the GPU’s memory bus bandwidth. In fact, if you want to get an idea of the performance of a piece of hardware, always pay close attention to the memory bus bandwidth. There are plenty of times where the CPU (especially with those wonderful NEON instructions) can go a lot faster than the memory bus.
James Forestier's profile photoJoseph Ferguson's profile photoAnthony FLORES's profile photoMoises De La Cruz's profile photo
Very interesting! I have always wondered though, since so many iOS proponents tout hardware acceleration as the reason that iOS is much better with animations than Android (although I have never had an iOS device, so I can't really compare - however, I can tell you that my OG Droid is pretty terrible (anxiously awaiting a Galaxy Nexus!)), is there another reason for this perception? I have always assumed this was just a matter of people comparing to under-powered and under-spec'd Android phones. But, then, does that mean that iOS simply has always had better hardware than was needed from the software? Is it a matter of iPhone before version 4 being so featureless that it basically didn't require any power to run? Was it a matter of Android being too feature-heavy and ahead of its time for hardware available, and we are just in the last year or so catching up? I feel like I'm framing these questions in a way that makes Android look better than iPhone out of the gate - but I've always loved the OS, it's features, and it's usability much more than Apple's take on things (although, again, never had an Apple product, just not a fan of how they treat their customers).
Tim Box
Very timely, I was looking for that info this week end. I have to ask the inevitable question why do apps like Pulse have so much trouble scrolling? And what is iOS & WP7 doing that makes it able to look smoother. Is it a case that they do not try to do to much, and is a reason they do not have widgets i.e. they could not make it look good?
I understand the desire to point out that this isn't purely an issue of HW accelerated vs non-accelerated. OTOH, I don't think the people actually buying phones care about this distinction at all. What actually does cause the stuttering, and why hasn't it been resolved in the three or so years that Android has been out? Until it is resolved, people will always come up with simple sounding "solutions" (HW acceleration). Once (if) it is resolved, all of those things will be forgotten. FWIW, the foreground vs background priority thing isn't really that great in 2.3 or 3.0, IMO. The system still gets really slow and stuttery whenever there is memory pressure.
i've read just this great article about "how the ui works" - but have a small question: the last days i have compiled ICS for the LG P500 device (its MSM7x27/Adreno200 based) and we (the developers) cant enable HW acceleration. Are these devices really to old, or do we need new proprietary firmware, new proprietary userspace drivers and a new kgsl module in kernel? BTW: the device is booting to the UI but all is very slow w/o hw acceleration.
Pretty epic, considering I understood almost all of it.
How does this impact CSS3 animations and transitions in the browser? What is iOS doing that makes CSS3 sing where it still stutters and jerks about on Android? This seems vastly improved in ICS, but I have yet to test its limits.

Also, I've often wondered about the JavaVM-on-Linux approach Google took with Android? Are the abstraction layers always going to add latency? Do you envisage a future where all Android applications are written in Go (or similar) and compiled to native code?

+Andreas Bratfisch: you can petition Qualcomm for ICS drivers for that SoC here:
I have an experimental ICS build running on my very similar HTC Desire with hardware acceleration, so there are definitely some tricks the community can deploy. Unfortunately, it still uses a Gingerbread kernel, so it is missing necessary hooks such as per-application monitoring of data usage. For the best result, the manufacturer needs to release their kernel sources, and the component manufacturers need to driver sources too.
Tim Box
There is a really great follow up to this.

It basically says that Android was designed as a Blackberry alternative and was rushed out to compete with the iphone. So the priority to make the UI no 1 was not there. the Iphone and WP7 have the UI as main feature so that is why they are a lot smoother.
I think most people are excited about ICS as they think it will fix the lag. They will not be happy when its turns out not to.
Well written. However, I ended-up feeling this post like an "excuse" rather than an acknowledgement of a problem and a proposal for a solution. It seems like only a complete over-haul of the rendering paradign is needed, rather than patches and the constant need of throwing hardware at the problem. The current direction is not what anyone would expect from Google, as it is well known for obsessing about efficiency and technical marvels.
I think that your article is very good and explains simply why the Android UI gets so laggy. In my opinion, ICS has yet to do, to compete with iOS. 
Add a comment...