By - 9 Posts - Public

Tales from the trenches. ● https://flutter.io/ ● (Photo by Ramesh NG)

Stream

Ian Hickson

Shared publicly  - 
 
The last week or so we've been working on polish issues so that our Flutter Gallery demo app looks and feels so good that anyone trying it is left with no doubt that Flutter is a serious option for writing mobile apps. This means looking at things like performance during page transitions, removing demos that aren't complete, and fixing silly trivial bugs that we've ignored for too long. In the latter category, for example, we have the bug I've been looking at today: getting the scroll friction in flings in lists to feel natural. This basically means figuring out what the magic number should be by tweaking it slowly bit by bit until it matches the platform! You can see my crazy setup in the photo below. I put two devices side by side and then do a fling gesture on both and see how close it gets.

Turns out iOS is very slippery, 0.125. Android is high friction, 0.002 if I remember correctly.
13
4
Add a comment...

Ian Hickson

Shared publicly  - 
 
This week was Google I/O, and as part of that we release a codelab! If you want to play with Flutter but our documentation so far has been a bit daunting, now's the time to try a tutorial with lots of guardrails.

https://codelabs.developers.google.com/codelabs/flutter/

Also, Adam recorded a tech talk following on from mine. He focuses on the rendering pipeline, talking about how we perform layout, how we do painting, and how we composite the final scene. In particular he goes into more detail than I did about why a lot of that can go really fast in Flutter while still having a really flexible layout system.

https://www.youtube.com/watch?v=UUfXWzp0-DU

Down in the trenches, today I finished implementing a big set of changes to our test framework. You can now use "flutter run" to actually watch your tests run. This is going to be hugely helpful in developing tests. Before, you had to write them essentially blind, because tests ran in this alternative universe where the clock runs very fast and where we don't bother actually computing any graphics, which meant that it was really hard to visualise exactly what was going on. Now, as you write the test you can run it right on the device, in real time.

The last major bug I had to fix with the test framework changes was in our handling of actual taps on the test. One of the things I sometimes have trouble with is figuring out how to write a finder to hit a particular widget. So I figured I'd make it so that when you tap the screen when you're running one of these live tests, it would dump to the console a list of possible finders you could use to get to the widgets at the point where you tapped the screen.

Now, writing test harnesses is always a bit of a confusing matter, because the whole point of a test framework is that it spends half its time lying to the rest of the system about what's going on. In particular, when tests are being run live, the Flutter test framework has to lie to the Flutter rendering framework about the size of the screen, because we have the convention that tests run in an 800x600 viewport with a device pixel ratio of 1.0, but the device is unlikely to be exactly 800x600@1.0. So to make it render right, I have the test framework subclass the binding, and then insert a transform when painting.

The problem now becomes how to handle hit testing. The fake events from the tests are in this 800x600 coordinate space, but the real events from the device are in the device's coordinate space!

To make this work, I tried various approaches. At one point, I tried to duplicate the rendering binding's hit testing logic, so I copied and pasted its implementation into the test binding. Then later I realised that wouldn't work, and instead I changed the event dispatch logic so that it would convert the test events into the "real" coordinate space, then dispatch those, then have the binding convert them back to the test coordinate space. That way, all the events (the fake ones from the test and the real ones from the device) would go through a single codepath, and I just needed to filter them out after hit testing to figure out what to do with them. As part of this, I created a Matrix4 with the transformation needed for converting "test" coordinate space units into "real" coordinate space units, and had a method that would take that matrix and invert it for the opposite (which is what I usually needed for hit testing).

This all worked great! All the tests ran fine with their fake events, and if I tapped the screen I got the right coordinate and all was good.

Until I hit the screen the second time. The second time, the coordinates were off. But the third time they were right! Then the fourth they were wrong again.

There then came one of the most confusing couple of hours of debugging I've had to do for a while. I eventually figured out that what was going on was that my method for inverting the matrix was actually inverting the "canonical" copy of the matrix, which meant that every other hit-test was using the wrong coordinates. But if I fixed that, then suddenly all the tests broke! And they broke quite badly, with assertions all over the place.

Finally I figured out why. The hit test function that I'd copied-and-pasted from the rendering binding into the test binding carefully did all the hit testing, then invoked the superclass hit test method, which should normally have been the gesture detector binding's but in this case was the rendering library's binding's, which did its own set of hit testing all over again. Which meant that when the tests sent any events, they first inverted the matrix, used that to hit test the widgets, then inverted the matrix again, hit test all the widgets again (getting a completely different answer), and then sent the events to both places. Since the wrong place didn't do anything (it was likely "off-screen"), the effect was that the second set of hit testing had no effect except putting the matrix back to its correct configuration. But when I fixed the inversion bug, well now every tap in every test would result in two taps in the same place, confusing the tests mightily.

A weird case of two wrongs making a right!
10
1
Add a comment...

Ian Hickson

Shared publicly  - 
 
Over the past few weeks the Flutter team has been mostly focused on performance and testing.

💠‬

For performance, the first step of course is measurement. To this end, Yegor has started doing continuous automated testing on actual devices, so that we have a clear baseline against which to measure our progress. Of course to make that useful we have to have performance tests, so Viktor wrote a test with some complicated layouts. This immediately uncovered that my MixedViewport class was terrible, so Adam rewrote that as LazyBlock. And on it goes.

LazyBlock is pretty neat. Instead of just giving it a list of children, you give it a delegate that produces children on demand, and LazyBlock only asks it for the children that it needs. As you scroll, it throws away the earlier children, and asks for the next ones. If you go back up, it does the same thing backwards. Eventually we'll even make it guess the right child number if you jump scroll, so that you can jump around a huge list of children without ever having to pay the cost of drawing any children other than the ones on the screen.

One widget we don't yet have is a LazyTable. That's something I plan to add soon. I did implement DataTable recently, you can see the demo in our gallery app which exactly matches the sample in the Material Design specification. The next step though is a paginated version and a scrolling version, reusing the same delegate approach that Adam used for LazyBlock. (Tables are tricky because you have to figure out what size to make the columns; for DataTable, it actually measures every cell before laying out any rows, which is quite expensive. For LazyTable, we'll have to use either fixed width columns or flex-style sizing.)

Lots of other work on performance has been going on too. Eric and Devon hooked things up so that from the Atom editor you can now with one checkbox turn on and off our PerformanceOverlay widget and our "repaint rainbow" feature, live while your app is running. Hans has been working on improving the performance of our gallery app. Seth has been running lots of our demo apps tracking down any where we miss frames and filing bugs on them. We even had the help of a nine year old yesterday, who found a bunch of bugs for us!

You can see a screenshot of the checkbox on the PR: https://github.com/dart-atom/dartlang/pull/960

💠‬

The other big push recently has been on testing. As we like to say, "write test find bug". The problem is that writing tests for Flutter has been a bit difficult so far, because you can't see what you're testing.

Yegor has been working on a way to test on-device, using "flutter drive", and as part of that he redesigned our API for testing in a way that makes a lot more sense. (Flutter drive is what he used to do the continuous performance testing I mentioned earlier.) I've now picked that up and I'm taking it to its logical conclusion: make it so that even for our unit tests, you can actually see what's going on if you run them on a device. The hope is that this will really simplify writing tests for Flutter, which will mean we write more tests (and hopefully will mean people writing apps for Flutter can write more tests, since we use the same testing strategy for people writing Flutter apps as we do on Flutter itself), which will mean finding more bugs! 
18
7
Add a comment...

Ian Hickson

Shared publicly  - 
 
Lots of progress recently in the Flutter universe. The prettiest progress was a demo that Viktor landed showing how to integrate our sprites library with our Material widgets library, which looks really sweet. I included a screenshot below, and you can see the source at: https://goo.gl/BvNnNp

Hans has been working on implementing the various patterns described in the Material Design "Scrolling Techniques" page, which has a lot of subtlety. I've included some screenshots of this below too. Hopefully if we can build all those techniques into our widget library, applications written with Flutter will be able to replicate those patterns easily by just setting a few flags. One open question is how much we should bake into the Scaffold widget and how easily we can make things overridable so that you can provide your own variations.

Meanwhile, I continued to work on making Flutter apps accessible. I've been focusing on Android so far, and I've hit a major milestone: you can now navigate Flutter apps with the TalkBack service, hear the labels, and press buttons! You can see this in one of the screenshots below, where I've turned on TalkBack's captions so you can see what it's reading out. There's still lots to do, and it's still relatively buggy, but I think we have a good foundation.

27
16
Raju Bitter's profile photoIan Hickson's profile photo
2 comments
Add a comment...

Ian Hickson

Shared publicly  - 
 
The big thing we're testing in Flutter land right now is Hot Reload. This is a feature from the Dart team that's going to make Flutter development massively easier. Instead of having to restart your app each time you make a change, you just have your IDE signal your device that the source code has changed, and we load the new code straight into the actively running VM without losing any of your state! Your app could be in the middle of an animation, and boom, half-way through we upload the new code and the animation continues running but with the new settings, without interrupting the animation or anything.

I've been using this feature to write code for a few days now and it is astounding what a difference it makes. You can try it today if you're using the master branch, just use "flutter run --hot". It's still a bit buggy (we don't yet support loading assets in this mode, for example, which is obviously a bit limiting) but that should get shaken out in a few days and then we'll land it on the alpha branch.

Conveniently, our framework really fits the hot reload model well. When you trigger a hot reload, we just mark every widget as needing to be rebuilt, and the whole app gets the new code applied, without losing state. This means you can tweak build functions, theme settings, and so on, and when you reload, the code takes effect. It would have been much harder to do this if our framework used a more retained-mode model like traditional UI frameworks, where once you've described the UI it stays in memory.

In other news, we now have a real logo! We've started deploying it, you can see it on our Web site and our G+ page and so on. I've been writing a widget that renders the logo, so that you can fly our colours if you want to. The logo is a heavily stylised blue wing in the shape of a sideways F. By the way, using hot reload makes writing canvas code, like this logo drawing code, so much easier. You just edit the coordinates, hit a key to save your changes, look at the device, and repeat.

The past few weeks I've been making sure our licensing story is nice and straight, including adding an AboutDialog widget and corresponding license screen. (Nothing drives home quite how many shoulders Flutter is standing on than paging through all the licenses of third-party packages Flutter uses. It really is a testament to the power of open source development.) Anyway, that's why I'm writing a FlutterLogo widget: my plan is, by default, to include it at the top of the license screen.
21
5
Add a comment...

Ian Hickson

Shared publicly  - 
 
For today's Flutter update, I will just describe what I'm currently trying to do.
I want to write a PaginatedDataTable widget. This is a wrapper around a DataTable, which I implemented a few weeks ago, with the header and footer that you can see in the Material Design specification:

https://www.google.com/design/spec/components/data-tables.html#data-tables-tables-within-cards

Doing this required me to make some changes involving the intrinsic sizing part of our RenderObject API (which I designed). This was the final straw for us, though. We've been feeling that this API was too complicated for a long time. It takes BoxConstraints, but most implementations don't know how to really use them, and most callers just pass unconstrained values.

So I start working on simplifying our intrinsic sizing API. Instead of BoxConstraints (which is min/max width and min/max height), I'll just pass in a single value, the height (for intrinsic width APIs) or the width (for intrinsic height APIs).

For reasons I've now forgotten, doing this required me to do something involving our graphics API, but the documentation for that part of our API was defficient. We try to follow the rule that if you look something up in our documentation and don't find the answer, you have to figure out the answer and then add it to the documentation. So...

I start a big project of documenting our core graphics API. It was about half-done before, with mainly the more complicated bits left, but as a side-effect this will push me to learn our graphics API better.

In doing this I find that the image we use to explain TransferMode has broken. It turns out that when I added that a few months ago, I did it by just having the Skia "fiddle" tool run a little C++ app on the fly, but since then the APIs have changed and so the app no longer works.

So I fix the app, but, additionally, to prevent the image from breaking in the future, I make a copy of it to store on our Web site so that the documentation can point to that instead.

In doing this, I find our Web site has some errors that are preventing it from uploading the image. Our Web site runs a script that analyses all the sample code and you can't update the site until the samples are fixed. This has been a great help in keeping our site up to date. Although it doesn't stop the site from becoming wrong when we change the APIs, it does let us know when we try to fix something else.

So I submit a patch to fix the sample code in the documentation for how to test a Flutter program so that it uses the new API.

In doing this, I found our Web site deployment script is using an option that no longer exists in Firebase. Turns out that Firebase was updated recently and we haven't yet updated to the new configuration.

And this is why trying to write a PaginatedDataTable widget involves editing a firebase.json file.
9
Add a comment...

Ian Hickson

Shared publicly  - 
 
For today's update, I give you: a video about Flutter!

We've started recording the occasional tech talk about Flutter's technology stack. We're more or less covering topics at random; in this talk I mostly try to answer the frequently asked question "Why does StatelessWidget have a build function on its widget while StatefulWidget has its build function on its State". Adam has a talk prepared where he goes through and explains our rendering pipeline, which hopefully I'll be able to share with you in a few weeks. If there's specific topics you think would benefit from this kind of approach, please don't hesitate to leave a comment.

I'm hoping to turn this talk into a document at some point. I have the speakers notes, which are more or less what I said in the video, it's just a matter of getting the time to do it.

Here's the video link: https://www.youtube.com/watch?v=dkyY9WCGMi0
11
6
Add a comment...

Ian Hickson

Shared publicly  - 
 
One of our goals with Flutter is to make developing with Flutter a pleasant experience for developers. As part of this, I've focused on the debugging experience.

Lots of our classes make ample use of asserts to help catch problems (either bugs in your applications, or, as it sometimes turns out, bugs in the framework itself). The meaning of these asserts aren't always very clear, though. Because of this, we've been changing some of them to exceptions with very elaborate messages.

For example, if you try to use a material widget without a "Material" object somewhere in its ancestor chain, instead of just asserting that there's no Material ancestor, we'll now output a paragraph-long message that explains the problem, gives you guidance for how to fix it, and tries to explain exactly where in the application the problem is.

I'm especially happy with the very detailed messages you get now when something is wrong with a BoxConstraints object or when a RenderBox fails to fit in its BoxConstraints. It now shows a message that really goes out of its way to track down the cause of the problem. I hope it helps someone!

You can find some more examples of this by searching for "FlutterError" in the repository. By the way, if you run into any asserts that aren't clear, please don't hesitate to file an issue. It'll help us prioritise which asserts to expand on next.

Another thing I've been working on recently to help the debugging experience is adding toString() implementations to pretty much every object in the framework. I want to make Flutter app developers feel like they can dump any random object from the framework to the console and fully expect to see useful information out, whether it's a core primitive like Point or Rect, or a high-level object like RenderTable.

For example, if you print an Animation object, you'll get output like:

AnimationController(▶ 0.000) ➩ Interval(0.0⋯0.9) ➩ Cubic(0.25, 0.10, 0.25, 1.00)

This shows that this is an animation object that's rooted at an AnimationController that is currently playing (▶) and whose value is currently 0.000 (it was probably just created), that the value then goes through (➩) an Interval class, and that the output of that goes through (➩) a curve, in this case a Cubic. The arguments for the Interval and the Cubic are also shown in parentheses (a lot of our toStrings use this style).

Again, as with the asserts, if you run into any objects that don't print something useful when you dump them, file a bug! I get a strange satisfaction from implementing these toString functions. Don't ask.

Of course, these debugging aids are of no use if the documentation is terrible. We did some small hackathons at the office recently to see what the "out of the box" experience is for people who've never used Flutter, and the biggest thing we learnt was that we really need to flesh out our online docs.

So, we've started really aggressively adding dartdocs to everything in the framework. Every time we commit any code, it updates our docs at http://docs.flutter.io/. We are also tracking how many members (methods, getters, setters, classes, typedefs, you name it) we have yet to document.

Right now we're at 2068 undocumented members, but that's down substantially from a few months ago.

We're trying really hard to make the documentation useful. For example, we have a rule that if the documentation is something that someone could have written if they knew nothing about Flutter except the names of the class and method in question, then it's not allowed to go in. Sometimes that makes it really hard to write the documentation, but I think it will pay off in the end!

At the moment we're adding to the documentation more or less at random (actually, more or less in the order that the analyzer lists the members that are lacking docs), so let me know in the comments if there are specific things that you'd like to see documented sooner rather than later, and I'll make sure those are done next!

16
Add a comment...

Ian Hickson

Shared publicly  - 
 
For the last few weeks I've been working on our accessibility story. I've been focusing on the code in the rendering library that walks the internal layout data structures and builds a tree that describes the application's semantics. To debug it, I've built a fake accessibility tool that then consumes this data and draws the entire description of the application over the top of the application using the foregroundPainter of a CustomPaint widget which wraps the application Widget tree.

This was working great, but I was cheating a bit for the interactivity: while all the application state was being drawn using the custom painter, the actual touch input logic was still being done against the "real" application beneath.

So the next step was to wrap the whole application in a GestureDetector with an IgnorePointer widget inside it. The GestureDetector would get all the touches, which it could then pass to the fake accessibility tool, which would then use the API to the accessibility logic in the rendering library to report the touch, thus completing the circle. The IgnorePointer widget would prevent the touches from reaching the "real" nodes beneath.

Sounds great!

But when I launched it, the screen was blank. After studying it for a minute, I noticed that actually the screen had been replaced by a single, screen-sized button: the GestureDetector I'd just created. My fake accessibily tool was essentially displaying the accessibility tree for itself. Oops.
8
2
Add a comment...