Profile

Cover photo
Elliott Hughes
Works at Google
Lives in California, USA
515 followers|119,104 views
AboutPostsPhotosVideos

Stream

Elliott Hughes

Shared publicly  - 
 
ndk-gdb or ndk-gdb-py?

I'd like to remove ndk-gdb (the shell script) in favor of ndk-gdb-py (the trivial shell script/batch file that runs the python version of ndk-gdb). If you have to use ndk-gdb for some reason, leave a comment.

#androiddev
10 votes  -  votes visible to Public
ndk-gdb
0%
ndk-gdb-py
100%
1
Add a comment...

Elliott Hughes

Shared publicly  - 
 
Q: Is clang the default compiler in M?

A: No. The default compiler is still GCC, and you can't successfully build the whole M tree with clang yet. AOSP master, though, can now be built with clang. And we have continuous clang builds to ensure that it stays that way.

(Note that "built with clang" doesn't mean "built without GCC"... There are projects using "LOCAL_CLANG := false" to force GCC even in an otherwise clang-built build. On the flip side, if you grep the tree for "LOCAL_CLANG := true" you can see that there are now some projects that are always built with clang.)

Other than source compatibility and performance problems (plus one instance of compile-time performance) the main clang stumbling block right now is the inability to implement _FORTIFY_SOURCE for clang. But we -- and the relevant SoC vendors -- are looking into these issues.

(Kernels are still built with a separate GCC 4.8 toolchain, different from the platform GCC toolchain. That's a problem for another day.)
31
5
Steve Jones (Trevor Drake)'s profile photoZoran Jovanovic's profile photoKarim Yaghmour's profile photoAnmar Oueja's profile photo
2 comments
 
We're definitely interested in hearing feedback from NDK users though. I didn't mention it, but at some point we'd prefer to only offer clang in the NDK too, and it's a lot harder for us to judge the point at which clang is good enough for/actively preferred by NDK users. (There are Google products that use the NDK, but that's a drop in the ocean of all NDK users.)
Add a comment...

Elliott Hughes

Shared publicly  - 
 
Android M command-line improvements.

toybox replaces much of toolbox.

The tools from http://landley.net/toybox/ replace most of the old toolbox tools in M. SELinux support and a few other things that were better in toolbox have been pushed upstream, and the following tools have been switched over (or added if they weren’t previously in toolbox):

    acpi
    basename blockdev bzcat
    cal cat chcon chgrp chmod chown chroot cksum clear comm cmp cp cpio cut
    date dirname dmesg dos2unix
    echo env expand expr
    fallocate false find free
    getenforce getprop groups
    head hostname hwclock
    id ifconfig inotifyd insmod
    kill killall
    load_policy ln logname losetup lsmod lsusb
    md5sum mkdir mknod mkswap mktemp modinfo more mountpoint mv
    netstat nice nl nohup
    od
    paste patch pgrep pidof pkill pmap printenv printf pwd
    readlink realpath restorecon rm rmdir rmmod route runcon
    sed seq setenforce setprop setsid sha1sum sleep sort split stat strings swapoff swapon sync sysctl
    tac tail tar taskset tee time timeout touch tr true truncate
    umount uname uniq unix2dos usleep
    vmstat
    wc which whoami
    xargs
    yes

The grep family remains the NetBSD grep, and dd and du are still from NetBSD.

For a variety of reasons, the following are still using the toolbox implementations: df, getevent, iftop, ioctl, ionice, log, ls, lsof, mount, nandread, newfs_msdos, ps, prlimit, renice, sendevent, start, stop, top, uptime, watchprops.

Other tools like mksh and strace have been upgraded to the current upstream versions.

#androiddev   #io15   
27
9
Karim Yaghmour's profile photoShaka Huang's profile photoTed Chien's profile photoKivava Chang's profile photo
4 comments
 
+Elliott Hughes I was mostly just curious, for the sake of completeness.  My initial implementation of transparent file compression used xattrs.  I've since switched to using an ext4 inode flag, so I'm not using them anymore.
Add a comment...

Elliott Hughes

Shared publicly  - 
 
Android M init improvements.

Android's init gains a long-requested feature in M: there's a new 'exec' command. If you're an OEM and have been using your own implementation, or you've been hacking tons of random crap into init itself (just like Google has for years), please give this a go and let us know whether it works for you. Documentation in system/core/init/readme.txt.

Another interesting change for OEMs (especially if you're seeing "init: could not import file '/init.unknown.rc' from '/init.rc'" in dmesg) is that ro.hardware is now expected to come from device tree under firmware/android/hardware. (You can just set hardware=tardis on your kernel command line, but device tree is the way forward.)

Boot time is becoming a hot topic, especially because no one like to wait while their TV (or car) boots. Bootcharting has been fixed so it works again, and is now always available; it's no longer compiled out. (Bootcharting documentation is in system/core/init/readme.txt too.) We log more to dmesg now explaining where the time is going, mainly to prove that slow boot times aren't init's fault, but also to draw attention to slow areas. One thing we found was that some SoC vendors felt that as long as they came in under the ueventd coldboot timeout of 5s, everything was fine. But taking 5s is insane, and some SoC vendors were trying to increase it rather than get their act together! In our tree the timeout is now 1s. If relatively weak hardware from a few years ago can do this in 0.2s, so you can you. If you're an OEM, you might want to check that your SoC vendor isn't being lazy. Check for a new "ueventd: Coldboot took 0.21s" line in dmesg.

#androiddev   #io15   
45
13
george oloo's profile photoAlexander Lent's profile photo
Add a comment...

Elliott Hughes

Shared publicly  - 
 
A nice success for the new "obsolete any bug that's gone untriaged for a year and let people refile if it's still a problem" strategy: https://code.google.com/p/android/issues/detail?id=162488
Android Open Source Project - Issue Tracker
3
Eric Wedel's profile photoElliott Hughes's profile photoBenoît `BoD` Lubek's profile photo
6 comments
 
I kind of understand what you're saying, but you had to know that doing this without any explanation would upset people... Right?  Personally I felt insulted.  I took this as a "we don't care" message.  I really don't see why I should open new issues after this incident.

A blog post (or a post on the Android Developers community, or on the Android Developers mailing list) explaining why you felt you had to do it, and encouraging  people to reopen issues that are still relevant would have been the minimum expected curtesy.

I haven't seen this discussed anywhere (I don't use reddit) before this post and I'm lucky I follow you because otherwise I would still be in the dark about this.
Add a comment...

Elliott Hughes

Shared publicly  - 
The New Horizons spacecraft sent by NASA to study Pluto uses a MIPS-based Mongoose-V chip clocked at a whopping 12 MHz.
1
Thierry Lombry's profile photo
 
You will see more renderings on my dedicated page at http://www.astrosurf.com/luxorion/alienworlds-renderings.htm
Add a comment...
Have him in circles
515 people
SMITH MICHELLE's profile photo
Richard Fearn's profile photo
andreas sunarko's profile photo
Daniel Oskarsson's profile photo
Viacheslav Antonenko's profile photo
Michael Bailey's profile photo
Chet Kener's profile photo
Fernando Cejas's profile photo
Michael Leibowitz's profile photo

Elliott Hughes

Shared publicly  - 
 
If you're a Windows adb.exe user who's been having trouble with the "Allow USB debugging?" dialog showing up all the time with r23,  you should find that this is fixed by r23-rc3 (https://dl.google.com/android/repository/platform-tools_r23_rc3-windows.zip), released earlier this week.

Sorry for the inconvenience. (This bug "only" affected Windows, so I didn't notice when I introduced it.)
20
3
Ali Muzaffar's profile photoKeegan Jacobson's profile photoAman Upadhyaya's profile photoLuka Jerkovic's profile photo
5 comments
 
Lol! Yes, Linux and Mac boxes its not an issue on. Even Nexus devices are not problematic as there is a universal driver that works well with them. Was just wondering if there was a solution for windows that was a little more universal (works on all Android phones not just the nexus phones). 
Add a comment...

Elliott Hughes

Shared publicly  - 
 
Android M C++ improvements.

libc++ replaces stlport for the platform.

stlport served us well for many years, but it hasn’t seen any updates in a long time and has fallen far behind current standards. In M, we've switched to libc++ (http://libcxx.llvm.org/) and are removing stlport. If you’re an OEM building code for the platform, libc++ is available by default. You don’t need to add anything to your makefiles. (We’ve also bumped the compiler’s default C++ standard to C++11.) What you do need to do as an OEM is stop referencing stlport; this shouldn’t require any more work than simply removing any line containing “stlport” from your makefiles.
35
7
Dominik Helleberg's profile photoZoran Jovanovic's profile photoKarim Yaghmour's profile photoFelix Homann's profile photo
2 comments
 
+Ilya Konstantinov That's a long story, so I've answered in a separate post.
Add a comment...

Elliott Hughes

Shared publicly  - 
 
Android M fastboot improvements.

Images no longer decompressed into RAM.

In M (and since SDK Tools 24.2.0), fastboot no longer decompresses into RAM. If you’ve had trouble in the past with the apparently infamous (but no one told us until we fixed this for unrelated reasons) “update package missing system.img”, that should now be a thing of the past. (For Mac OS and Linux users at least; from the comments there's another bug that affects Windows users, not yet fixed.)

  target reported max download size of 536870912 bytes
  archive does not contain 'boot.sig'
  archive does not contain 'recovery.sig'
  fastboot(43183,0xa04921d4) malloc: * mach_vm_map(size=1981542400) failed (error code=3)
  * error: can't allocate region
  * set a breakpoint in malloc_error_break to debug
  failed to allocate 1979559444 bytes
  error: update package missing system.img


fastboot reboot bootloader.

You also no longer need to remember whether it’s “fastboot reboot-bootloader” or “fastboot reboot bootloader”. Both work now, just like in adb.

#androiddev   #io15   
53
7
Dave Smith's profile photoSteve Jones (Trevor Drake)'s profile photoGabriel Gattringer (GGab)'s profile photoSudhir Khanger's profile photo
44 comments
 
+Fedor von Bock my Nexus 9 running the developer preview shows the same warning 
Add a comment...

Elliott Hughes

Shared publicly  - 
 
If you wished the Google I/O keynote had more coverage of dynamic linker changes, here are the big changes in M:

Fixes to library search order.
We made fixes to library search order when resolving symbols, to support ASAN. In Android L-MR1, load order switched from depth-first to breadth-first to fix dlsym(3). In Android releases < M, the default search order was to try the main executable, LD_PRELOAD libraries, the library itself, and its DT_NEEDED libraries in that order. In >= M, for any given library, the dynamic linker divides other libraries into the global group and the local group. The global group is shared by all libraries and contains the main executable, LD_PRELOAD libraries, and any library with the DF_1_GLOBAL flag set (by passing “-z global” to ld(1)). The local group is the breadth-first transitive closure of the library and its DT_NEEDED libraries. The M dynamic linker searches the global group followed by the local group. This allows ASAN, for example, to ensure that it can intercept any symbol.


RTLD_LOCAL.
The dlopen(3) RTLD_LOCAL flag was ignored <= L but is implemented correctly >= M. Note that RTLD_LOCAL is the default, so even calls to dlopen(3) that didn’t explicitly use RTLD_LOCAL will be affected (unless they explicitly used RTLD_GLOBAL). With RTLD_LOCAL, symbols will not be made available to libraries loaded by later calls to dlopen(3) (as opposed to being referenced by DT_NEEDED entries).


GNU hashes.
The GNU hash style available with --hash-style=gnu allows faster symbol lookup and is now supported by the M dynamic linker. (Use --hash-style=both if you want to build code that uses this feature >= Android M but still works on older releases.)


Correct soname/path handling.
The dynamic linker now understands the difference between a library’s soname and its path  (public bug https://code.google.com/p/android/issues/detail?id=6670). Android M is the first release where search by soname is implemented. Earlier releases would assume that the basename of the library was the soname, and used that to search for already-loaded libraries. For example, dlopen(“/this/directory/does/not/exist/libc.so”, RTLD_NOW) would find /system/lib/libc.so because it’s already loaded. This also meant that it was impossible to have two libraries “dir1/libx.so” and “dir2/libx.so” --- the dynamic linker couldn’t tell the difference and would always use whichever was loaded first, even if you explicitly tried to load both. This also applied to DT_NEEDED entries. Some apps have bad DT_NEEDED entries (usually absolute paths on the build machine’s file system) that used to work because we ignored everything but the basename. These apps will fail to load on M.



Symbol versioning.
Symbol versioning allows libraries to provide better backwards compatibility. For example, if a library author knowingly changes the behavior of a function, they can provide two versions in the same library so that old code gets the old version and new code gets the new version. This is now supported in M.



Opening shared libraries from an APK.
In M, it’s possible to open a .so directly from your APK. Just use System.loadLibrary(“foo”) exactly as normal but set android:extractNativeLibs="false" in your AndroidManifest.xml. In older releases, the .so files were extracted from the .apk file at install time. This meant that they took up space in your .apk and again in your installation directory (and this was counted against you and reported to the user as space taken up by your app). Any .so file that you want to load directly from your .apk must be page aligned (on a 4096-byte boundary) and stored uncompressed. (In M, the zipalign tool will take care of alignment, and Android Studio will support this workflow.)

#androiddev   #io15   #ndk  
31
12
george oloo's profile photoTatsuo Nagamatsu's profile photoEric Hung's profile photoFabrice Di Meglio's profile photo
 
Good news. I guess with this, Android dynamic library handling is finally (at least) as unbroken as it is on modern Linux distros.
Add a comment...

Elliott Hughes

Shared publicly  - 
 
I've definitely assumed there are no security implications to wild memcpys before because "obviously" they'll hit an unmapped or protected page.

http://googleprojectzero.blogspot.com/2015/03/taming-wild-copy-parallel-thread.html

In their exploit, rewriting the munmap(2) relocation to point to system(3) is a handy shortcut, but useless on Android since Jelly Bean --- RELRO (Read-Only RELocations) prevent that kind of thing. (The platform is built with full RELRO, and if you have native code in your apps, you should build with it too.)
Posted by Chris Evans, Winner of the occasional race Back in 2002, a very interesting vulnerability was found and fixed in the Apache web server. Relating to a bug in chunked encoding handing, the vulnerability caused a me...
7
Nick Kralevich's profile photo
 
If you're using the NDK to compile, you get full RELRO support by default (https://android.googlesource.com/platform/ndk/+/f74c373729bcd1519debe03cda90ef3fd3366848)
Add a comment...

Elliott Hughes

Shared publicly  - 
 
I thought https://blog.whitehatsec.com/north-koreas-naenara-web-browser-its-weirder-than-we-thought/ was mildly interesting, but "it’s in UTF-8 charcode, and not something that you might expect like BIG5 or ISO-2022-KR or SHIFT_JIS or something" was a weird thing to say. Big5 is for Traditional Chinese and Shift_JIS for Japanese. They can't even encode Korean, as you can see from a quick test:

$ cat y.java
import java.nio.charset.*;
public class y {
 public static void main(String[] args) throws Exception {
  System.err.println(Charset.forName("Big5").newEncoder().canEncode("잘 먹었어요"));
  System.err.println(Charset.forName("Shift_JIS").newEncoder().canEncode("잘 먹었어요"));
  System.err.println(Charset.forName("UTF-8").newEncoder().canEncode("잘 먹었어요"));
 }
}
$ javac y.java && java y
false
false
true
$
Naenara is a North Korean web browser built into Red Star OS. This is a review of Naenara which is build on Firefox.
3
Add a comment...
People
Have him in circles
515 people
SMITH MICHELLE's profile photo
Richard Fearn's profile photo
andreas sunarko's profile photo
Daniel Oskarsson's profile photo
Viacheslav Antonenko's profile photo
Michael Bailey's profile photo
Chet Kener's profile photo
Fernando Cejas's profile photo
Michael Leibowitz's profile photo
Places
Map of the places this user has livedMap of the places this user has livedMap of the places this user has lived
Currently
California, USA
Previously
England - Switzerland - Germany
Work
Occupation
Software Engineer
Employment
  • Google
    Software Engineer, present
  • BlueArc
  • GeneData
Basic Information
Gender
Male
Relationship
Married