L Android release has been fantastic! Except, that it blows up my current build setup.
On both my laptop and build box, I use a ~30gb ramdisk as $OUT to save wear and tear on my SSD drives. This also doesn't hurt compile times and keeps many "stale dependency issues" at bay when rebuilding often.
I originally set this up by adding 1 line to /etc/fstab:
tmpfs /out tmpfs nodev,nosuid,size=30000M 0 0
And then exporting this in .bashrc:
However, when building L Android for Arm64, the size of the $OUT directory has nearly doubled! Alas, my poor "old" hardware now seems inadequate for compiling to RAM ... or is it?
Enter: zRAM block device driver: http://en.wikipedia.org/wiki/Zram
zRAM is used as a compressed memory block device. Normally, enabled as SWAP in Linux to avoid slowness due to hard disk paging, this driver also supports actual filesystems where they are compressed in memory.
Questions: Can we effectively replace the previous ramdisk setup with a zram block device to allow for a larger RAM-based $OUT directory? If so how large can one expect to make this device using ~30gb?
Answers: Yes. Using zRAM, I've seen compression rates which effectively double the size of actual RAM used. Right now, I'm using a max of about 51gb (~27gb actual memory used). You could go larger if needed.
For reference, my system is running Ubuntu 14.10 and this requires a 3.17+ Linux kernel (more on why later)
Step 1: (if present) comment out the fstab line which sets up the ramfs device. We'll be using a zram block device instead now.
Step 2: Ensure that you have the following export in your .bashrc file:
Step 3: Add the following lines to your /etc/rc.local file, making sure it lands above the "exit 0" line:
echo 4 > /sys/block/zram0/max_comp_streams
echo lz4 > /sys/block/zram0/comp_algorithm
echo 50G > /sys/block/zram0/disksize
mkfs.ext4 -O dir_nlink,extent,extra_isize,flex_bg,^has_journal,uninit_bg -m0 -b 4096 -L "zram0" /dev/zram0
mount -o barrier=0,commit=240,noatime,discard /dev/zram0 /out
chmod 777 /out
Let's break down each of these lines:
1. Insert the zram kernel module and create a new block device (zram0)
2. Enable up to 4 compression streams if needed
3. Set lz4 compression algorithm for the block device (generally considered the best compression for performance)
4. Set 50gb disksize limit (compressed memory, so actual memory use is about 1/2 of this in most cases). This setting should be replaced with a new "mem_limit" sysfs entry added in later kernels: https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/drivers/block/zram?id=9ada9da9573f3460b156b7755c093e30b258eacb
Oddly, I was unable to find this entry in the 3.17.4 kernel I'm using now.
5. Setup ext4 on the zram block device. Drop journaling as it's not needed for this use-case and will help with speed.
6. Mount the block device as /out, notice that this line includes "discard". More discussion on that below.
7. Lastly, I set 777 permissions on /out directory. You don't need to. Here I was being quite lazy while fixing a permission error that Android build was giving me. This should be something less than 777.
Reboot and check "mount" command to see your new compressed ramdisk:
/dev/zram0 on /out type ext4 (rw,noatime,barrier=0,commit=240,discard)
When you run builds in Android, a new sub-directory will be created under /out depending on what your Android folder name is. Example: /out/mydroid-aosp/... To clear out a build from the ramdisk you can execute: rm -rf /out/mydroid-aosp
(Warning: not for those prone to very bad typo's -- don't blame me if you delete the entire contents of your device... you've been warned. No really.. don't delete your stuff on accident, use a GUI or something if you're concerned about this command.)
Let's talk about why kernel 3.17.x is needed and why "discard" is included in mount line.
Previously, the zram driver was intended as a SWAP device and all was well. Later, once the filesystem use-case came into play, an issue was found where memory wasn't being released during the discard operation. See kernel commit:https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/drivers/block/zram?id=015254daf1753003c19c46b90ee85a963260d270
This was added in the 3.17 kernel to fix this issue. (hence the kernel version requirement)
Without this commit, memory will never be freed up by the block device. Ending in terrible horrible unsufferable system lag once your box enters an endless loop of disk swapping.
The reason "discard" is added to the mount line is for convenience. As files are deleted, memory gets freed up. It is not required, and will add a few more cycles to your CPU.
Instead, you could manually run "fstrim /out" to trigger this memory free after clearing out files. It's up to you.
Perhaps this will help someone else out there, in the same situation.