Ubuntu performance fine tuning
Applications have different characteristics. Parts of a binary typically can be swapped out to leave more memory for data. Inmemory databases, like memcached should never swap out.
PostgreSQL, MySQL, MariaDB heavily make use of memory buffers, which become pretty useless, when your database is much larger than available memory. Double buffering (database own buffers) and Linux kernel buffering accesses to database file itself - is pretty useless.
Every VM can run identical binaries. Linux supports Kernel Samepage Merging. So, 20 identical Apache binaries share same code. In that case, swapping dramatically decreases performance of 20 virtual machines (or containers) at the same time.
Mobiles, looking up mobile web pages over bandwidth limiting carriers, causes TCP/IP buffer size to dramatically increase. Setting TCP/IP connection timeout, lowering TCP/IP buffer size (per connection), helps.
High loads typically cause SSH login delays. Administrative access feels unresponsive. Rising priority for certain ports, short packages solves this Linux specific problem (FreeBSD hasn't!).
Game servers typically require kernel realtime patches for reducing latency.
Running hundreds of fully virtualized machines, e.g. KVM on many cores finaly require setting different parameters, depending on load characteristics. Same for running thousands of Linux containers. Depending on VM and container namespace, Linux allows to set parameters individually for each VM/container.
Some CPU's have direct connection to network cards, others only indirect. Pinning network I/O intense processes to these hardware preferred CPUs (and interrupts) can dramatically increase performance.
SSD drives typically show very low latency and high IOPS. Optimizing for that is ok. When most IO are read accesses. When only a single process keeps on permanent writing to file system (longer than 20 seconds), performance of SSD untypically collapses, leading to performance far below magnetic drive. Reason: SSD RAM write buffers saturated. Same for SSD drives, that tend to "full". Leaving more than ⅓ free on FLASH drive helps.
... and, and, and ...