Rain WikiWindows · Linux · Android

OOM killer

ConceptAll Linux distributions

The OOM (out-of-memory) killer is the part of the Linux kernel that ends a process when the system runs out of memory and cannot free any. It chooses the process by its OOM score, usually the one using the most memory. The kernel log records every OOM kill.

1How the victim is chosen

Each process has a score in /proc/<pid>/oom_score that grows with its memory use. /proc/<pid>/oom_score_adj, from -1000 to 1000, adjusts it; -1000 means the process is never chosen.

2Finding OOM kills

The kernel writes 'Out of memory: Killed process' to its log. journalctl -k | grep -i oom or dmesg | grep -i oom shows these messages, with the process name and its memory use.

3Prevention

Add memory or swap, limit memory-hungry services, or set limits per service with systemd options such as MemoryMax=. Some distributions also run systemd-oomd, which acts earlier based on memory pressure.

How to investigate an OOM kill

  1. Search the kernel log: journalctl -k | grep -i -A5 'out of memory'
  2. Note the killed process and its memory size.
  3. Check current memory and swap: free -h
  4. Find the biggest processes: ps aux --sort=-rss | head
  5. Add swap or limit the service, then watch whether it happens again.

Cautions

  • Setting oom_score_adj to -1000 for many processes can leave the kernel with no safe choice.
  • Turning off memory overcommit protection without understanding it can cause failures elsewhere.

Sources

  1. proc(5) manual page: oom_score and oom_score_adj man7.org
  2. Out Of Memory Management (kernel.org) kernel.org

Note: RainServer offers monitoring and safe cleanup for Linux servers; it shows what will change and asks first.

Last reviewed: