An attacker rarely arrives as root. They arrive as www-data through a web application flaw, or as a low-privilege user with reused credentials. Everything that makes a compromise expensive, persistence that survives reboots, log tampering, access to credentials for other systems, usually requires escalating from that starting point.
Which makes escalation the most valuable thing to detect. It is a narrow, noisy step in an otherwise quiet intrusion.
Key Takeaways
- Escalation is a chokepoint. Detecting it catches the intrusion before the expensive part.
- The common paths leave artefacts: SUID binaries, sudoers changes, new users, added SSH keys, capability grants, writable units and cron entries.
- Almost all of these locations are static on a healthy server, which makes them ideal for file integrity monitoring, low volume, high signal.
- Successful escalation via a legitimate mechanism looks like normal administration, so knowing your own baseline is what makes it visible.
- Once root is achieved, your logs and tools stop being trustworthy. Detection must be off-host to be reliable.
Table of Contents
- How escalation actually happens
- The artefacts worth watching
- Detecting the attempt, not just the result
- Why off-host detection matters
- Reducing the surface first
How escalation actually happens
In practice, on real servers, it is usually one of a small number of things. Exotic kernel exploits get the attention; misconfiguration does the work.
Overly permissive sudo rules. A user allowed to run a single command as root, where that command can spawn a shell or write arbitrary files. sudo vim, sudo find, sudo tar, sudo systemctl, each of these grants full root to anyone who knows the technique. The rule looked narrow when it was written.
SUID binaries. A binary owned by root with the setuid bit runs as root regardless of who invokes it. Some legitimately need this. Anything unexpected in that list is a serious finding.
Writable service definitions. A systemd unit, timer, or cron entry that a low-privilege user can modify, executing as root on the next trigger. This is quiet and reliable, which is why it is popular.
Credentials on disk. Database passwords in a world-readable config file, cloud provider keys in an environment file, a private key with wrong permissions. Often the escalation is not technical at all, it is reading a file that should not have been readable.
Kernel and service vulnerabilities. Real, and periodically severe, but less common in incidents than the four above. The defence is patching known CVEs against your actual installed packages rather than reading advisories in general, which is the subject of how to track CVEs for the packages installed on your Linux servers.
The artefacts worth watching
The useful property of all of these locations is that they are static on a normal production server. Nothing writes to them during ordinary operation, so any change is worth attention. That is a much better signal-to-noise ratio than watching application directories.
Sudo configuration:
cat /etc/sudoers
ls -la /etc/sudoers.d/
cat /etc/sudoers.d/* 2>/dev/null
A new file dropped into /etc/sudoers.d/ is a common escalation and persistence step, and it is easy to miss because the main sudoers file is unchanged.
SUID and SGID binaries:
find / -xdev \( -perm -4000 -o -perm -2000 \) -type f -exec ls -la {} \; 2>/dev/null
Capture this on a freshly provisioned server and keep it. Diffing against that baseline is far more useful than reading the list cold, because deciding whether a given SUID binary is legitimate requires knowing what shipped with the system.
File capabilities, which are frequently overlooked because they do not appear in a SUID search:
getcap -r / 2>/dev/null
A capability like cap_setuid on an interpreter is equivalent to a root shell and looks like nothing in a permissions listing.
Users, groups, and authentication:
awk -F: '$3 == 0 {print}' /etc/passwd # any UID 0 account other than root
awk -F: '$2 == "" {print $1}' /etc/shadow # empty passwords
getent group sudo wheel adm 2>/dev/null
tail -30 /etc/passwd
A second UID 0 account is unambiguous. Additions to sudo or wheel are worth verifying against your own change record.
SSH keys, which are the most common persistence mechanism after cron:
for d in /root /home/*; do
[ -f "$d/.ssh/authorized_keys" ] && echo "== $d" && cat "$d/.ssh/authorized_keys"
done
ls -la /etc/ssh/sshd_config.d/ 2>/dev/null
An added key gives durable access that survives password rotation entirely, which is why rotating passwords after an incident without checking authorized_keys accomplishes very little.
Scheduled execution and units:
ls -la /etc/cron.d/ /etc/cron.*/ /var/spool/cron/crontabs/ 2>/dev/null
systemctl list-timers --all --no-pager
find /etc/systemd/system /usr/lib/systemd/system -newermt "-7 days" -type f 2>/dev/null
That last command, units modified in the past week, is a fast, high-yield check that most people never run.
Detecting the attempt, not just the result
Everything above finds escalation after it succeeded. Catching the attempt is better, and the signals are noisier but real.
Sudo failures and unusual sudo usage:
grep -E "sudo:.*(COMMAND|authentication failure|NOT in sudoers)" /var/log/auth.log | tail -30
journalctl _COMM=sudo --since "24 hours ago" --no-pager | tail -30
An entry showing www-data invoking sudo is worth immediate attention, because a web server process has no business escalating. Repeated NOT in sudoers entries mean something is probing what it can run.
The general shape of the signal is a process doing something outside its role: a web server user spawning a shell, a service account reading /etc/shadow, an interpreter launched from a writable temporary directory. None of these are individually conclusive; all of them are unusual enough to warrant looking.
This is where per-server process baselining is worth more than any static rule. The set of processes a given server normally runs is stable and knowable, and a new executable appearing in that set, particularly one running from /tmp, /dev/shm, or a home directory, is a strong signal that requires no rule about specific binary names. It is also how mining payloads get caught, as in how to detect a cryptominer on a Linux server.
Why off-host detection matters
Here is the constraint that shapes everything about detecting escalation: the moment escalation succeeds, the evidence lives on a machine the attacker controls.
Root can edit /var/log/auth.log. Root can replace ps, ls, and find with versions that hide specific processes and files. Root can stop your monitoring agent. Root can preload a library into every process on the system through /etc/ld.so.preload, which makes the output of your investigation tools fiction.
Three implications follow, and they are the difference between detection that works and detection that feels like it works:
Ship telemetry off the host immediately. A log entry that has already left the server cannot be edited retroactively. Local-only logging is only as trustworthy as the machine's integrity, which is precisely what is in question.
Treat agent silence as a signal. If an agent stops reporting, that is either a dead server or a deliberate action. Both need investigating, and neither should look identical to "everything is fine." Tamper resistance and offline detection are not optional extras here; they are the part that makes the rest meaningful.
File integrity monitoring should alert off-host too. A FIM alert written to a local log that root then edits has accomplished nothing. The paths listed above are exactly what to watch, and the practicalities of keeping that output readable are in what file integrity monitoring actually catches on Linux.
Reducing the surface first
Detection is the second job. The first is having less to detect.
Audit sudo rules for commands that can spawn shells or write arbitrary files, and replace broad grants with specific ones. Remove the setuid bit from binaries that do not need it. Ensure no low-privilege user can write to a systemd unit, timer, cron entry, or any script those invoke. Fix permissions on configuration files containing credentials. Keep packages current against known vulnerabilities.
Most of this is on a practical Linux server hardening checklist for small teams, and the SSH-specific portion, which matters because SSH is how most low-privilege access arrives in the first place, is in SSH hardening checklist for production Linux servers. If you are working through this because you suspect something has already happened, how to tell if your Linux server has been compromised is the broader sweep.
SecAI covers this pattern with file integrity monitoring on the sudoers, SSH, cron, and systemd paths above, per-server process and listener baselining that flags new executables and unexpected privilege behaviour, and agent tamper-resistance with offline detection so silence is treated as a finding rather than as health. Telemetry leaves the host as it is generated, so a root-level attacker cannot retroactively edit what was already reported, alongside the rest of the detection stack.
Where escalation detection fits among the other controls worth having is covered in the guide to autonomous Linux protection.
Frequently Asked Questions
What is the fastest single check for privilege escalation?
Look for UID 0 accounts other than root, then authorized_keys for every user, then files in /etc/sudoers.d/. Those three cover the most common persistence-with-privilege patterns and take under a minute.
Can escalation happen without any file changes? An in-memory kernel exploit can grant root without touching disk, but attackers almost always then establish persistence, which does touch disk. Purely memory-resident intrusions exist and are rarer than the alternative.
Why does a web server user running sudo matter so much? Because there is no legitimate reason for it. Application code should never need to escalate. A single occurrence is worth investigating properly rather than dismissing.
Is disabling sudo entirely a good idea? No. It pushes people towards logging in as root directly, which is worse, you lose the per-user attribution that makes an audit trail useful. Keep sudo, keep the rules narrow and specific.
If I find escalation, is cleaning up enough? Rarely. Once something has run as root, proving the machine is clean is harder than rebuilding it. Rebuild from a known-good image, restore data selectively, and rotate every credential the server had access to.