Fix Linux EMFILE 'Too many open files' (ulimit & systemd)

EMFILE means a process hit its open file descriptor limit. Raise the nofile limit with ulimit, limits.conf, or systemd, depending on how the process starts.

Quick answer

Raise the open file limit (RLIMIT_NOFILE) for the user or service that's hitting EMFILE. Check with ulimit -n, then set it in /etc/security/limits.conf, in the systemd unit with LimitNOFILE=, or per-process with prlimit.

What's actually happening

On Linux, every open file, socket, pipe, and eventfd consumes a file descriptor. Every process has two limits: a soft limit (the one currently enforced) and a hard limit (the ceiling the soft limit can be raised to without root). When a process tries to open the 1025th descriptor and its soft limit is 1024, the kernel returns EMFILE — the error string is Too many open files. You'll often see it as a secondary symptom: nginx logs worker_connections are not enough, Java throws java.io.IOException: Too many open files during accept(), or a Postgres backend dies mid-query. The real cause is almost never that the process genuinely needs more files than you have. It's that your application has a leak (sockets not closed, FILE* handles from fopen() never fclose()'d), or the default limit is stuck at 1024 because whoever launched the process didn't push it up.

Fix it

Confirm the limit and the count.
```
cat /proc/$(pgrep -f myapp | head -1)/limits | grep 'open files'
ls /proc/$(pgrep -f myapp | head -1)/fd | wc -l
```
If the count is right at the soft limit, EMFILE is the answer. If it's much lower, you have a leak and raising the limit only delays the crash.
Raise the limit for an interactive shell (temporary).
```
ulimit -n 65535
```
This only affects the current shell and its children. It won't survive logout, and it can't exceed the hard limit. Check the hard cap with ulimit -Hn.
Make it persistent for a user. Edit /etc/security/limits.conf (or a file in /etc/security/limits.d/):
```
myuser  soft  nofile  65535
myuser  hard  nofile  65535
```
On RHEL 7+ and derivatives, systemd ignores limits.conf for services started by systemd, so this only helps login shells via PAM. That trips up a lot of people.
For systemd services, set it in the unit. Drop-in override is cleanest:
```
systemctl edit nginx
# then add:
[Service]
LimitNOFILE=65535
```
Then systemctl daemon-reload && systemctl restart nginx. The reason this works where limits.conf doesn't: systemd calls setrlimit() directly on the spawned process, bypassing PAM entirely.
Verify after restart.
```
systemctl show nginx -p LimitNOFILE
cat /proc/$(pgrep -f nginx | head -1)/limits | grep 'open files'
```
If the soft limit still reads 1024, your override didn't apply — usually because the drop-in went to the wrong path or you forgot daemon-reload.

If that doesn't work

Check the kernel-wide ceiling. /proc/sys/fs/nr_open caps how high any single process can set its hard limit (default 1048576 on modern kernels). /proc/sys/fs/file-max caps the total across the system. On busy boxes with thousands of sockets, you may need sysctl -w fs.file-max=2097152 and a line in /etc/sysctl.d/.
For a running process you can't restart, use prlimit.
```
prlimit --pid 12345 --nofile=65535:65535
```
Raising the soft limit only works if the current hard limit allows it. Raising the hard limit requires CAP_SYS_RESOURCE, so run as root.
Find the leak. If lsof -p <pid> | wc -l climbs steadily over minutes, you're leaking descriptors, not underprovisioned. Look for socket handles in CLOSE_WAIT (peer closed, your app didn't), or fopen() calls with no matching fclose(). Raising the limit doesn't fix this — it just moves the failure later.

Prevention

Set LimitNOFILE explicitly in every systemd unit you write. Don't rely on the 1024 default, which is a relic from the 90s when it was generous. For file servers, proxies, and anything that accepts TCP connections, start at 65535 and monitor. Also grep your codebase for fopen, open(, and socket( and make sure every call site has a corresponding close path on the error branches too — that's where the leaks hide.