System Configuration

Helps in preventing SYN flood attack on the system. A value of 0 will disable it.

For more details, please see SYN Cookies and WSAECONNRESET Errors.

A value of 1 enables timestamps for TCP.

Many distributions turn timestamps off to reduce performance spikes related to timestamp generation.

From Red Hat’s white paper on network performance tuning (see Related Links):

In the original TCP definition, the TCP segment header only contains an 8-bit value for the TCP Window Size, which is insufficient for the link speeds and memory capabilities of modern computing. The TCP Window Scaling extension was introduced to allow a larger TCP Receive Window. This is achieved by adding a scaling value to the TCP options which are added after the TCP header.

While this is already enabled on most modern distributions, it may still be set to 0 in some environments. Setting this value to one enables window scaling.

This is the maximum number of unhandled client connections a socket can handle; most default settings have this as 128, which can cause resets and other connection failures, particularly for clients issuing many commands very quickly (such as automated build systems, among others).

This is critical for servers under high loads, as this value controls incoming traffic between the network connection and actual processing by the TCP stack and onward.

Note: Max value based on older kernels; newer kernels can set this to a higher value, but it will be silently capped at the somaxconn value.

This value sets many half-open connections for which the client has not yet sent an ACK response can be kept in the network queue. More connections than this would be dropped.

Note: This value may still be TOO LOW; Many distributions set this to about 2048 now, and there are arguments for raising this value to 4096 or as high as 3,240,000.

There is a trick to calculate this; set net.ipv4.tcp_syncookies to 1 (enable cookies) and use this command to look for how many cookies your system is sending under high loads:

The 'SYNs to LISTEN' message is an indication that your server could be dropping packets due to this value being too low.

Note: This value is also limited by somaxconn and netdev_max_backlog settings in terms of kernel configurables, and ultimately limited by the maximum number of file handles the OS can handle, as two handles are opened for each network connection. For example:

Here the OS is reporting 8,160 file handles out of over 3 million available.

Two values are set to specify a range of ports available to client/server applications for this host. The default value provides 28,232 ports, but servers with a lot of connections being held open by custom clients, large numbers of pull threads, and high overall loads may exhaust this number. The new range nearly doubles the number of available ports to 55,535 ports.

How many seconds to wait for a final FIN packet before the socket is forcibly closed. A FIN packet is usually sent from server or client to terminate a connection, after establishment of TCP 3-way handshake and successful transfer of data. Set at too high a value and the system can run out of ports, file handles or memory. Too low a value could leak delayed packets.

Note: Some older distributions set this as high as 180 seconds. This may be a value that can be tweaked lower, but not by very much.

The interval between the last data packet sent (simple ACKs are not considered data) and the first keepalive probe to determine if the connection should be held open. Once set, this value is not used again.

Note: This value works closely with the tcp_keepalive_intvl and the tcp_keepalive_probes values (below):

So the adjusted values for those settings substantially reduce the wait until the first keepalive ack from two hours to 15 minutes, shorten the time between acks from 75 to 30 seconds, but increase the number of probes from 9 to 20. This should prevent most completely dead connections from tying up server resources, while providing connections over poorer network conditions a chance to prove that, while mostly dead, they are still partly alive.

The interval between sequential keepalive probes, regardless of what data was exchanged after keepalive probes are sent after the keepalive interval has been reached.

Note: See net.ipv4.tcp_keepalive_time, above.

The number of unacknowledged probes to send before considering the connection dead and notifying the application layer to drop the connection.

Note: See net.ipv4.tcp_keepalive_time, above.

If the software interrupt doesn’t process packets for a long time, it may cause the NIC buffer to overflow and, hence, can cause packet loss. netdev_budget shows the default value of the time period for which software interrupts should run.

Most higher end network cards (10Gbps or higher) need this to be set to 600; currently there is no benefit to increasing this value further.

Sets the maximum size for the read buffer for network connections, in bytes.

Note: This is connected to wmem_max, tcp_rmem, and tcp_wmem values, detailed below.

The chief risk of increasing this value is using too much memory; however, the default value (about 200K) is typically much too small for higher end systems to function well. This changes the maximum value to 16MB in size, which is only used as needed; the starting values for the read and write buffers (set by tcp_rmem and tcp_wmem) prevent this from using too much memory. This can be increased, but there is a point of diminishing returns; only benchmarking the specific server and loads will confirm that the newer values help or do nothing to improve performance.

Sets the maximum size for the write (send) buffer for network connections, in bytes.

Note: See net.core.rmem_max for more details.

Contains three values that represent the minimum, default and maximum size of the TCP socket receive buffer. Values are in 4K pages, so a setting of 1 would be 4096 bytes.

Note: See net.core.rmem_max for more details.

Contains three values that represent the minimum, default and maximum size of the TCP socket write (send) buffer. Values are in 4K pages, so a setting of 1 would be 4096 bytes.

Note: See net.core.rmem_max for more details.

The kernel option that affects the memory allocated to the list maintained by the kernel that contains "extra" packet information like SCM_RIGHTS or IP_TTL. Increasing this option allows the kernel to allocate more memory as needed for more control messages that need to be sent for each socket connected (including IPC sockets/pipes).

The tcp_mem variable defines how the TCP stack should behave when it comes to memory usage.

The first value specified in the tcp_mem controls the lower threshold; below this point, the TCP stack does not attempt to control memory usage by network sockets. The second value tells the kernel at which point to start controlling network socket memory usage. The final value is the maximum amount of memory to be used for networking; if this value is reached, TCP streams and packets start getting dropped until a lower memory usage is achieved. This value includes all TCP sockets currently in use.

By default, TCP saves various connection metrics in the route cache when the connection closes, so that connections established in the near future can use these to set initial conditions. Usually, this increases overall performance, but may sometimes cause performance degradation. If set, TCP will not cache metrics on closing connections.

Enable select acknowledgments (SACKs).

Accept ICMP redirect messages. accept_redirects for the interface will be enabled if:

Note: a per interface setting (where “interface” is the name of your network interface); “all” is a special interface: changes the settings for all interfaces.

0 - No source validation.

1 - Strict mode as defined in RFC3704 Strict Reverse Path Each incoming packet is tested against the FIB and if the interface is not the best reverse path the packet check will fail. By default failed packets are discarded.

2 - Loose mode as defined in RFC3704 Loose Reverse Path Each incoming packet’s source address is also tested against the FIB and if the source address is not reachable via any interface the packet check will fail.

Current recommended practice in RFC3704 is to enable strict mode to prevent IP spoofing from DDoS attacks. If using asymmetric routing or other complicated routing, then loose mode is recommended.

The max value from conf/{all,interface}/rp_filter is used when doing source validation on the {interface}.

Note: Some distributions enable it in startup scripts.

Note: per interface setting (where “interface” is the name of your network interface); “all” is a special interface: changes the settings for all interfaces.

Accept ICMP redirect messages only to gateways listed in the interface’s current gateway list. Even if disabled, RFC1122 redirect rules still apply. Overridden by shared_media. secure_redirects for the interface will be enabled if at least one of conf/{all,interface}/secure_redirects is set to TRUE, it will be disabled otherwise.

Note: per interface setting (where “interface” is the name of your network interface); “all” is a special interface: changes the settings for all interfaces.

Send redirects, if router. send_redirects for the interface will be enabled if at least one of conf/{all,interface}/send_redirects is set to TRUE, it will be disabled otherwise.

Note: per interface setting (where “interface” is the name of your network interface); “all” is a special interface: changes the settings for all interfaces.

Accept packets with SRR option. conf/all/accept_source_route must also be set to TRUE to accept packets with SRR option on the interface.

If set non-zero, then the kernel will ignore all ICMP ECHO and TIMESTAMP requests sent to it via broadcast/multicast.

Forward Packets between interfaces.

While not commonly used, those settings have been changed for customers in the past; currently set aside as there is some question as to their general suitability. Do not use those settings unless you have a solid basis for doing so, and test thoroughly for any undesirable performance or resource usage impacts after changing.

Note: Those settings may be moved to another knowledge base article to avoid article size limitations.

An NMI watchdog is a watchdog which will generate a non maskable interrupt, i.e. the interrupt handler will get executed no matter what the CPU state is.

This is very useful in scenarios where you are getting unexplained system freeze scenarios, as the NMI watchdog interrupt handler will simply kill whatever process happens to be freezing the CPU at the moment. This way, your CPU gets freed up AND you get a detailed stack trace of why your CPU got frozen up in the first place.

The soft lockup detector monitors CPUs for threads that are hogging the CPUs without rescheduling voluntarily, and thus prevent the ‘watchdog/N’ threads from running. The mechanism depends on the CPUs ability to respond to timer interrupts which are needed for the ‘watchdog/N’ threads to be woken up by the watchdog timer function, otherwise the NMI watchdog - if enabled - can detect a hard lockup condition.

If a CPU fails to respond for the number of seconds the value is set to, throw a kernel panic. As a very busy system may see a delay in responding to an interrupt as a possible crash, increasing this value may reduce kernel panics at the risk of failing to detect a "real" panic until 3 minutes have passed.

This value controls the soft lockup detector thread’s behavior when a soft lockup condition is detected as to whether or not to gather further debug information. If enabled, each cpu will be issued an NMI and instructed to capture stack trace.

This feature is only applicable for architectures which support NMI.

This value controls the hard lockup detector behavior when a hard lockup condition is detected as to whether or not to gather further debug information. If enabled, arch-specific all-CPU stack dumping will be initiated.

According to the Linux kernel documentation:

It is a 'magical' key combo you can hit which the kernel will respond to regardless of whatever else it is doing, even if the console is unresponsive.

As this functionality can be a security risk, this is often already off by default.

The default coredump filename is “core”. By setting core_uses_pid to 1, the coredump filename becomes core.PID. If core_pattern does not include “%p” (default does not) and core_uses_pid is set, then .PID will be appended to the filename.

Defines the maximum size in bytes of a single message queue.

Defines the maximum allowable size in bytes of any single message in a message queue. This value must not exceed the size of the queue (msgmnb).

Defines the maximum shared memory segment allowed by the kernel, in bytes. For database workloads, generally a value no larger than 75% of the total memory on the system.

Defines the total number of shared memory pages that can be used on the system at one time. For database workloads this value is set to the result of shmmax divided by the hugepage size.