Keep Alive

I'm so much used to mosh that I'm always surprised by how fast a plain ssh connection runs into a timeout. Or worse into a kind of half-terminated hanging connection with an apparently unresponsive terminal.

Which is weird, since there's already one measure in place that is intended to avoid this situation: TCPKeepAlive, which is enabled by default. To quote the man page of sshd_config:

On the other hand, if TCP keepalives are not sent, sessions may hang indefinitely on the server, leaving "ghost" users and consuming server resources. The default is yes (to send TCP keepalive messages), and the server will notice if the network goes down or the client host crashes. This avoids infinitely hanging sessions.

What irritates users most in this situation is the unresponsive terminal, which seems to no longer accept any commands, and won't close unless the ssh connection is terminated by killing the process. But there's no need to kill, as ssh offers several escape sequences that also take care of this case:

 ~.  - terminate connection (and any multiplexed sessions)
 ~B  - send a BREAK to the remote system
 ~C  - open a command line
 ~R  - Request rekey (SSH protocol 2 only)
 ~^Z - suspend ssh
 ~#  - list forwarded connections
 ~&  - background ssh (when waiting for connections to terminate)
 ~?  - this message
 ~~  - send the escape character by typing it twice
(Note that escapes are only recognized immediately after newline.)

To prevent this thing to ever happen again, two options can be set either on the server or the client.

On the server side, one can set the following options in /etc/ssh/sshd_config:

TCPKeepAlive no (default yes)
ClientAliveInterval 30
ClientAliveCountMax 240

To quote again the man page of sshd_config:

ClientAliveInterval Sets a timeout interval in seconds after which if no data has been received from the client, sshd(8) will send a message through the encrypted channel to request a response from the client. The default is 0, indicating that these messages will not be sent to the client.

ClientAliveCountMax Sets the number of client alive messages which may be sent without sshd(8) receiving any messages back from the client. If this threshold is reached while client alive messages are being sent, sshd will disconnect the client, terminating the session. The default value is 3. If ClientAliveInterval is set to 15, and ClientAliveCountMax is left at the default, unresponsive SSH clients will be disconnected after approximately 45 seconds. Setting a zero ClientAliveCountMax disables connection termination.

On the client side, corresponding options exist in etc/ssh/ssh_config, but changing them occurs better on a per-user basis in ~/.ssh/config (instead of adding them manually to each connecting ssh call via the -o command-line parameter):

TCPKeepAlive no (default yes)
ServerAliveInterval 30
ServerAliveCountMax 240

The meaning is the same as above, but the roles are reversed: now, the client sends an alive message to the server every 30 s, and the client drops the connection if it didn't receive an answer from the server within 2 hours.