How To Write a TELNET Server or Client
Introduction
TELNET is a protocol designed way back when dinosaurs still roamed the earth, chasing cavemen and operating large mainframe computers connected to remote line printers. The protocol doesn’t see much use in mainstream computing, although it’s still popular on various IBM mainframe installations and, which you the intrepid reader of my humble blog are more interested in, text-based Multi-User Dungeons and similar online games.
Modern TELNET clients are vastly different than the line printers of yore. We have graphical terminals in which our screen can update instantly, and the drawing cursor can move about freely painting characters anywhere it wants in an assortment of colors and styles. Line printers, on the other hand, simply printed horizontally, occassionally chugging down a line and continuing onward. We generally don’t care about those anymore, though. Very few people writing a TELNET server today are expecting their client to be using a line printer. Most of those writing TELNET apps today are probably writing MUDs, or modifying MUDs, or writing a client for MUDs.
“So,” you ask, “how does one write a TELNET server or client, anyway?” The answer is thankfully quite simple. I am going to assume that a familiarity with the basic networking APIs is already possessed, but if not, a few minutes on Google should help.
Basic Concepts
For the most part, TELNET is simply nothing more than sending characters back and forth between the client and the server. Old line printers and networks were half-duplex, meaning that only one side could send data at a time, and the other side had to wait for permission to send. While the protocol still technically uses those rules, they are ignored by all MUD software, as well as most general TELNET software, so we won’t worry about those. A server which simply sends ASCII text to its client and receives ASCII text in response is a completely functional TELNET server, and a client likewise is the same in reverse.
There is more to TELNET, howerver. TELNET offers a variety of options, which range from options like enabling full-duplex mode (not really necessary these days) up to controlling the display of what a user types on his screen. TELNET does not control things like cursor positioning or text color, however. Those are a separate protocol, which I’ll touch on briefly later in this article.
From the perspective of MUD developers, possibly the most interesting feature of TELNET is the ability to control the display of what a user types. Normally, when a user types a key in his TELNET client, it is immediately displayed on his screen, and then sent to the server. This makes typing nice and quick. However, sometimes the server wants more advanced control of the display of input, such as to synchronize it with its own rendition of the screen… or to suppress it entirely, such as when a user is entering his password.
TELNET makes use of a simple control code scheme. In a way, you can this of this as being analogous to the \ escape sequences found in almost all programming languages. For example, \n produces a newline in a string, while \\ must be used in order to create a single backslash in the string. TELNET does the same thing, except instead of a \, it uses a special value called IAC (Interpret As Command), which is equal to the number 255. (You might note that 255 is the largest integer that can be stored in a single 8-bit byte.)
When operating in half-duplex mode, for example, one end of the communication must send the GA (go ahead) signal to let the other end know that it can begin sending. This is done by sending two bytes over the network pipe, IAC GA (255 249). If the client used the interupt key (control-C), the client might send the interrupt signal to the server by sending the two bytes IAC IP (255 244).
Normally TELNET is not 8-bit clean. That means that the plain text data sent between the two ends can only be 7-bit ASCII values. It is possible to put TELNET into binary mode, however, which allows the use of 8-bit values. In this case, it may be necessary to send the value 255, but not have it interpreted as a TELNET command. Just like the \ escape sequence, this is done by doubling up the special character. So, to send the value 255 and not have it processed specially, send the two bytes IAC IAC (255 255).
Option Negotiation
Being able to send 8-bit data over the TELNET connection is pretty handy. It lets you support non-ASCII character encodings, like UTF-8 or ISO-8859-1. You know how to properly send the character value 255 without confusing TELNET, but the client or server you’re talking to keeps doing funny things when you send it values over 127 (that is, any value that doesn’t fit in 7 bits). That’s because first you must negotiate the BINARY option with the remote end of the connection.
This is where TELNET option negotation comes in. TELNET has four special codes for negotiating options: WILL, WONT, DO, and DONT. These commands are a little different compared to normal commands. They start with the special IAC value just like other TELNET commands, but they are three bytes long instead of two. The third byte sent is the option code for the option you are negotiating. The BINARY option is code 0.
So, what do those four negotiation commands mean, exactly? Each has two meanings, based on context. For example, the WILL negotation either means, “I am willing to use this option, if you are,” or it can also mean, “I am acknowledging your request to begin using this option.” Let’s say that you are writing a client that wishes to enable BINARY mode. You must ask the server if it would like to do so, by sending it the sequence IAC WILL BINARY. The server will then respond with one of two commands: either IAC DO BINARY (”I accept”) or IAC DONT BINARY (”I refuse”). If the server accepted, your client is now free to send 8-bit data to the server.
However, the server is not at this point permitted to send 8-bit data to the client. The server might request it in the same fashion as the client, but with the roles reversed. On the other hand, the client could request that the server start sending 8-bit data by telling the server to enable the BINARY option. This is done by using the DO command, by sending the bytes IAC DO BINARY. The server will then respond with either IAC WILL BINARY (”I accept”) or IAC WONT BINARY (”I refuse”).
All TELNET option negotiation works this way. One end either advertises that it capable of using the option with WILL or requests the other end to use the option with DO, and the other end responds in the affirmative or negative. However, things can get a little more complicated. Let’s say we have a naive client talking to a naive server. The client wants to enable BINARY mode, so it sends IAC WILL BINARY. The server accepts, and responds with IAC DO BINARY. The client however, being naive and incomplete, doesn’t know if the server is acknowledging a prior request or initiating a new request. The client assumes it might be initiating a new request, and sends the appropriate response IAC WILL BINARY. The server, also naively written, believes the command to be a new request, and responds with IAC DO BINARY. The client and server are now sending back these two commands over and over, eating up bandwidth and not really accomplishing much.
For this reason, a complete TELNET implementation must track the state of each option for both the local end and the remote end. Each option has three states: enabled, disabled, or unknown. This can be implemented with two 256-element arrays containing an enum denoting the enabled/disabled/unknown state of the option. All elements in both arrays are initialized to unknown. An option with a value of unknown is effectively disabled, but there is more to it, and yes, the local option set also needs the unknown state. Say that you have a client talking to a buggy server that requests the BINARY option be enabled, but doesn’t actually support the option and gets into the infinite loop described above. The server sends the IAC DO BINARY sequence. Your improved, non-naive client looks at its local option array and sees that the BINARY option is set to unknown. The client now enabled the option and responds with IAC WILL BINARY. The buggy server responds with IAC DO BINARY. The client sees, however, that the option is already enabled. Therefor, it does not need to send a response. This effectively breaks the loop caused by the buggy server. Additionally, the client can look at the array of server options and effectively knows not to send a request or response to a server option that is already set to enabled or disabled.
Now, in practice, it is usually not necessary to use all those arrays of flags. Most MUD servers and clients do not, and they work just fine. This is particular because most of the options used in MUDs are “one way” options; that is, only one end of the connection ever requests them. A MUD client generally never ends an initial IAC DO ECHO (which would tell the server to echo everything the client sends back to the client), so when the client receives an IAC WILL ECHO it knows that the server is requesting to enable the option itself, and the command is not a response acknowledging that the client begin echoing data back to the server (which would pretty badly break things for the client). So long as the client never talks to a really broken server, a client could get by with just a handful of flags for the options it supports. The same goes for the server. Just be careful for any options that both the server and client use (like BINARY) to make sure you only respond when the other end is requesting the option, and not when the other end is acknowledging the option, and your application will work just fine so long as the other end isn’t totally broken.
There is a general rule that will help for implementations that don’t use the full 256-element arrays. When receiving a request to enable an option your application does not support, always refuse. When receiving a request to disable an option your application does not support, don’t respond at all.
Let’s take a brief look at that ECHO option. ECHO is option code 1. For MUDs, and in truth most TELNET applications, the server is the only end that ever performs echoing. If a client echoed back everything the server sent to it then it would probably result in another infinite loop. The server would say something, the client would send it back, the server would interpret that as a command and say something back (possibly just an “unknown command” error), and the client would echo that back to the server, which would interpret it as a command… bad stuff.
However, it’s generally pretty nice when the user types something in and whatever he typed shows up on his screen. A TELNET client will generally always prefer this, and by default it will print anything anything the user types on the user’s screen. A server will sometimes want to disable this, most commonly when it is requesting a password. However, TELNET has no option for “hide the user’s input.” Instead, we have to use a sneaky trick. If the server sends an IAC WILL ECHO, that means that it is willing to echo back everything the user types. Pretty much all clients will agree to this, and they will respond with a IAC DO ECHO. At this point, the client no longer prints the keys the user types in. The client is expecting the server to do this itself. However, nothing actually requires the server to do so. It could echo back the user’s input after transforming it (turning it into stars), echoing it verbatim, or just echoing nothing. When the server is finished retrieving the user’s password, it then tells the client that it no longer wants to echo by sending IAC WONT ECHO. The client then acknowledges this with IAC DONT ECHO, and will start displaying what the user types in again.
Note: the Windows TELNET client is notoriously broken with its handling ECHO. The client will gladly accept when the server sends IAC WILL ECHO, but when the server sends IAC WONT ECHO, the Windows client will not start echoing local characters any more. Also note that, unlike almost every other client, the Windows client only operates in character mode. That means that each character is sent to the server as it is typed, while most clients only send whole lines. There are ways to tell a client to go into character mode or into line mode, but the Windows client only supports character mode.
So, now you have option negotiation working, as well as 8-bit support with proper escaping. However, you’ve heard about this NAWS thing, which lets your client tell the server how big the display window is so that the server can do fancy layout. NAWS is option code 31. A server that wants window size information will send an IAC DO NAWS, and a client which supports it will respond with IAC WILL NAWS. But… now what?
Sub Options
Option negotation is only capable of enabling or disabling an option. However, some options, like NAWS, control features which need to be able to send more complex data using the protocol. The NAWS feature needs a way for the client to tell the server the number of rows and columns in the client’s display.
For features like these, TELNET uses the SB command, which is called a “sub option.” SB is code 250. This command is rather special. It starts with three bytes: IAC, SB, and then the option code, such as NAWS. It is then followed by an arbitrary number of bytes, which we’ll call the payload. End of the of the sub option is marked with the two byte sequence IAC SE. SE is code 240. So, what do those bytes between the initial three byte sequence and the ending two byte sequence mean? Well, it depends on the option.
NAWS send two 16-bit integers as its sub option payload. Each integer is in network byte order. The first integer is the number of columns (width), and the second integer is the number of rows (height). So, a client with 80 columns and 24 rows would, after the NAWS option has been enabled with option negotiation, send the byte sequence IAC SB NAWS 0 80 0 24 IAC SE.
One must be careful when writing code to handle sub options. A very large number of MUD servers and clients do not do this properly. Let us pretend, for a moment, that a user has some particularly large terminal… say, 255 columns and 61440 rows. The NAWS sub option byte sequence would be IAC SB NAWS 0 255 240 0 IAC SE. However, remember that IAC is 255 and SE is 240. That means that the bytes are equivalent to IAC SB NAWS 0 IAC SE 0 IAC SE. See the problem? Correlcty implemented software will parse that as a sub option with a single byte in its payload, followed by a zero byte and then an IAC SE sequence, which is illegal. Plus, the NAWS sub option would be the wrong size, which is also illegal. The correct thing for the client to do is to escapse that byte equal to 255 with a double IAC sequence, just like the \\ escape. So the correct thing for the client to send woul dbe IAC SB NAWS 0 IAC IAC 240 0 IAC SE. While it looks like the payload is 5 bytes, the server would convert the IAC IAC into a single byte equal to 255 in the buffer it stores the sub option payload in. However, many incorrectly written MUD servers do not do this; after receiving IAC SB NAWS, they then look for exactly 4 bytes for the payload (ignoring the values of those bytes, even if they contain 255), and then immediately expects IAC SE (sometimes they don’t even check that they actually get IAC SE, they simply read in two bytes and call it done). It is thus impossible to write a client that will be able to handle this situation both with correct servers (which require that the IAC be escaped) and incorrectly written servers (which require that the IAC not be escaped).
Fortunately, the scenario is rather unlikely to occur. There is little benefit in a client that display 61440 lines, even if your screen could somehow handle it. Furthermore, while the proper escaping of IAC bytes within a sub option payload is essential for some options, almost none of those are used in MUDs and thus they should never be sent to those poorly written MUD servers. However, if you’re writing new software, even for a MUD, it is a very good idea to correctly process all sub option commands. The correct way to handle NAWS is to use a buffer to read in the sub option payload (performing IAC escaping as you do so), and once the IAC SE is read, to then check that the payload buffer has exactly 4 bytes in it before processing the command.
Alright, so now you have all the low-level TELNET machinery working, and you’re even supporting cool things like window size notification. Now there’s that tricky deal with actually displaying and sending text properly. See, while I wasn’t lying when I said TELNET just sends raw text back and forth for input and output, there are a few tricks to how that text is interpreted, especially if you want to support fancy colors and stuff.
Newlines
Welcome to the shortest section of this article! TELNET newlines are expected to be the two-byte sequence CR LF. That’s byte values 10 and 13, or \r and \n. Just sending \n by itself or just \r or sending \n \r may cause some funny things to happen.
When reading and writing text files, a newline is usually represented by just a plain LF, or \n. Even on systems that store a CR LF sequence in text files, like Windows, the standard file I/O facilities will automatically translate back and forth between \n and \r \n when reading and writing text files. However, when you are displaying text to a terminal, even on systems like UNIX, the terminal might be in a mode in which a solo LF (line feed) only does what it was originally meant to: cause the cursor (the print head in old line printers) to move down a line, but not return to the start of the line. The CR (carriage return) character tells the cursor (or print head) to return to the beginning of the current line. So, in order to move down to the beginning of the next line, you’d need to send \r \n (or \n \r).
TELNET, being a protocol designed specifically for driving those old line printers, works the same way. Even on modern systems, many clients will treat a solo LF as just a line feed, and many servers will not recognize a solo \n as being the end of a command. So, if you’re writing a server and your output only uses a \n for newlines, be sure to translate those into \r \n (CR LF) when you send the data to the client. If you’re writing a client, be sure to send \r \n whenever the user hits enter.
Now, getting newlines to work properly is thrilling and all, but even more thrilling is that there’s not much more to say about TELNET itself. Sure, there are some extra commands to learn (the GA command used in half-duplex mode, which TELNET is in by default, can be handy to learn about, especially since MUDs use it for some fun tricks), and some other options that can be useful, but there’s no more actual protocol machinery to learn about. Now it’s on to colors and cursor control, which isn’t actually a part of TELNET at all.
ANSI Terminal Escapes
First, let’s look at terminal types. See, a long time ago (and, actually, right now, too) there were a bazillion different line printer and graphical terminal products on the market. Infuriatingly, pretty much every single one had its own proprietary protocol for controlling its special features. Even two terminals made by the same company would often have different (though usually similar) protocols for controlling color codes, cursor positioning, and other features. Even today, the text console on Linux uses a slightly different protocol than the text console on various other UNIX and UNIX-like operating systems, which themselves are all different from each other. Graphical terminal emulators, which is what most of us are using (and which includes your average MUD client), can have their own protocols, too. The modern xterm variation (xterm is a standard graphical terminal emulator for Linux/UNIX systems) is very slightly different than the popular terminal emulator I use on my Linux desktop, for example.
In order to properly handle all of these different terminals, a TELNET server would need to ask the client what kind of terminal they are using (yes, there is an option in TELNET for this), and then consult a library that maps common operations, like “clear the screen,” into the proper sequence of control codes for that particular terminal type. If you’re writing a real TELNET server, you’re going to need to get familiar with the termcap and/or terminfo libraries, as these provide those services.
MUD servers and clients, however, don’t need to care about such things. See, all modern terminal types, while they have slight differences, are based off of the ANSI terminal specification. This specification includes a number of common control codes, like setting terminal color, clearing the screen, or moving the drawing cursor to a specific position in the terminal window. A MUD server need only support these ANSI terminal codes, and can safely assume all clients support them as well. Most regular TELNET clients, running on modern terminal emulators, will do so. Most users of MUDs use a specialized MUD client which will interpret the control codes itself, and then translate those into whatever commands are appropriate for the display, so even a MUD client running on some non-standard terminal will still be compatible with a MUD server that only uses ANSI control codes. So, we’re just going to talk about ANSI control codes from now on.
Now that we’ve gone through three paragraphs of boring and mostly useless exposition, let’s get to the meat of things!
All control codes begin with the ESC character (\e, or 27). In general, the ESC character will be followed by a [ (left-hand square bracket), and then possibly by a command payload, followed by an ASCII letter denoting the actual escape command. For example, to clear the screen, you might use ESC [ 2 J ESC [ H. The J command does various screen-related actions, and the payload 2 is what tells the J command to clear the screen. That technically just clears the screen, though, leaving the cursor at whatever position on the screen it was already at. The H command tells the cursor to return to the upper left corner of the screen (Home).
Setting the color, and an assortment of other visual display settings, is done with the m command (Mode). The payload for the m command is one or more numeric values, separated by a comma. The value 0 means “reset the display settings to the default.” The value 31 means “set the text color to red.” So, to display the phrase “Red Baron” with the word “Red” in the color red and the rest in the default color, the server would send ESC [ 31 m R e d ESC [ 0 m _ B a r o n. (The _ represents a space.)
Remember that you can include multiple values in your payload for the m command. If you want to display something in green (code 32) and wanted to make sure that all otehr display mode settings, like background colors, were disabled, you would sent ESC [ 0 ; 3 2 m.
You can set the cursor position using that H command we saw before. Simply provide the row and column, separated by a semi-colon, in the payload. So, to move the cursor to the second row at column 20, send ESC [ 2 ; 2 0 H.
That’s pretty much the gist of ANSI control codes. You can find a fairly complete list of codes here.
Remember that not all commands have a [ after the ESC character. A decent strategy for parsing these control codes on the client end is to look at the first character after each ESC. If the character is a [ then keep buffering input until a letter character is received, then process the buffer. If the character after the ESC is not an [, then immediately process the command.
For MUD servers, it is a good idea to also include a basic ANSI control code parser, solely for the purpose of stripping such codes out of input sent by users. While you’re at it, be sure to strip out lone CR or LF characters not part of a newline, the BEP character (\a), and other character codes. Imagine a user who sends a command line to your server with “say ” followed by a couple dozen BEP characters in it - every other player in the room will be treated to a long series of annoying beeps (if their client is supports it, which some do). You can just strip out every non-printable character, which is any code less than 32 (or just is the C isprint macro from ctype.h). On a similar note, remember to always escape IAC bytes in your output that aren’t meant to be a part of a TELNET command, otherwise malicious users might find interesting ways to break other users’ clients using commands that let them send text to other players.
And that’s all, folks!
Syndicated 2007-12-10 04:48:22 from Sean Middleditch
