Bandwidth used to be expensive. Now, it's cheap. Local and long distance phone companies are scrambling to drop rates, increase capacity and be part of the explosive growth in data services. This is good news for those of us building enterprisewide networks. It allows us to satisfy the demands of users for faster lines and reliable services, and to build wide area networks (WANs) that reach into every nook and cranny of the organization. That means central sites, branch offices and even the homes of telecommuters.

Linking far-flung locations does not require a lot of money or big pieces of hardware. Such equipment--most often referred to as a branch office or remote office router--typically needs nothing more than a single Ethernet or Token-Ring interface, a single WAN port and some degree of expandability. But don't let the name fool you. Most of the routers in this category can easily keep up with high-speed data transmission and could manage any single WAN link required by almost any size office.

In putting this article together and to help us clearly understand the nature of the hardware, we put 16 branch office routers through a variety of paces. As you will see, there are many differences between each piece of hardware and different vendors' hardware will be suitable depending on your own specific needs. We set up a frame relay network, detailed in ''Building a Frame Relay Network,'' on page 64, and put each of the 16 routers through a series of tests.

We found that the Cisco product line is solid, full of features and is an excellent choice in almost any environment. It is the standard by which all others are measured. However, if your primary environment is frame relay, take a close look at the Digital DECbrouter 90. You get Cisco software and manuals, with better performance for less money than a fully configured Cisco 2500.

For pure performance and a wide range of protocols, the 3Com product line is another excellent choice. The long boot time of the 3Com Netbuilder we tested was annoying and unnecessary, but the snappy performance and command line interface made it a favorite. When 3Com releases compression over frame relay, it will be an even stronger contender.

Even though the ACC Nile doesn't meet our criteria for remote configuration, the performance of ACC's express queuing combined with compression over frame relay add up to winning features. The Nile performs particularly well in 56-Kbps environments and should be on your short list.

In a restricted environment where IPX and TCP/IP are the only protocols you will care about, Cisco/Newport's LAN2LAN is a great product. Its hardware data compression makes the router fast, even if it doesn't have the same feature set offered by Cisco, Digital, 3Com and ACC.

Frame relay brings special requirements for congestion control that only a few vendors have included in their product. Bay Networks' Wellfleet Access Node, ACC's Nile, Cisco/Newport Systems Solutions' LAN2LAN and CrossComm's XL5 lead the pack in frame relay congestion management. The serious frame relay manager will want to look at those as well as Ascom Timeplex's Access Router, which not only can attach to a frame relay network, but can also be a frame relay network. We cursed the documentation but took advantage of this feature in our test lab.

Can They Do T1?

The jump from a 14.4-Kbps or 28.8-Kbps asynchronous modem to a private or frame relay T1 (1.536 Mbps available bandwidth) line is a big one, but not an expensive one. Our test network used both 56-Kbps and T1 frame relay circuits to evaluate how well these routers handle low- and high-speed transmission.

Our first lesson in network economics came when we looked at bandwidth in our test networks. Although the T1 network cost four times as much as the 56-Kbps network to maintain, our performance tests gave the T1 network a 30-to-1 edge in data transfer rates over the 56-Kbps network. Network latency, an important part of network performance for interactive users and transaction-based protocols such as those used in file sharing systems like Novell NetWare, also improved in the T1 network by a factor of 7 to 1. The chart below provides a summary of our findings.

Our goal here was to stress the routers as much as possible, so we concentrated on testing over the T1 network to see who could take the heat. Testing routers is notoriously difficult, since minor changes in configuration can cause major differences in results. Because of TCP/IP's enormous popularity and the wide availability of test equipment, we used TCP/IP as our base protocol to test router throughput and response to stress. In general, these results will apply to any routed protocol, such as NetWare's IPX, AppleTalk's DDP or DECnet's NSP, with minor variations caused by different protocol processing overhead. If you are bridging rather than routing, these results probably don't apply to you.

Our baseline tests were set up to separate the routers that could handle T1 speeds from those that couldn't. To test throughput, we set up six workstations--three masters and three slaves. The masters opened up TCP connections to the slaves and sent 8,192-byte packets of random data to them. Throughput was measured by counting the total number of bytes received by the slaves over a 10-minute period, once the test network reached a steady state.

Most routers had no problems with T1. The top performers clustered around a throughput of between 1.425 Mbps and 1.465 Mbps. Within the error range allowed by our testing, these routers all managed to push through as many packets as the T1 could take. Five of the systems (Cray's BR6000, Source-Comm's XANbox, Motorola's 6520 MPRouter, Morning Star's Express Plus and Livingston's IRX-111 ) showed less than ideal throughput rates, losing between 56 Kbps and 256 Kbps of the T1 channel.

For raw throughput nothing beats data compression. Only two of the routers tested (ACC's Nile and Cisco/Newport's LAN2LAN) offer compression over frame relay (3Com is currently beta testing the capability). Cisco/Newport's LAN2LAN does data compression on their WAN interface card and achieved a stunning 2.5-Mbps throughput, using random data on a T1 frame relay circuit. ACC's Nile, which uses the same CPU for compression and routing, turned in throughput of almost 2.0 Mbps, using the same random data stream. The chart on page 70 provides throughput rates for every router we evaluated.

There are no standards for compression over frame relay, so any network using these routers in compression mode will require the same hardware at both ends.

Real-World Throughput

Although some networks will simply pump data through these routers the way we did, many will use additional features, such as routing protocols, multiple protocol routing, security filters or protocol translation. Because of the enormous variation in capabilities, it would be impractical to devise a fair test that showed how the routers would respond when they were busy exercising all of their features. Nevertheless, we did some sample tests to see how turning on filters, additional routing protocols and protocol translation would affect throughput.

Most routers suffered between five and 10 percent loss from their unburdened T1 throughput numbers, although some took a more dramatic hit. With only a few configuration changes, one router went from 1.428 Mbps to about 550 Kbps. In practice, network managers can always make good routers go bad. As an example of an extreme case, when we fed one router RIP routing tables of 2,000 entries every 30 seconds, it would suffer between 10 percent and 20 percent packet loss during the time it took to process the routing updates.

To fairly test the routers, we decided to see how transaction-based protocol response time would degrade as the routers were stressed at T1 speeds. Our test network in this case had 10 systems: three masters and three slaves stressing the routers by sending large packets (512-byte payload) through them at maximum speed to load the T1 line down and one master and three slaves sending smaller packets (1-byte and 256-byte payloads) at intervals of one second. The goal was to simulate a steady background stream of data transfer with occasional transactions. Ideally, the latency experienced by small transactions on a loaded line would not vary significantly from an unloaded line.

Our tester measured average latency between a master sending a packet to the slave and the time the slave's response was received at the master. As a baseline, we ran the tests using a 50/50 mix of short and long packets only. Response time on all routers was fairly uniform, in the range between 17 milliseconds and 22 milliseconds.

Next, we turned on the background traffic and ran two sets of tests, one with short packets and one with long packets. Four routers emerged as high-performance leaders: 3Com's Netbuilder Remote Office, ACC's Nile, Digital's DECbrouter 90 and Cisco/Newport's LAN2LAN. In both sets of tests, these four performed significantly better than the competition. Two of the routers, ACC and Cisco/Newport, both support compression over frame relay, which gave them significant advantages in latency.

These tests will be good predictors of the performance of any transaction-based protocol over T1 WAN links. Interactive terminal traffic, small NetWare or AppleTalk file operations, and data collection applications all have profiles similar to the traffic simulators we used.

Although the differences are measured in tens of milliseconds, these delays in response time will be noticeable to users as they accumulate across a transaction. For example, an MS-DOS user reading a short file on a NetWare server will pass at least eight packets over the network, four in each direction. Over Ethernet, this transaction would typically be completed in less than 10 ms. With the typical delays seen on an unloaded T1, this transaction would stretch out to 80 ms. On a loaded T1, this would shoot up to at least 280 ms, a difference quite noticeable to users accustomed to speedy response on the local LAN.

These differences will be even more dramatic when 56-Kbps circuits are used. In that case, transactions across a loaded line will stretch from 10 ms to between 4,000 and 8,000 ms.

One surprising result from this test was the difference in performance between the Digital DECbrouter 90 and the Cisco 2500 system. Both use the same base software, version 9.14 of Cisco's router operating system. However, the Digital hardware (also sold by Cisco as the 2000) offered better performance than the Cisco hardware running the same software.

Features and Capabilities

All the routers we looked at route TCP/IP traffic pretty well. From there, the similarity ends. Every router vendor is seeking out a niche to distinguish their product from the rest of the crowd. Some vendors, such as 3Com, Cisco, Digital and Bay Networks, aim to be all things to all people, by supporting every known network protocol and offering a dozen different hardware configurations.

Others, such as ACC, Ascom Timeplex, Hewlett-Packard and Retix, have taken a more modest approach, identifying very popular protocols and routing environments and aiming to support those. At the lowest level of multiprotocol support are products that aim for a specific environment, such as pure TCP/IP, NetWare or AppleTalk, from companies such as Cray Communications, Engage, Livingston, Morning Star, Cisco/Newport and Source-Comm.

To help tell them apart, we picked some key characteristics and looked at each product from a different point of view.

Hardware Flexibility

We restricted our tests to products with a list price of $4,000 or less because we felt that you shouldn't have to pay more than that for a simple frame relay router. Within that price range we found significant differences.

Most vendors offer only a very limited configuration: one, two or three synchronous serial ports (for WAN connections) and a single Ethernet port for the LAN. The limitation on synchronous ports isn't a big deal for most sites, because one seldom needs more than one or two synchronous connections at a time in a frame relay environment. Total throughput could also be a problem if they had more ports, since few of these boxes could handle more than a small number of T1 circuits at full speed. If you really need a bunch of serial ports, perhaps for low-speed connections, the Motorola 6520, Livingston IRX-111, Morning Star Express Plus and Cisco/Newport's LAN2LAN can all add four (or more) synchronous lines in a single low-cost chassis.

The lack of support for multiple Ethernet ports at this price point can be a problem. As many organizations connect to the Internet, the need for simple packet filter routers that don't cost a fortune has grown. Products we looked at from Cisco's 2500 series, Motorola's 6520, Morning Star's Express Plus and Cisco/Newport Systems Solutions' LAN2LAN can all grow beyond a single Ethernet connection.

Configuration

Configuring routers is a hellish experience. All vendors have a different paradigm for how they handle the problems of interfaces, protocols and services. Engage's Express Router, Cisco/Newport's LAN2LAN and Source-Comm's XANbox all require a PC or Mac interface to configure the router and aim the configuration at a part-time network manager interested only in minimum flexibility. Of course, in a branch office environment, this style may be just what the network manager is looking for.

Of these three, Cisco/Newport Systems Solutions does the best job. Engage offers the choice of Mac, PC or Unix user interfaces, but only the Mac is out of beta testing and shipping as we go to press. Source-Comm literally had to write new code before we could completely configure frame relay. The support was excellent, but it would have been nice not to have needed it. All three products are severely limited in what you can do with the simplified user interface. For example, with the Engage Express Router, it's impossible to build an AppleTalk network over non-mesh frame relay PVCs.

CrossComm's XL5 and Bay Networks' Wellfleet Access Node both required a fairly extensive application to be installed on a Windows-based PC for configuration. These products offer a complex set of interfaces and routing options, which makes their configuration utilities similarly complex. Nevertheless, the PC-based configuration utilities kept us from having to learn any of the native commands of the routers. Since network managers usually have an infinite amount of knowledge to memorize and only finite capacity, the approaches taken by Bay Networks and CrossComm have long-term advantages. Another plus that branch office managers will appreciate is the capability to save and edit configurations of remote routers locally for later downloading.

The next best thing to menus on a PC are menus on a terminal. This is the approach taken in the Ascom Timeplex Access Router, the Motorola 6502, the Cray Boundary Router, the HP 27289A and the Retix ROUTERXchange. For the simple set of services offered by the Cray router, this works pretty well. The more complex environments in the other routers made the menus more distracting than helpful after we became familiar with the products.

Motorola takes the prize for the most difficult router to configure. Of the 16 different interfaces we had to learn, the Motorola router was the most difficult to understand.

At the bottom of the slick user interface barrel are routers from 3Com, ACC, Cisco, Digital, Livingston and Morning Star. These actually require you to type in whole words and sentences to make them work. It may be our bias, but this style of configuration is our favorite. Routers configured in this way are easy to document, easy to reproduce and simple to modify.

All these routers take some getting used to, but we think that network managers who have lots of routers to manage will appreciate the power of a command-line interface. The only loser in this bunch is the CrossComm XL5, which has an artificially short command buffer so that some commands that are more than about 70 characters simply cannot be entered from the keyboard. For this reason, we consider it unconfigurable from the command line.

Flexibility is another dimension to the configuration matrix. We feel that any router you can't ''telnet'' to is flawed, because it severely limits the ability of the network manager to fix problems on the fly. Products such as the ACC Nile, the Engage Express Router, the Cisco/Newport's LAN2LAN, the Source-Comm XANbox and the Bay Networks Wellfleet Access Node all need simple remote configuration and don't have it. ACC promised this upgrade would be available soon, but the rest won't be joining it any time soon.

Multiple Protocols and Routing

We found that not every protocol is implemented in the same way. Although every router supports TCP/IP, not all did so with the same elegance. Most problems in heterogeneous configurations came from defects in the routing protocols. We did not test every router against every other router, but we did do a great deal of mix-and-match testing. In general, all of the TCP/IP-over-frame relay implementations would interoperate with each other. Some routers insist on learning TCP/IP routing information through RIP. This is fine if you want to run RIP, but we weren't excited about that requirement.

As the oldest of the TCP/IP routing protocols, most network managers are trying to move away from RIP. Buying new equipment, such as the Motorola 6520, the Cray Boundary Router, the Engage Express Router, the Livingston IRX-111, the Cisco/Newport LAN2LAN, or the Source-Comm XANbox, which requires RIP as a routing protocol, is a poor investment.

Most network managers will be interested in protocol support for TCP/IP, NetWare's IPX, AppleTalk and general Ethernet bridging. For those shops with piles of old gear around, nothing beats the Motorola 6520 MPR. Although it's not a winner in the TCP/IP routing game, the 6520 provides support for legacy protocols we've never heard of from manufacturers who have been out of business for years. If, for example, you've got to move data from an old Burroughs machine across a contemporary WAN, Motorola will support it--no one else provides the level support Motorola does.

For the more traditional legacy needs of IBM shops, 3Com, Cisco, CrossComm and Digital all come to the rescue--although Digital's offerings outside the Cisco-based software are poor in this area. Network managers with substantial amounts of SNA traffic should consider these products for their router needs.

Having an Internet connection doesn't guarantee good support. Xyplex connected to their MAXserver router in our test lab for two whole days and still couldn't get it to work over a frame relay network.

Joel Snyder is a senior partner with Opus One, Tucson, Ariz. He can be reached on the Internet at jms@opus1.com. Ehud Gavron president of ACES Research in Tucson, Ariz. He can be reached on the Internet at gavron@aces.com.

Copyright 1994 by CMP Publications. All rights reserved.