VIA (Virtual Interface Architecture) is an application interface that gives user level processes direct but protected access to network interface cards. This allows applications to bypass IP processing overheads (copying data, computing checksums, etc.) and system call overheads while still preventing one process from accidentally or maliciously tampering with or reading data being used by another.
VIA is an industry standard for high performance communication on system area networks.
VIA consists of three components: a user-level library, kernel code that sets up protected connections but then steps out of the way, and the network interface card itself.
VIA is designed to allow acceleration by special hardware. Such hardware directly reads data structures in user memory and moves data to and from user memory, requiring no intervention from the host processor in normal operation except for connection setup. VIA is designed so that this hardware is relatively simple and easy to build. VIA can also be implemented entirely in software, where for protection data is sent using a fast system trap to priviliged code (but not the main body of the kernel, as in a system call) and received by low level interrupt handlers inside the kernel. This is the technique used by U-Net.
For more information about VIA, see these documents:
M-VIA (Modular-VIA) is a complete high-performance implementation of the Virtual Interface Architecture for Linux. It was written at the NERSC center at Lawrence Berkeley National Laboratory. M-VIA consists of a loadable kernel module and a user level library, and also requires a modified device driver (which is also a loadable kernel module). In the case of VIA over loopback, M-VIA uses a separate loopback device driver since there is no need for it to interact with the normal loopback device.
M-VIA is designed to be easily portable to new network devices. If an appropriate device class already exists for a network card, only a few changes to the device driver are needed.
M-VIA coeexists with traditional networking protocols, so you can run TCP/IP on the same physical network.
M-VIA passes the strict Intel conformance tests.
M-VIA does not require special hardware. It can run on vanilla network hardware and can also run on "VIA-enabled" hardware. While the VIA standard was designed so that it is easy to accelerate with hardware, it does not require such hardware.
As of the release 1.0 (September 25, 1999), M-VIA supports:
M-VIA 2 will add support for several high performance VIA-aware NICS, including Giganet, Myrinet, and Servernet II.
We are recommending that people wait until after release 2.0 before developing new drivers, as the basic basic internal design is likely change. At that point, you can develop one yourself, or let us know what you want and we'll work on whatever is most commonly requested. Please send requests to firstname.lastname@example.org.
U-Net is a research project whose central idea was that it is possible to allow an application to interact directly with network hardware while maintaining protection - that is, not allowing one process to read or modify data sent to or from another process. Of course this isn't completely possible without some support from priviliged (kernel mode) code, but U-Net showed that it can be done with very little overhead.
VIA is a standard with support from powerful players, not a research project, so it has the possibility of being widely used. The fundamental idea of providing a protected virtual network interface to user-level applications apparently comes from U-Net. VIA goes beyond U-Net in several areas: it is designed to be easily accelerated by hardware; it allows direct transfer to/from any area of user memory from/to the network, while U-Net restricts communication to a static queue area (in this respect, U-Net looks more like a real network interface). U-Net/MM, a successor to the original U-Net, allows buffers to be in any location in memory, and does not require them to be pinned, as VIA does. VIA provides more functionality, including RDMA reads and writes, and connection management.
M-VIA 1.0 requires a kernel from the 2.2 series, 2.2.2 or later. We recommend 2.2.5, 2.2.12 or higher.
This depends on the program of course. Currently you have to start M-VIA processes by hand -- there is no "viarun" that distributes VIA processes over a cluster. After the processes are started, they communicate with each other use an accept/connect protocol. To connect to another process you need a hostname, and the hostname-to-address translation is done through /etc/vip_hosts. See the vpingpong example in the M-VIA distribution for an explicit example.
Included with MVICH is a simple process startup mechanism that will start up all processes automatically.
Absolutely. The loopback device is recommended for lowest overhead, though an M-VIA device associated with a real network will also work.
Note that VIA communication on an SMP is very efficient, as it only requires a single memory-to-memory copy, directly from the memory of one process to the memory of another. This is not possible with standard shared memory mechanisms unless data from at least one of the processes involved in the communication is allocated in a shared segment. Normal shared memory transfer requires two memory-to-memory copies using an intermediate buffer. Because of this, VIA communication within an SMP should have twice the bandwidth of other message passing approaches.
Yes. Configure the network as your normally would for IP. Of course the VIA-aware device driver must be loaded for VIA to work also.
M-VIA was developed under U.S. Government contract at Lawrence Berkeley National Lab. The M-VIA license was developed by LBNL lawyers. The M-VIA license is similar to the BSD license, but has extra language to make the lawyers happy.
It is our intent to make M-VIA as useful as possible, and we believe the license accomplishes that. If you have questions or concerns about the license, please write to email@example.com. A few folks have pointed out some silly wording in the license. We are in the process of getting that fixed, but it involves lawyers, so please be patient.
A rudimentary version of MPICH for VIA called MVICH is now available. We do not consider it to be ready for prime time, but we are making it available to people who are interested in working with alpha-quality software and helping to debug it. Please contact firstname.lastname@example.org if you would like to try it.
We understand that LAM has been ported to VIA as well.
We're working on it and expect to have a rudimentary version available in September 1999.
As source code is available, and the license allows it, you're welcome to do M-VIA development. We will continue M-VIA development and M-VIA distribution, and encourage you to keep us informed of what you're doing and to send us your code to be incorporated into the release. We recommend that you do not do serious development until release 2.0, as the internal structure of M-VIA may change substantially. Thereafter it is likely to be more stable, though the standard disclaimer of "research prototype software" applies.
The following numbers were collected on PII-400 machines with Kingston fast Ethernet cards (Tulip) connected back to back and Packet Engines GNIC-II (hamachi) cards connected back to back. The kernel version was 2.1.131, with SMP enabled, and the M-VIA version was 0.9.2.
|Network||Protocol||Latency (us)||Bandwidth (MB/s)|
|Packet Engines GNIC II||TCP||59||31|
|Packet Engines GNIC II||M-VIA||19||60|
|Tulip Fast Ethetnet||TCP||65||11.4|
|Tulip Fast Ethetnet||M-VIA||23||11.9|
The VIA numbers are as good as any latency from any high-performance protocol that we know about (e.g., it's about the same as U-Net). Note that Linux TCP number is also the best in the business - to our knowledge the it is the best TCP latency on any operating system or hardware, so beating it by a large margin is non-trivial. We believe that the basic hardware overheads for these NICs are about 20us. For instance, the same tests on 200 MHz Pentium Pros gives nearly the same M-VIA latency, but much higher TCP latency. The important point is that M-VIA adds very little overhead to this lower limit imposed by hardware.
We agree some of the names are broken, but unfortunately they are specified in the VIA standard, so there's nothing we can do about it. If we changed the names, VIA applications wouldn't be portable.
Good question -- Linux TCP performance is indeed quite good. There are several reasons you might want to use M-VIA.
The M-VIA license allows modification, redistribution, and commercial use.
Valid quality of service values are not specified by the VIA standard. M-VIA accepts the value "0" (zero) only. Any other value will result in an "Invalid QOS" error.
The VIA standard specifies three levels of reliability: unreliable delivery, reliable delivery, and reliable reception. Reliability generally depends on hardware. Some hardware will provide reliability (e.g. Servernet II, Giganet); other hardware (e.g. Ethernet) won't.
For higher-level applications, such as MPI, it is not entirely clear whether unreliable or reliable delivery is better (unless, or course, the hardware is reliable, in which case it is better to have reliable VIA). Reliability can be added at any level of the protocol stack and there are good theoretical arguments that implementing it at higher levels can result in more efficient communication. We are building MPI on both unreliable and reliable VIA. Some applications, such as video streaming, do not need reliability.
One possible cause is that you are running an SMP kernel with M-VIA compiled for non-SMP. Verify that kernel you are running was built from the same kernel source (specifically, the include files) that was used to compile M-VIA.
There are several possible causes.
Without hardware support, M-VIA is zero-copy (no memory-to-memory copy) on sends, and one-copy on receives. The copy on receive is unavoidable for any protected communication without special hardware suport.
With hardware support, M-VIA will be zero-copy on both send and receive.
Sortware-only M-VIA uses a fast trap on the sender to avoid the overhead of a system call. On the receiver, data is received directly with a low-level hardware interrupt that does all the processing necessary. With hardware support, M-VIA will not require the fast trap and will move data into the receivers memory without interrupting the processor.
M-VIA and the Berkeley VIA Project (BVP) are separate implementations, but we are working together.
More information about BVP is available at the BVP home page.
NGIO and Future I/O were proposed industry standards to replace the PCI bus. A nasty battle of industry giants has recently been averted as the two camps have agreed to unite behing the System I/O (SIO) standard. See press releases at http://www.futureio.org and http://www.ngioforum.org/press/index.html. The System I/O model is that peripherals will be connected to a switched SIO fabric at speeds up to 6 GByte/s. VIA will be a high-performance communication interface for SIO and will thus be a critical part of SIO infrastructure.
We have learned a number of things from M-VIA 1. One of those is that good support for VIA-aware hardware (hardware that understands VIA descriptors and has support for the memory protection features of VIA) requires modularity in the user-level VIPL library, as well as inside the kernel. For this reason, it is harder to port M-VIA 1 to this type of hardware than we had hoped. M-VIA 2 will feature a completely redesigned internal structure, that makes it easier to write network "plug-ins" that enable VIA for specific networks. M-VIA 2 is under active development. We will do a developer's release (for developers of M-VIA plugins) before the end of September, 1999. M-VIA 2 will support Giganet, Myrinet, and Servernet II.
There best references are the VIA spec itself, and the Intel
Developer's guide. There are no textbooks on VIA that we are