The Electronic Mail Service
at UWO

Peter Marshall, Info Tech Services, UWO <peter@julian.uwo.ca>

Revision: 1.17 This report is available in its entirety in a PostScript version suitable for either viewing or printing or as plain ASCII text.

Contents

1 Background and Summary

At an ITS management meeting on 1995-11-10 the future of Emc2/TAO was discussed. At that meeting we decided that

...it is unlikely that we will have and support only one e-mail package on campus. The university community would not accept the idea of being "forced" to use only one package. The ideal will be to offer a suite of e-mail packages that are inter-operable and fairly easily supported.
To aid in moving toward that ideal, a report was commissioned which
reviews and rationalizes the packages currently supported and their uses/inter-operabilities.
This is that report and more.

E-mail has evolved into an essential core communications service at UWO. With the imminent shutdown of the Emc2 and VAX/VMS systems we conclude that an experienced, cross-group team needs to be immediately established to investigate future e-mail system possibilities and directions for the campus.

2 A Brief Chronology of E-mail at UWO

E-mail at Western has had a long and distinguished career.[*] The following summary provides some perspective on how we reached our current state. That state will be described in more detail in later sections.

late 70s
Single computer, e-mail system on the DECsystem-10 timesharing system
early 80s
the MLnet project linked campus computers together using serial links -- fed e-mail into the X.400-based CDN network as well as having UUCP-based links (both via slow serial dial-up connections)
mid 80s
BITNET connectivity added, IBM PROFs system installed on central VM/CMS system for administrative e-mail. MLnet was used to gateway mail between campus machines.
1988
Internet connectivity established
May 1992- March 1993
CEDAR (Campus E-mail for the Desktop Architecture Review) group -- a joint CCS/DAS group that set out to
...to describe a campus-wide system that we could use to unify and simplify the provision of e-mail services at UWO. (From Focus volume 8 number 1, Electronic Mail for the 90s, March 1993.)

The group quickly discovered that this wasn't possible, but that a suite of ``state of the art'' programs could and would be supported to usher in a new era of functionality, Internet-compatibility and ease of use to replace the line-oriented mail programs with their roots in the 1970s. The recommended programs:

April-October 1993
Administrative Systems tested and then introduced Emc2/TAO via an article in the Information Exchange newsletter: Emc2/TAO - A New Email System, September 1993.

By in large, most of the conclusions of the 1993, CEDAR group still stand. At that time the group studying the issue was

...confident that there is no commercial software that fits smoothly into the Internet environment that is as good and cost effective as the programs described in the articles [in the Focus issue cited above].

The basic conclusion that open, standards-based, post-office protocol version 3 (POP3)-based mail systems should be the ones that we move toward still represents our current direction. With the release in 1994 of the MS-Windows version of Eudora, we have consolidated on Eudora for both the MAC and the PC platform while the use of the DOS-based NUPOP has fallen away as the adherents of the non-windowing world disappear. The LAN-based solution of Pegasus/Charon has evolved into Pegasus/Mercury with Mercury providing POP3-based services as well as a gateway from the LAN-based Pegasus system.

2.1 A Note on Emc2/TAO

Emc2/TAO was chosen as the replacement for the existing VM/CMS-based PROFs system for a number of reasons:

Emc2/TAO has never lived up to its initial promise, the promises by the vendor to provide a more standards-based system and continues to be plagued by operational and compatibility problems. Nevertheless it is widely used by many people throughout the University, especially the non-academic areas. Many of the non-email features of Emc2/TAO, like scheduling, have been widely praised.

3 Mail Systems on Campus

There are basically four types[*] of mail systems represented by the approximately 136 mail servers[*] currently registered on campus:

These mail systems provide e-mail service to about 3,300 of the 6,750 faculty and staff (or about one half)[*] and an estimated 8,000 students (under a quarter of the total). It should be noted that since e-mail is still by no means universal we cannot yet eliminate other forms of communication in its favour.

In this section we'll briefly describe and distinguish each of these, itemize the specific implementations currently in use on campus and provide a cursory glance at the use that is being made of each system.

3.1 LAN-based E-mail

LAN-based systems use the file system of a LAN server (like a Novell server) to transfer mail between users of that server. They typically employ an SMTP gateway to import and export messages to other systems. By far the most used LAN-based e-mail system on campus is the freeware package Pegasus using Mercury as the SMTP gateway process. (Older versions of Pegasus used a stand-alone PC for its SMTP gateway known as Charon, but these are pretty much gone from the campus now.) There are a number of commercial mail packages that use a similar approach, but on campus we are aware only of the Microsoft Mail System in use at PTCE. According to their support technician, Charles Deschenes,[*] they'd be happy to upgrade that to something that would use a POP3 approach as the Internet gateway for their system is ``flaky''.

The teaching hospitals and the Addiction Research Foundation (ARF) use commercial LAN-based e-mail systems (primarily WordPerfect Office) running on Novell servers. Many UWO faculty use the hospitals as their ``home'' e-mail base.

Note: The local LAN-based model and the POP3-based model can be supported by the current client portion of at least Pegasus and newer versions of Microsoft Mail (aka Microsoft Exchange). Thus the choice of client doesn't always dictate the mail delivery model. Increasingly, previously LAN-only clients can make use of the often more flexible POP3-based service.

3.1.1 Usage

We know that there are approximately 22 LAN-based e-mail servers on campus but due to the distributed nature of this approach it is difficult to determine the volume of use. While we don't have a count of the number of messages sent within such systems we do know that during a typical day (at the end of November 1995) about 14% of the messages that pass through the ITS central mail servers originate on LAN-based systems.

If we look at the UWO Directory we find that of the 3301 addresses currently registered there, approximately 15% have listed addresses from LAN-based servers.

3.1.2 Problems

The simple SMTP gateways provided by these products rely on a smart SMTP gateway to deliver most e-mail. This can present a heavy load[*] on the central server(s) that provide this service (julian.uwo.ca in our case). LAN servers do handle client interactions that otherwise would have to be provided by a central POP3 server (currently also provided on julian) so there are advantages to the distributed approach as well.

One difficulty with most LAN-based mail systems is the complexity of accessing them via dial-up connections or from dumb terminal connections at conferences. Often a Citrix or PC-Anywhere-style of connection (using 2 PCs) is necessary (and often not available). A second problem is that most of these servers are currently managed quite independently of others -- which leads either to the requirement for lots of (expensive) expertise at many sites or, more often, poorly managed and sometimes unreliable mail services.

3.2 POP3 Servers

Post Office Protocol version 3 or POP3-based servers use a simple, well defined Internet protocol (described in RFC1081). Essentially a client program provides a user interface and some local storage while the Post Office computer receives and transmits messages when asked. The Post Office is located on a machine that is available 24 hours a day, while the client is only connected to retrieve mail. Typically, current client programs converse directly via an SMTP (Simple Mail Transport Protocol) server like sendmail on a Unix machine to transmit messages.

The primary POP3-servers are popper from Qualcomm[*] for Unix systems, the mercury gateway that runs on Novell servers and the VAX/VMS-based POP3 server from TGV (used at SSCL, but not ITS because our VMS mail software predates TGV). Lately, commercial products like Microsoft Exchange are appearing that support POP3 clients directly.

3.2.1 IMAP Servers

IMAP servers, which can be considered a variant of the POP3-style of server, extends POP3 to allow more of the work to be done on the server. All mail and folders for IMAP users remain on the server and there are many more message manipulation functions available to client programs that use this server. This is essential for people that need to access their mail and mail folders from more than one system (e.g. from on-campus as well as from a home-based machine). In addition, IMAP provides a better level of security for sending messages since IMAP clients use the IMAP server (where the user has been authenticated) to deliver mail messages. Most POP3 implementations, in contrast, don't have message-level control of message deleting and rely on un-authenticated SMTP connections to send mail.

Public domain and commercial IMAP servers are available for Unix and VMS systems. SSCL is currently running an IMAP server (TGV's) on their VMS systems.

3.2.2 POP3-based clients

Currently at UWO the primary client is Eudora for Windows and for Macs. A number of sites that use the Pegasus client in the LAN environment use a POP3-version of the Pegasus client communicating to a Mercury SMTP/POP3 gateway on a Novell server or to a Unix POP3 server. With the recent popularity of Netscape and the development of a built-in POP3-based client for that package in version 2, its use as a mail client has increased. Netscape's client handles embedded hypertext links and graphics better than most clients but current versions don't offer much in the way of sorting and filing of mail messages.

All of the common POP3 clients support the MIME (Multipurpose Internet Mail Extensions) standard for encapsulating binary attachments.

Pine and some of the host-based mail clients can also make use of IMAP or POP3-based servers. These are only in use at SSCL.

3.2.3 Usage

Of the e-mail that passes through the main campus mail hubs, about 32% originates on POP3-based clients. As of November 1995 we were typically seeing over 1500 different POP3 users during a typical day. About 88% of these originated via modem connections.

Of the approximately 10,000 registered users of panther/mustang, almost 60% of them use the system during an average week and about 30% make use of the POP3 service. (About 40% of users are inactive!)

3.2.4 Problems

A major problem with the current POP3-based service is that it assumes that users will use only a single PC to read and file their mail. If this doesn't hold, as when someone has both a home machine and a office machine, then there is no easy way to file a message at work and then access it from the home machine.{[*] The design of the system also often pushes users into leaving all their mail on the server and only down-loading copies to their PCs. This allows a Pine session to get at their messages from any dumb terminal connection but can also lead to excessive disk usage as no messages are ever deleted from the server.

An IMAP replacement for POP3 addresses this problem by keeping the messages on the server so that any machine can access them as well as programs like Pine from a simple terminal session. But it comes with a price: Unless the IMAP client caches e-mail, it means that off-line reading of mail is impossible.

3.3 Host-Based E-mail

These are typically mail systems that run on large timesharing hosts. They communicate locally via the native file system and use sendmail or sendmail-like programs to communicate via SMTP to the rest of the Internet. The interfaces range from simple line-oriented, to full-screen and to X-window-system based applications.

Unix:
Pine, mh, xmh, mail, xmail, mailtool, elm
Cyber:
X.400-based mail system (single host system)
VMS:
Unix mail port (ITS), vendor with TGV enhancements (SSCL), Pine (SSCL)
MVS:
Emc2/TAO (also a client/server version)

Of the above list, all are supported by ITS or SSCL with the exception of xmail, mailtool and elm. The UWO Library Systems Group provides some support for elm on the Sun platform.

Pine and mh support binary document exchange using the MIME format.

3.3.1 Usage

By far, the most popular of the host-based e-mail systems is Pine. About 80 to 90% of users that log into the ITS Unix systems use Pine They generate over 3,500 messages per day. But we know that many Eudora users also use Pine so it is difficult to determine the numbers that use Pine exclusively.

According to the UWO directory, about 82% of faculty and staff list addresses on machines with host-based mail systems. Of these, 56% use Unix or VAX-based systems that also support POP3 clients. 25% are registered with Emc2/TAO addresses.

3.4 Proprietary client/server-based tools

The client-server version of Emc2/TAO and the VAX/VMS Pathworks mail system in use at SSCL are both examples of E-mail programs that, while following a model similar to POP3 or IMAP-based services, have been developed using private and proprietary interfaces and protocols. This means that the vendor's server must be used with the vendor's client program. Typically there is a high degree of integration and blurring between the client and the server in such systems.

3.4.1 Pathworks

The Pathworks system supported by SSCL currently has a large number of users in the Faculty of Social Science but the Lab's stated[*] direction is toward IMAP and POP3-based clients using standard servers. We don't have a lot of experience with Pathworks. Our assumption is that either SSCL shields us well or (more likely) it works well.

3.4.2 Emc2/TAO

Emc2/TAO clients:

The Emc2/TAO server

4 Recommendations - Principles

Or lessons that we've learned.

5 Recommendations for the near term

There are two areas for near term recommendations:

  1. those that deal with simplifying and reducing the numbers of clients and servers supported on the campus network for e-mail and related communications services, and
  2. those that involve an extension or enhancement of our existing service.

5.1 Simplification

To be considered:

5.2 Extensions

To be considered:

6 The Future

We see the personal communication facilities characterized by e-mail today evolving into a multi-media exchange and collaboration system. The evolution will be slow and based on standards widely adopted around the Internet.

Voice, video, hypermedia-links will become commonplace in interpersonal communications. Filing systems must become more automatic and easier to search. World-wide directories (probably not based on X.500) will be integrated into the suites of programs that people use for communicating. POP3 will be superseded by protocols more powerful than IMAP to allow a better and more flexible split between the workstation and the server for mail functions. The distinctions among the clients for e-mail, web-browsing, news reading will continue to blur, as we are already seeing in a crude way with Netscape. Remote access with local functionality and full access to archives, databases and other related functions from anywhere will be required and expected from tools that will help us to collaborate more effectively.

The area will continue to be volatile with some campus groups being much further ahead than others. The only way to bring some semblance of order and to permit universal interoperability is to implement standards that are widely supported and adopted by our peers in the academic community. These standards will continue to evolve.

7 Next Steps - Back to Reality

It is clear that we need to move toward the next generation of mail programs if only to handle the immediate needs of

  1. the current users of Emc2,
  2. the current users of ITS's VAX mail system and
  3. the transition of users from the character-based Pine system toward a GUI-based interface.

The first two are critical as the machines that support these applications are disappearing and, in the case of Emc2, ITS no longer has experienced server support people. The third can proceed at a pace comfortable to the users. One reason to accelerate a move from Pine might be to provide the ``universal'' mail service via a client-server (IMAP) model since that model is, in some ways, easier to control than a system that requires a direct login to a general purpose computer.

To this end we suggest that a working group be established to determine in detail what enhancements to the campus electronic mail infrastructure will be required to accommodate the needs of users while providing continuity with the existing world-wide e-mail system. Since there are major server and client issues that need to be addressed and coordinated this must be a cross-group team.

The group should include as a minimum:

About this document ...

The Electronic Mail Service
at UWO

This document was generated using the LaTeX2HTML translator Version 96.1-c (Feb 29, 1996) Copyright © 1993, 1994, 1995, 1996, Nikos Drakos, Computer Based Learning Unit, University of Leeds.

The command line arguments were:
latex2html -split 0 -t mail.tex -address Peter Marshall, ITS, UWO <peter@julian.uwo.ca> Last update: 96-04-11 00:08 by peter -show_section_numbers -dir /usr/internal/nst/peter/public_html mail.tex.

The translation was initiated by Peter Marshall on Thu Apr 11 00:08:49 EDT 1996

...career.
Thanks to the pioneering and continuing work of Reg Quinton as well as significant contributions from Colleen Bretzlaff, Eva Placko, Debbie Jones, Elizabeth Watt and a host of others at ITS.
...Gateway
Ibid
...types
with some fuzziness and blending in the classification
...servers
The number of mail servers on campus continues to grow. In 1988 we had 19, in 1992 we had 80 and now we have almost 140. We should remember that while many of these machines are single user workstations some are large servers like julian/mustang with over 9000 registered users.
...half)
These numbers are extracted from the UWO Directory.
...Deschenes,
Conversation, January 1995.
...load
During November 1995 we were averaging a message every 3 seconds, 24 hours a day - up from one every 6 seconds in January 1995.
...Qualcomm
Makers of both the public domain and commercial versions of Eudora.
...machine.{
A connection to the office machine's file server is a possibility as is a PC-Anywhere or Citrix style of connection.
...stated
Conversation with Les Flodrowski, January 1995.
...Notes
But this package, from the outside, seems to be very closed and proprietary although Lotus is working on opening things up.
 


Peter Marshall, ITS, UWO <peter@julian.uwo.ca> Last update: 96-04-11 00:08 by peter