ITS Mail Working Group Interim Report * August 27, 1996, v1.13 This document is also available on-line in hypertext format via the World Wide Web as . Contents 1 Introduction 2 2 Keeping Everyone Informed 2 3 EMC2 Research 3 3.1 EMC2 Training . . . . . . . . . . . . . . . . . . . . . 3 3.2 Survey of Key EMC2 Users . . . . . . . . . . . . . . . 3 3.3 Related Issues and Projects . . . . . . . . . . . . . . 4 4 New Directions 5 4.1 IMAP Decision Confirmed . . . . . . . . . . . . . . . . 5 4.2 IMAP4 Client Programs . . . . . . . . . . . . . . . . . 7 4.3 Planned Activities . . . . . . . . . . . . . . . . . . 9 5 Recommendations 9 A More Details on Related Issues and Projects 12 A.1 Issues . . . . . . . . . . . . . . . . . . . . . . . . 12 A.2 Projects . . . . . . . . . . . . . . . . . . . . . . . 13 ------------------------- *. Mail Working Group: Peter Marshall (leader),Colleen Bretzlaff, Debbie Jones and Reina Tebby all of ITS at UWO 1 1 Introduction At the end of March 1996, after receiving the report The Electronic Mail Service at UWO,(1) ITS management followed the recommendations of that report by forming a small working group consisting of the authors of this report to ...look at the alternatives and interoperability of email applications. EMC2 will need to be phased out and the VAX will be replaced within the next year (or so). ( ITS Management Minutes Mar 28(2)) The goal of this project is to recommend a campus mail system(3) to replace the existing EMC2 system (as a first priority) and to become a general, ITS supported, e-mail system that VAX users and others can gradually adopt. Client, server and desktop interoperability issues will be addressed. It is important to note that the project will focus on the e-mail service rather than trying to replace the ``Total Office Automation'' aspects (scheduling, bulletin board, conferencing services, FAX gateway and forms system) of the existing EMC2 product. We expect that other project groups will be formed to work with us on replacing the non-e-mail functions of that system. A recommendation from this group was expected by the end of the summer, but this report recommends that this be adjusted to the end of 1996. Implementation and transition phases will follow during 1997. In the following sections we'll outline the activities and work initiated so far and those planned for the next few months. Our activities have concentrated in three areas: keeping everyone informed; EMC2 research; and defining the new direction. 2 Keeping Everyone Informed o An InTouch article describing the group including some preliminary information for users was written and included in the Summer 1996 InTouch(4). o This Interim Report was prepared for ITS Management and will also be posted to the ITS homepage in the ITS Special Projects section. ------------------------- 1. URL: http://www.uwo.ca/%7epeter/mail.html 2. URL: http://www.uwo.ca/its/itsonly/itsmanarchive/ 3. By a campus mail system we mean both the client front-end program and the back-end systems used to gather, deliver and sort e-mail. It does mean that we adopt a common protocol but that doesn't necessarily mean a single client for all users. 4. URL: http://www.uwo.ca/its/doc/newsletters/InTouch/intouch.html 2 o In early September we will be scheduling a meeting with the campus postmasters (with an invitation to the TUMs group) to inform them about the project, give them a chance to ask questions and to assure them that they can participate. 3 EMC2 Research 3.1 EMC2 Training The three group members that aren't EMC2 users took a short course on the EMC2 package to familiarize themselves with the facilities and features of the package. 3.2 Survey of Key EMC2 Users A rough survey for existing users of EMC2 was developed to the point that it could be used as a tool in interviews conducted by team members. We felt that an informal, hands-on approach to gathering this information was important. A group of 12 key users (representing many others) of the existing EMC2 system were identified and surveyed via interviews. In general, this group was supportive and not at all adverse to the move away from EMC2, especially if they could make the move on their own timetable --- most preferred a summer transition. The group consisted of the following people, included is their department followed by any special reason for interviewing them: o Alcina Stoetzer (President's Office) o Jan VanFleet (Secretariat) o Tammy Robins (Registrar) --- Scheduling and heavy Folder user o Scott MacCrone (Registrar's Office) o Paul Martin/ Karen MacIntyre (Finance) --- use of Forms and Fax o Karen Chelladurai (Graduate Studies) --- representative EMC2 user o Gerry Roberts (Alumni) --- representative EMC2 user o Mary Anne O'Brien (PeopleSoft) for Bob McKerlie (ITS) --- PeopleSoft compatibility o Denis Regnier (ITS) --- Forms user o Geri Harness (ITS) --- front-line EMC2 support 3 3.3 Related Issues and Projects Many issues were identified by the mail working group and reinforced by our interviews with EMC2 users. We classified these issues into three categories to define how they should be addressed. 1. Some have either already been or will soon be addressed by this working group --- Mail Working Group. 2. Others will need to be solved by the follow-on group(s) that will be responsible for the transition of the existing EMC2 users to the new system --- Transition Team. An embryonic version of this team consisting of Reg Quinton and Chuck Reid has already started work on some of these issues. 3. Still others have been identified as having major impact on the mail replacement project but due to the mandate of the group and the independence of the undertaking, we recommend that they be considered as separate projects --- Spinoff Project. These projects should get underway as soon as possible in order to permit an orderly move from the current system to the new one. See Appendix A for more details on these projects and issues. 1. Mail Delivery to Home Directory Project in progress 2. Addressing and UWO Directory Spinoff Project 3. Mailing Lists and Nicknames Transition Team 4. Old Mail Folders Transition Team 5. Keeping Existing Important EMC2 Features o Departmental Mailboxes Mail Working Group o Standard Binary Attachments Mail Working Group o Auto Reply/Vacation Mail Working Group o BBS/Shared Folders Spinoff Project o Calendaring/Scheduling Investigative project in progress o Forms Spinoff Project o FAX Gateway Spinoff Project 4 6. New Features MAPI, Spellchecker Mail Working Group 7. Keeping the client lean Mail Working Group 8. IDs and Passwords Too many... Spinoff Project 4 New Directions 4.1 IMAP Decision Confirmed Our preliminary investigations of the Internet-based e-mail field showed us that an IMAP4-based(5) service would likely be the best upgrade to the existing systems. Some excellent papers are available at the University of Washington's IMAP Information Centre(6) that outline the reasons why we should implement an IMAP4-based system at UWO. In brief, IMAP4 solves many of the problems associated with the earlier proprietary EMC2-based and POP3-based client/server e-mail models: o Messages are stored and retrieved by the client using an open, widely implemented protocol: IMAP4. o Mail and filed mail are accessible from anywhere since they are stored in a central database. o Mail and folders are reliably backed up. o Off-line mail with synchronization when the user next connects is supported. o Powerful filing and searching capabilities are built into the server. The IMAP4 approach is becoming universal in the industry with the Internet Engineering Task Force (IETF) close to finishing a common standard. Vendors from Netscape, Sun and IBM to Microsoft are now supporting, endorsing and implementing this standard for e-mail. The major push to make IMAP universal has just started (the first IMAP conference was held in January 1996 with attendees that included the who's-who of the e-mail world) so UWO will be well positioned to adopt this technology as we replace our old systems over the next year or two. We believe that an IMAP4-based service will meet the e-mail requirements for the most users for at least the next five years. In June we installed the University of Washington's IMAP4 server on the mail hub machine julian.uwo.ca for testing by the working group. ------------------------- 5. Internet Message Access Protocol 6. URL: http://www.washington.edu:1180/imap/ 5 Standards Groups o Internet Mail Consortium(7) is an industry group that includes many of the major e-mail players including Qualcomm (Eudora), Lotus, Sun Microsystems, NetManage (Z-Mail). This group is heavily involved in the evolving IMAP4 definition and are committed to supporting the protocol. They also sponsor periodic interoperability tests among competing mail systems. o Two members of the working group attended the IETF 36 and related INet'96 conferences at the end of June. The conference helped provide a feel for where the Internet community is going with the e-mail related directory services (Lightweight Directory Access Protocol (LDAP)) and with the long term standardization of the Application Configuration Access Protocol (ACAP) which will allow the implementation of a general and centralized user profile database. Both of these issues will significantly affect the campus mail system over the next few years. Our Peers o Since the Social Sciences Computing Lab is already running an IMAP service for their users, we interviewed the people responsible in that group to see what they're using and where they are going. The Lab is keen to work with us on finding a common solution. They're currently recommending PC-Pine (a simple IMAP4 client) to their users until ITS recommends a more sophisticated package for the administrative users. SSCL has done some preliminary work evaluating GUI-based mail clients. At the time that we interviewed them, the package from Siren Systems looked the best. o The good reviews for Siren were echoed by the University of North Carolina. They have recently chosen this package for their campus-wide mail service and have nothing but praise for Siren Systems. UNC intends to start rolling out the package to thousands of campus users in September 1996. o At the IETF/INet conference we talked extensively with the staff at UofT who are responsible for their mail system and got some good perspectives on what will work and what won't (and how rapidly the whole IMAP client area is evolving). Most other University sites are currently sitting tight with their current mail systems as they don't have a major upgrade pressing on them like our pending shutdown of the EMC2 system and the field is rapidly evolving. ------------------------- 7. URL: http://www.imc.org/ 6 4.2 IMAP4 Client Programs LAN vs Internet Mail We decided to put our primary efforts into finding an e-mail system that has grown out of the open Internet tradition rather than a LAN-based system that is gradually adding Internet capabilities. This has the advantage of fitting well into our existing mail infrastructure (i.e. the back-end) and into the Internet without the need of troublesome gateways. The disadvantage (and thus tradeoff) is that the LAN-based systems currently present a somewhat glossier application to the end-user --- the client programs are more mature. We have noted that the gap between the two types is shrinking as more fully featured Internet Mail applications appear. PeopleSoft For the PeopleSoft project, it is important that simple MAPI-based interface is provided by the client applications. All of the leading commercial products do provide the simple hooks required. POP3 We also expect that POP3-based clients (like the current freeware version of Eudora) will continue to be supported for quite some time in parallel with the IMAP4 service. Indeed the most popular IMAP4 server includes POP3 support. Some Promising IMAP Clients Using a list of client programs(8) supplied via the IMAP Information Centre at the University of Washington, we identified the major players in the market and will be concentrating on these. All of the software is currently under very active development so there is some risk in choosing any of them. Indeed, we expect the list to change during the next few months of our investigation. Our hope is that a clear winner will emerge before the end of the year. Siren Systems Siren(9) can use a IMAP4 server, but to take advantage of all of its features you have to use a IMAP server with extensions from Siren. They have promised to make their server IMAP4 compliant in the near future. From our preliminary trials it looks as if Siren has the best user interface so far (we haven't been able to test the X-window or MAC versions yet). This is the commercial system favoured by SSCL in their evaluation of mail clients. It has been chosen and heavily endorsed by the University of North Carolina as their campus-wide mail package. ------------------------- 8. URL: ftp://ftp.cac.washington.edu/imap/imap.software 9. URL: http://www.aeinc.com/siren/ 7 Siren includes MAPI support on the Windows platforms which is important for PeopleSoft compatibility. Solstice Internet Mail Sun's server and clients(10) are still very early on in their development --- production versions mightn't be ready for our deadlines. The product does provide MAPI support. It is not available for Macs. We are working with the local Sun office to try to get more information about the future of this product and to perhaps participate in a beta-testing program. Current indications are that Sun may be relying on Netscape to provide a client rather than further developing a client. They may be moving towards just supporting an industrial strength IMAP4 server. Pine The University of Washington's Pine(11) and PC-Pine programs are free but not available on the Mac platform --- Maildrop has been suggested as a free IMAP-based client for Macs. Pine does not provide a MAPI programming interface for PeopleSoft. Nevertheless it provides an excellent migration path for existing Unix terminal users since the learning curve for them would be very gentle. Moving people to the PC version of Pine would significantly off-load our central time-sharing systems while providing a better interface to our users. Most terminal users do use PCs, but they use the mainframe-based Pine rather than the POP3-based Eudora so that their folders are always accessible on the central system. PC-Pine provides this facility without requiring a ``dumb terminal'' connection while providing users with a quick and familiar e-mail environment. Simeon At first glance, Simeon(12) seems to have a very difficult user interface, but a predecessor is used extensively by UofT. Also this company seems to be in the forefront of IMAP4 and related protocol development. Perhaps with more familiarity we could learn to love the flexibility of the interface. Simeon presents a common look-and-feel on all platforms (MAC, Windows and Unix). This means, however, that it is native to none and because it was developed with an abstraction layer above the operating system, it tends to be slow on all platforms. ------------------------- 10. URL: http://www.sun.com/solstice/Networking- products/sims/index.html 11. URL: http://www.washington.edu:1180/pine/index.html 12. URL: http://www.esys.ca/ 8 Netscape Netscape(13) has been promising to include an IMAP4- based client in their v4 browser. They have said that they would deliver this product by the end of 1996. Currently version 3 of the product is still being beta tested so there has been no chance to test this (vaporware) product. Netscape recently hired the key author of Stanford's X-11 IMAP4 client, ML. Using Netscape as the mail client has some advantages in that versions of Netscape is available widely on campus already and it is well accepted. Judging by its current POP3-based mail capabilities Netscape might not have the specialized mail handling features of the other stand-alone programs. The question is ``Will that matter to the majority of our users?'' Z-Mail Pro NetManage has recently announced the next generation of the very sophisticated Z-Mail package, Z-Mail Pro(14). The new version now embraces IMAP4 enthusiastically. York University chose Z-mail (in its POP3 incarnation) as their campus mail system a few years ago. Its glossy user interface coupled with its cross-platform support (Windows 3.x Windows 95, Windows NT, Mac, X-window and dumb-terminal) make it a major contender (especially if we could get a copy of it!) 4.3 Planned Activities o Sun will be providing us with reference sites for their Solstice software and we'll follow up with them. o Follow up with York University as Z-mail users now that Z-mail Pro is about to be released with IMAP4 support included. o In early September we will be scheduling a meeting with the campus postmasters (with an invitation to the TUMs group) to inform them about the project, give them a chance to ask questions and to assure them that they can participate. o Short listing one or two packages and piloting them on a large subset of our supported platforms (Windows 3.x, Windows 95, Mac Solaris and possibly Windows NT). 5 Recommendations 1. That the timeframe for this project be adjusted to expect a final report for the end of 1996. This will allow the ------------------------- 13. URL: http://home.netscape.com/ 14. URL: http://www.netmanage.com/netmanage/cham50/zmail.html 9 Figure 1: Rough Mail Transition Timeline working group to do a more thorough investigation but more importantly allow the vendors of IMAP4 products a few more months to get their products to market. Since there isn't yet a clear winner and the users of the current systems tell us that a conversion at their pace in the summer of 1997 would be best, an early spring 1997 start for the transition to the new system seems feasible. Figure 1 illustrates via a time line the recommended sequence of events. 2. That a IMAP4-based mail system be adopted as the basis for the next generation mail system on campus with selection, implementation and user transition to occur over the next 18 months. 3. That Reg and Chuck continue their investigations into pulling information out of the EMC2 database with the eventual goal of feeding this information into the new system. We are confident that the hard part of this is getting the information out while getting it into one of these new systems will not be a major task and certainly not highly dependent on the IMAP4 client chosen. 4. That the non-e-mail facilities provided by EMC2 that are somewhat peripheral to this project be pursued by appropriate teams in cooperation with this working group while this group concentrates on e-mail and the facilities very closely related to that service. Spinoff summary: (a) Addressing and Directory Issues (b) BBS/Shared Folders (c) Calendaring/Scheduling (d) Forms 10 (e) FAX Gateway (f) IDs and Passwords 5. That e-mail should be considered as a major part of the VAX shutdown/transition. 11 A More Details on Related Issues and Projects A.1 Issues The following issues were identified by the group and reinforced by our interviews with EMC2 users. Some have been addressed already while others will solved during the following 6 months by the working group in cooperation with others at ITS. Still others will be left to the new groups mentioned below Addressing New mail addresses for EMC2 users o It was decided that the uwoadmin.uwo.ca name should be retired. o What should the name of the mail domain be? julian.uwo.ca, uwo.ca, maildrop.uwo.ca? o Which names need to be supported for EMC2 users? the ccscib@xxx.uwo.ca type or First.Last@xxx.uwo.ca or cbretzlaff@xxx.uwo.ca? Note that the First.Last@uwoadmin.uwo.ca style of address is not unique but seems to work quite well at larger institutions like UofT. The UWO Directory The use of uwoadmin.uwo.ca addresses for non-EMC2 users. Mailing Lists and Nicknames (a transition issue) o Some users may need help to move extensive local mailing lists to a new system. Reg and Chuck are looking into this issue. o Users are currently using private mailing lists as a workaround for the lack of nicknames in EMC2. Old Mail Folders (a transition issue) o Can old folders be moved to the new system? Reg and Chuck are looking into this issue. o There are some users that make good use of the mail foldering system as a filing system. We'll have to find automated ways to move these massive stores -- but perhaps for most we could ask them to weed their collection and convert or move only the important 12 messages. Keeping Existing Features Many existing users have expressed a need to maintain (in some form) some of the advanced features of the existing system: Departmental Mailboxes Shared spots for corporate mail --- or filtering/pre-sorting? Private Conferences EMC2 feature, not extensively used, could mailing lists handle this? Attachments standardized (MIME, UUENCODE), easy to use. Timed Mail Ability to have mail go out at a pre-determined time -- and for reminders. Could be handled by a scheduling/calendaring package. Auto Reply Vacation messages Important New Features Features that aren't currently available in EMC2. MAPI Support A minimal implementation the Microsoft's Mail Application Programming Interface is important for the workflow-enabled applications within the PeopleSoft suite. Spell Checker A good and personally extendable vocabulary is important. (We never purchased the spell-checking package for EMC2.) Keeping the Mail Application Small EMC2 is seen as a resource hog. A.2 Projects The following projects have been identified by the group as having a major impact on the mail replacement project but due to the mandate of the group, they should be considered as separate projects. Note that while some of these are already in progress, some are not but should be soon in order to permit an orderly move from the older mail systems. We would like to request that cooperating groups be established soon to investigate and make recommendations for these areas. Mail Delivery to Home Directory A performance issue. 13 o Delivery on all Unix systems changed from central mail spool to user's home directory o Project lead by Reg and Dave Kinchlea to be implemented over the next few months --- should be done before the end of August. Addressing and the UWO Directory See above for details. BBS/Shared Folders A current EMC2 feature o News-type facility where users mail to bulletin board (e.g. financial exchange rates) --- the Web may be the place for these. o Some of repositories are private (which isn't possible with Usenet News) o General purpose ones not currently used much. o Some listserv subscriptions are gated into the EMC2 system --- News is tool to handle this. o How many in total? o Look into converting these for users who rely on them extensively. Calendaring/Scheduling A current EMC2 feature o Not part of this project. o Debbie, Anne and Mike Cote have experimented with some standalone packages and chosen one that seems to be the best. o May be implemented this fall (Sept-Dec) so likely complete before e-mail transition (if we decide to make this a campus-wide project) Forms A current EMC2 feature o Used by finance message passing system. o Fees Office use forms and then FAX to Bank of Montreal. 14 o Currently there are 3 types of forms in use across campus --- EMC2 forms, Jet Forms and Web-based forms. FAX Gateway A current EMC2 feature o How much is this used? o Currently only handles outgoing FAXes but only for EMC2 users. o May come with the mail server software but certainly could be handled by many public domain or shareware products on many platforms. IDs and Passwords Too many... o Users have at least 4 possible passwords (LAN, corp data, e-mail, dial-in access) --- the new scheduling package (see above) may add another o Should they all be the same? Should they be coordinated? 15