Automatic Testing Basics
RAWI 980601


Goal: Test all of WaterGate's functions without user interference and
      generate a report showing which tests were not passed.
      The Normal, Overlay, DPMI and OS/2 versions must be tested
      separately.


This can be done by preparing ready test sets that are fed to WaterGate
one by one. A test set consists of a configuration, set of input files
and set of expected output files.

The automatic test package consists of
- one main batchfile that runs categories of tests
- one batch file per category
- a program that manages the log and error report
- a program that can prepare a configuration based on an ascii input file
- a program that can check the output for unexpected results
- a program that can test fields in certain user/area records

Each test consists of
- a batch file that performs the test and checks the results
- special programs to test specific issues

The test consist of the creating the start-point, creation of the
configuration, creation of required directories, the running of the
application, the verification of the output files and log, cleaning
up the expected output and checking for additional unexpected output
or left-over input files.

All the test can be run in their own environment by running COMMAND.COM
for each test. A substituted drive T: is used to create the test
environment.

The categories are

PKTIN   - PKT input: PKT2+ and older formats
PKTOUT  - PKT2+ output
P2KIN   - PKT2000 input: netmail, echomail and mixed format
P2KOUT  - PKT2000 output: netmail and echomail
UUCPIN  - UUCP in: mail and news
UUCPOUT - UUCP out: mail and news
SMTPIN  - SMTP input: mail
SMTPOUT - SMTP output: mail
BAG     - BAG input: mail, news, mixed mail and news
POP3    - POP3 input: mail
LS      - List Server functions
AFIX    - Areafix functions
NFIX    - Newsfix functions
FIXFWD  - Areafix/Newsfix forwarding
LISTS   - Mailing lists
MSG     - *.MSG: import, export, purge, link
SQUISH  - Squish: import, export, purge, link
JAM     - JAM: import, export, purge, link, renumber
WILDCAT - WildCat: import, export, purge, link, renumber
FD      - FrontDoor: inbound, outbound
BINK    - Binkley: inbound, outbound
DBRIDGE - d'Bridge: inbound, outbound
TUNNEL  - Tunnels: MailTunnel and SEAT, in and out

Initial tests (the first few months) should focus on the most complex
cases.

File structure:

\wg-test.<ver>                  <ver> = WaterGate versions
alltests.bat                    Main batch that runs all the tests.
                                Runs the test for all the applications.
                                Takes path the location of the applications:
                                alltests d:\source\wtrgate\bin\

apptest.bat                     Runs all the tests for a certain application.
                                Examples:
                                apptest d:\source\wtrgate\bin\wtrgated.exe

cattest.bat                     Special batch to run the all the tests of a
                                certain category. Takes executable path as
                                second argument.
                                Examples:
                                cattest pktin
                                cattest pktin d:\source\wtrgate\bin\wtrgateo.exe

catntest.bat                    Runs one test of one category.
                                Examples:
                                catntest pktin 001 c:\vp11\out\wtrgate.exe

 \bin                           Assist programs reside here

 \<cat>                         One sub-directory for each category
 c_<cat>.bat                    Batch for testing all of this category

 \<nnn>                         All tests are numbered with three digits
 test.bat                       The batch that runs this test



Environment variables

WTRGATE        Set for each test to point to system directory
EXEVER         Set to Normal, Overlay, DPMI or OS/2
