Embedded Terminal Server HOWTO

Appendix A - Troubleshooting

1. Examine messages with a tail -n 100 /var/log/messages. Look for dhcpd related problems.
2. Verify that a firewall is not interferring with the operation of openMosix or the DHCP peering commmunications.
3. Ping from the secondary to the primary and visa versa. Verify the network settings of the secondary are correct.
4. Switch back to /etc/dhcpd.conf.singleserver configuration and boot the servers one at a time. Make sure they both still work.
5. If the two dhcpd servers don't synchronize...
1. Make sure the system clocks on the primary and secondary are synchronized!
   
2. try deleting the existing lease files and restarting the boxes:

rm -rf /var/lib/dhcp/dhcpd.leases
touch /var/lib/dhcp/dhcpd.leases

We're not supposed to have to do this... but if all else fails, try a reboot. :o)

Appendix B - Optimization

Server Hardware Optimization

• Turn off sound card support and disable unnecessary parallel and serial ports in the BIOS.
• Disable USB support in the BIOS if not needed.
  
• Use PCI network cards in the server. Avoid all legacy ISA cards if possible.
• If you must use an ISA card, manually assign it's interrupt in the BIOS.
• If the hard drive and CDROM are sharing an IDE cable, put the CDROM on it's own cable and IDE controller before installing Linux.
• Set the BIOS power on mode to "Last State" or "Power On" so the box will reboot without intervention in the event of a power failure.
• Turn off "Spread Spectrum" BIOS support if your electro-magnetic environment or country's laws don't require it.
• Make sure that your network cards and switch/hub hardware are compatible. Some vendors don't auto-negotiate and co-exist well.
  

DHCP Load Balancing Optimization

DHCP load balancing is controlled by the "split" or "hba" parameter values in the primary's failover declaration. The split option's parameter sets a threshold for deciding when a request is handled by the primary or secondary. The hba option's bit map can be used to precisely set which server will handle the request. This technique isn't perfect, but it does afford a large degree of control over which workstations will be serviced by the primary and the secondary. There's more about this in RFC-3074 and the dhcpd.conf man page.

How it works

A terminal's dhcp request provides the client NIC card's mac address. The dhcpd server "hashes" the 48 bits of this address into a repeatable 8 bit value. The purpose of hashing is to scale the large mac address value and make appear somewhere in the range of 0 to 255. "Somewhere" is an important concept because this process essentially converts a mac address into an 8 bit pseudo-random number.

So after hashing, a given mac address will appear somewhere in a field of 0 to 255. If the hash algorithm is good, 50% of the requests will hash to values under 128 and 50% will be above 128... so "split 128" will produce a primary hit 50% of the time and "split 192" will produce a primary hit 75% of the time. A "split 255" will include ALL the mac addresses and produce a primary hit 100% of the time. Both servers have a copy of the split value and knows when to respond. That's how "split" works.

Unlike the threshold approach in split, the hba option gives you a way of mapping each request to a bit in a specified array. With hba, the 8 bit hash is used as an index into the hba (Hash Bucket Array) table. If there's a 1 at the index, the primary server is used. If there's a zero, the secondary will pick up the request. Both servers have a copy of this table so they know which one should respond.

The hba table is useful where the terminal macs don't hash "nicely" into an even distribution across 0 to 255. For example, it's possible, but not likely, that a majority of your macs will hash into the range of 129 to 255... so a split 128 will still see all your requests routed to the secondary. Most of the time this won't happen... but if it does, the hba option gives you a way to smooth out the differences.

If you end up with an imbalance, you can resolve it with a little trial and error table substitution. Remember that dhcd.conf is the file you'll need to edit... and when you make a change, you'll need to restart DHCPd on both servers to get them to initialize and grab an updated copy of the table... then boot up your terminals and see if you like the new load distribution. Here are a few examples:

Split 50/50:

hba ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff: # same as split 128
    00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00; # 128 bits on, 128 bits off

hba ff:00:ff:00:ff:00:ff:00:ff:00:ff:00:ff:00:ff:00: # 8 bits on, 8 bits off
    00:ff:00:ff:00:ff:00:ff:00:ff:00:ff:00:ff:00:ff;

hba aa:aa:aa:aa:aa:aa:aa:aa:aa:aa:aa:aa:aa:aa:aa:aa: # 4 bits on per byte
    aa:aa:aa:aa:aa:aa:aa:aa:aa:aa:aa:aa:aa:aa:aa:aa; # 10101010

Split 75/25 (75% for the primary)

hba ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff: # same as split 192
    ff:ff:ff:ff:ff:ff:ff:ff:00:00:00:00:00:00:00:00;

hba e7:e7:e7:e7:e7:e7:e7:e7:e7:e7:e7:e7:e7:e7:e7:e7: # 6 bits on per byte
    e7:e7:e7:e7:e7:e7:e7:e7:e7:e7:e7:e7:e7:e7:e7:e7; # 11100111

hba bb:bb:bb:bb:bb:bb:bb:bb:bb:bb:bb:bb:bb:bb:bb:bb: # 6 bits on per byte
    bb:bb:bb:bb:bb:bb:bb:bb:bb:bb:bb:bb:bb:bb:bb:bb; # 10111011

Split 25/75 (25% for the primary)

hba ff:ff:ff:ff:ff:ff:ff:ff:00:00:00:00:00:00:00:00: # same as split 64
    00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00;

hba 24:24:24:24:24:24:24:24:24:24:24:24:24:24:24:24  # 6 bits off per byte
    24:24:24:24:24:24:24:24:24:24:24:24:24:24:24:24; # 00100100

hba 42:42:42:42:42:42:42:42:42:42:42:42:42:42:42:42: # 6 bits off per byte
    42:42:42:42:42:42:42:42:42:42:42:42:42:42:42:42; # 01000010

Get the idea?

These examples are for fine tuning and extreme situations. The hash is pretty good and most of the time, you won't need to bother with any of this... just set a split 128 and you'll probably be content with the load balancing. The finer grain adjustments can come in handy if you're dealing with servers that aren't the same speed or have different roles... in which case you'll want most of the load to fall on the faster and more capable machine. These techniques give you a way to make each server work as hard as you want it to... and no harder.

openMosix Optimization

Appendix C - Backup and Disaster Recovery

A cheap, simple and expeditious backup strategy is to store all the unique files for distinguishing a primary and secondary server (during step 4 and 5 of the installation process) to a floppy (or other backup media) and make a duplicate of the primary hard drive . If a primary server fails, this drive can be used to get it back online immediately. If a secondary fails, the backup drive can be installed and the secondary configuration information from the floppy can be used to set it up for secondary operation. This floppy will include all network settings for the primary and secondary and any configuration changes made during the original installation that distinguish them. For LTSP this will include:

/etc/ltsp/i386/etc/lts.conf
/etc/dhcpd.conf

It's possible to create the backup floppy after the install but much easier to do it as you edit the files during the forking of the primary and secondary configuration.

There is no provision in this procedure for user data that has been stored on the ETS. If a network file server with a suitable backup strategy is available, one approach is to force terminal users to store their data there. If user data is stored on the ETS, you'll have to concern yourself with keeping the home directories on the two servers in sync in addition to backing up the data.

Appendix D - Security Considerations

Here are a few ideas for securing your cluster servers. This is NOT a complete solution and NOT guaranteed to make your system secure. It's just a few things I've done to make it less easy for people to get into the box and make unauthorized changes. There is nothing better then having the servers locked up and only available to authorized individuals.... but if the hardware must be physically exposed, here are a few suggestions:

Physical Security

1. Set an adiministrative password for the BIOS on each machine.

2. Set the BIOS boot order as C, CD, A. 

3. Edit /etc/lilo.conf on both servers and restrict kernel parameters from being passed during the boot process:

etc...
vga=normal
default=linux
restricted # Add
password=yourpassword # Add
keytable=/boot/us.klt
lba32
prompt
warn
... etc

4. After editing lilo.conf, run lilo to install the changes.
5. If possible, make the case more difficult to open. Getting inside and shorting the CMOS battery will reset the BIOS defaults.

System Security

1. Install firewall rules to control traffic into and out of the server's "outside" interface. (eth0 in the example)
2. Never place any ETS interface directly on the internet. ETS systems should always be behind a secure and well maintained firewall.
3. If the internal LTSP or openMosix networks aren't physically secure, don't deploy until they are. Add firewall rules as appropriate.
4. Configure the firewall to block internal LTSP and OM services to the outside interface (eth0 in the example).
5. Any exposed service is a potential risk but SSH and WebMin can be quite useful for remote administration.

Appendix E - Firewall Issues

A firewall was incorporated into the ETS to provide control over workstation access to the internet and other resources on the trusted internal network. The Shorewall firewall was chosen because of it's outstanding capabilities, ease of configuration and excellent documentation. Below are sample configuration files from a working ETS that define the firewall's interfaces and zones:

/etc/shorewall/interfaces
/etc/shorewall/zones

The default policy is to completely block the servers (and therefore workstations) from accessing the internal network. The policy also allows any traffic from the workstations to the server and the server to the workstations. Below is the sample configuration file that implements this policy.

/etc/shorewall/policy

Rules can override the default policy with specific IP's, ports and protocols. The rules in the following example provide the box with NTP, Samba file and printer sharing, DNS, SSH, VNC and WebMin services from the internal network. Domain authentication service also requires that ports 135, 139 and 1024:65535 be open for tcp traffic and 137, 138 and 1024:65535 be open for udp... which is a ridiculous range. I decided to simply open the server up to the Primary Domain Controller (PDC in the sample file) since it is a "trusted" machine on the network. The Backup Domain Server (BDC) should also be included but isn't in the example provided. Below is a sample configuration file that implements these rules.

/etc/shorewall/rules

The easiest way to determine if you've got a firewall issue with a service is to simply turn off the firewall and pass everything through. With Shorewall, this is easily accomplished from the command line.

shorewall clear

If the service starts working after you've done this, you'll have to fix your rules... then try again with:

shorewall restart

Here are some other handy shorewall commands for troubleshooting and determining status:

shorewall check              # validates your ruleset... handy after you've made changes
shorewall status    # gives a detailed report about what's been going on with the firewall... very informative!
shorewall monitor 5   # shows what the fw is doing in real time... example toggles through screens every 5 seconds.

If you're logged in remotely via SSH, don't do a:

shorewall stop

... or it will be the last command you'll type in that session! :o)

Appendix F - Enterprise Integration

The ETS is a central platform for launching a variety of applications in an institutional or enterprise environment. Most often, the required applications will include, web browsing, email, word processing. In order to centrally manage user logins, NT domain authentication is often required along with Samba printer and file sharing.

In order for the ETS to interoperate with existing Windows NT and 2000 environments, domain authentication of user logins is often necessary. Since this can be a "challenge" to get working, here are the basic steps and principle configuration files from a currently running ETS installation. The examples are for Samba 2.2.3a running on a Mandrake 8.2 system with an NT 4 domain controller. LTSP is running gdm for the display manager.

Cautionary note: Before making any changes to any Linux authentication systems, make sure you have a rescue disk ready in case you break something and can't log back in! Ok, with that said, let's proceed:

1. Install Samba (SMB and NMB) and Winbind.
2. Verify that these three services will be started at boot.
3. The following are the main winbind configuration files. Use these for examples and backup originals before making mods! :o)

/etc/nsswitch.conf          <-- winbind is added in passwd, shadow and group.
/etc/samba/smb.conf       <-- modify "workgroup" for your domain name. Tune other items as needed.
/etc/pam.d/system-auth  <-- comes with winbind as system-auth-winbind... copied to system-auth without changes.
/etc/pam.d/gdm               <-- this file should't need modification... shows the way gdm references system-auth.

4. Boot order is important: SMB and NMB must start before winbind. (probably not a problem for most distros)
5. You can manually start the services, but I'd reboot just to be on the safe side: verifies service startup and boot order.
6. After the boot, verify that SMB, NMB and winbind are all running.
7. Join the domain that you've configured /etc/samba/smb.conf for by using the command:

smbpasswd -j <DOMAIN> -r <DOMAIN_CONTROLLER> -U <DOMAIN_ADMIN>

8. You'll be prompted for your domain admin password.  Type it in!
9. You'll get a message that your machine name has joined <DOMAIN> if the operation was successful.
10. If you joined successfully, try the following command:

wbinfo -t

11. If you get a "Secret is good", you're ready to authenticate users.
      Try the following command on a known userid:

wbinfo -a MYDOMAIN\\myuserid%mypassword

12. If everything is working ok, you'll get that the plaintext authentication was ok and encrypted authentication failed:

plaintext password authentication succeeded
challenge/response password authentication failed
Could not authenticate user MYDOMAIN\\myuserid%mypassword with challenge/response

13. If your configuration isn't working or the userid%password is bad, you'll get the following:

plaintext password authentication failed
Could not authenticate user MYDOMAIN\\myuserid%mypassword with plaintext password
challenge/response password authentication failed
Could not authenticate user MYDOMAIN\\myuserid%mypassword with challenge/respons

14. If you want encrypted (NTLM) authentication, you'll have to re-compile Samba and configure for NTLM.
15. If you have a trusted internal network, you may choose to live with plaintext authentication... your choice.
16. Time to try logging in with LTSP and gdm. The format is a little different for graphical logins: DOMAIN\userid
17. On successful login, /etc/pam.d/system-auth's session library mk_homedir will create the following home directory:

/home/DOMAIN/userid

18. /etc/skel will also be copied to the users dynamically created home directory.

1. If you aren't able to join the domain or there's no response from your domain server, you may have a firewall issue.
2. If you can't join the domain, your login may not have sufficient NT rights to "add workstations to a domain".
3. If you can't join the domain even if everything looks right, delete the files:

        /etc/samba/MACHINE.SID and /etc/samba/secrets.tdb , restart NMB and winbind... then try re-joining.

4. Don't forget to restart SMB and winbind after making smb.conf or other samba related configuration changes.
5. running wbinfo -t and getting "Secret is good" is the best indicator that you've successfully joined the domain.
6. If your "Secret is good", try the following commands to verify access to the domain:

wbinfo -a  domain\\userid%password         # authenticate a user
wbinfo -m          #will list all the trusted domains that can be reached from your machine.
wbinfo -u           #will list all the users of the domain you've joined.
getent passwd  #will list the contents of the passwd file... which will include domain users.
getent group     #will list all the groups from the local machine and the domain.

7. Make sure that SMB and NMB start before winbind in the rc.d-SysV start ordering.
8. Check and re-check your /etc/samba/smb.conf file... it's the key!
9. If it's inconvenient to test with the actual domain controller, set up a Samba PDC for testing.

1. The wbinfo utility wants a specific format for authenticating users: wbinfo -a domain\\userid%password
2. Graphical logins like gdm want domain\userid
3. One slash or two... watch your context!  Success or failure depends on it!
4. Sometimes "things" get confused between Samba and the PDC. If everything looks right but it's not working, try a reboot.
5. In the next version of Samba/winbind, everything in this procedure could be completely wrong!
    Such is life in the wonderful and dynamic world of Linux!

Configuring the ETS for Enterprise Applications

The following section is under development. More information will be included in the next document release.

Web Browsing

While Mozilla 1.0.1 is currently the browser of choice for the ETS, the Phoenix Project provides a lightweight, "lean and mean" version of Mozilla for embedded applications where reduced footprint and high performance are critical. Phoenix is just a browser... no email, IRC or news clients are included, so it is ideal for ETS installs where the server has limited memory and/or disk space. The Phoenix browser also has a set of concise and attractive themes that can be useful for customizing the overall appearance of an ETS.

The default install limits Mozilla and Phoenix to a single user instance. When you try to bring up a second instance, you end up in the Profile Manger. On a single user system, this can be pretty annoying. And while the wisdom of this is hotly debated, occasionally it's useful to allow the same terminal login to bring up a browser on different workstations without tripping over the profile manager. Someone actually submitted bug 122698 to bugzilla.mozilla.org and a revised startup script was posted by the Mozilla team. This script also happens to work with version 0.4 of Phoenix as well... just rename the original phoenix startup script to phoenix.orig and name the script in it's place.

File and Printer Sharing

While all Samba share configuration is done via /etc/samba/smb.conf, Webmin and the ETS Administration System provide a much easier and safer way to manage this subsystem. The Samba SWAT utility should be installed (comes with Samba) to provide the greatest linked functionality from Webmin.

Email

One of the most comprehensive and feature laden e-mail clients for Linux is Evolution from Ximian. It is highly recommended for enterprise environments. The general release of Mozilla also provides mail and news clients that are good for less demanding applications.

Word Processing

The most widely used and comprehensive and cost free office suite is OpenOffice. For a little money,  you can purchase a supported version of Sun's StarOffice. There are small differences but in general, StarOffice and OpenOffice are functionally identical. Sun provides generous discounts for educational institutions and excellent documentation. OpenOffice is currently provided with the Mandrake 9.0 and Red Hat 8.0 distributions.

There are many configuration issues with OpenOffice and StarOffice that are of interest to ETS users. Information regarding optimal setups for these applications on the ETS are currently being acquired and will be included in future releases of this document.

Wine Integration - Running Legacy Windows Applications under Linux

The most popular way to run legacy Windows applications under Linux is to use Wine. There are a number of different "flavors" of wine which have all been derived from the original Wine Project. Each variation has chosen a different application area to focus on and refine. Some are free.. others aren't. Be forewarned, that making Windows applications run under Wine (which is still in an "alpha" state) isn't an easy task. Windows is a single user platform so running Windows applications in a multi-user LTSP environment will require special considerations and great care.

The Wine Project maintains the base source distribution and provides links to pre-built binaries for selected platforms and distributions. In addition to being a major contributor to the free version of Wine, Codeweavers sell an excellent proprietary version called CrossOver Office that is capable of running Microsoft Office 97 and 2000, Visio, Lotus Notes, Quicken and a few others. Codeweavers is an outstanding company and their products are well supported and priced economically.

Transgaming has created another proprietary Wine flavor called WineX which is oriented toward running games and real time graphical programs and works quite well. The company charges for their binary packages, but you can download the cvs sources and use this installation procedure, a header and this file to get it running on your system. Be sure to verify that you'll be in compliance with Transgaming's license if you decide to use WineX.

Since the goal of the ETS project is a freely distributable package, we're going to use the free version of Wine from the Wine Project for the configuration and installation example that follows. Since Wine is very I/O, hardware and machine dependant, it's usually a good idea to compile it on the specific ETS server where it's going to be running. An example installation procedure is as follows:

Installing Wine

1. Log in as a non-root user.
2. Pick a mirror and download the most current source release (currently Wine-20021031.tar.gz ~7 Mb) to your user's home directory... then make your home the current directory and uncompress the archive by typing:
3. cd ~
4. tar -xvzf Wine-20021031.tar.gz
5. This will create the entire source tree (~35-40 Mb) under a subdirectory named Wine-20021031. Make this the current directory and start the compile/install process by typing:
6. cd Wine-20021031
7. ./tools/wineinstall

After a brief configuration, the wineinstall script will ask if you want to compile wine and install it as the root user... reply with a yes. The rest of the build process will take a *LONG* time... especially on the low end machines. The build will deposit 400+ megabytes of "stuff" in the Wine-20021031 subdirectory... so be sure you have at least this much space available before you begin.

8. After the build completes, the script will ask you for your root password so the binaries can be installed. Type it in.

Next, you'll be asked a few questions. Here are the answers:

9. yes to a Wine-only install.
10. Accept the suggested user based directory for the "fake C drive"

That's it... the Wine binaries are now installed. Type wine from the command prompt and you should see the "Wine 20021031" in the resulting usage prompt

Configuring Wine

The installation process creates a hidden subdirectory in your home called .wine. We'll be doing virtually all of our work with the applications in this area so let's make it the current directory and display its contents by typing:

cd ~/.wine
ls

The most important file in the ~/.wine subdirectory is config... open it with your favorite text editor and read the header. The purpose of this file is to map Windows to Unix resources. You'll see Windows drive letters to Unix paths, fonts, multi-media drivers, serial com ports, registry keys... virtually everything that a Windows program will request can and must be properly mapped to a corresponding Unix resource.

Some of the default values for drive letter to Unix paths are wrong for Mandrake 8.2 and must be fixed before we can verify our installation. Adjust your config file to match the following:

[Drive A]
"Path" = "/mnt/floppy"

[Drive D]
"Path" = "/mnt/cdrom"

Also, comment out the Drive F entry with semicolons:

;[Drive F]
;"Path" = "${HOME}"
;"Type" = "network"
;"Label" = "Home"
;"Filesystem" = "win95"

Verifying your Wine installation

Run the following commands (as a non-root user) to verify the Wine installation:

cd ~/Wine-20021031
./tools/winecheck | more

Fix whatever the script complains about. The latest version of winecheck and a how-to can be found here

Fine Tuning your Wine Installation

1. Windows programs are generally designed for a single user environment. If you want to run one in a multi-user environment, you may need to make an install for each user.
2. ... more later

Appendix G - Self Test Procedures

An automated Self Test and Notification System (STANS) for verifying that the ETS is fully operational is planned but not implemented at this time. The mon utility will handle some of this but the higher level diagnostics will include::

1. Verification that the servers are within operational limits (RAM, disk space, CPU load, etc) and performs a general health check.
2. Verification of the LTSP server status and configuration.
3. Verification of openMosix and check that all mapped nodes are present and operational.
4. Verification that Samba is running and options such as Winbind, file and printer mapping are operational.

Appendix H - Installing openMosix

Installing an openMosix binary package is a simple and quick procedure. The specifics presented here assume an RPM based system using lilo for the boot manager:

1. Download the desired RPM kernel and userland tools from the openMosix site on SourceForge.
2. Install the two packages with the following commands: (version numbers are examples)

rpm -i openmosix-kernel-2.4.19-openmosix4.i686.rpm
rpm -i openmosix-tools-0.1.3-6.i386.rpm

3. Edit /etc/lilo.conf. The original should look something like this.
4. Add an entry for the openMosix kernel. The additions should look like this when you're finished.
5. Run lilo to install the new configuration.
6. Edit the /etc/mosix.map file and add the other nodes in your openMosix cluster.
7. Verify that the openmosix service will be active at boot time.
8. Reboot.
9. Verify that the other nodes in your cluster are visible by running the mosmon utility.

If you're having trouble seeing your other nodes, verify that you don't have a firewall issue. I cannot find any documentation on the ports and protocols openMosix uses for inter-node communication so I can't specify them. Fortunately, the installations I've made so far haven't required the openMosix interface to be restricted. For runtime configuring and programming, see the openMosixAPI.

Appendix I - Fail-over Procedures

An ETS hard fail-over occurs when the primary or secondary looses contact with it's DHCP peer. If a server hardware failure is responsible, any client sessions running on the failed machine will be lost. Rebooting the affected workstation will re-establish a session with the remaining server via the DHCP fail-over mechanism.

It is sometimes necessary to select a fail-over condition in order to perform maintenance on the primary or secondary servers. Unfortunately there is no way to do this that is transparent to the users on the server that is targetted for shutdown. <<<*caution:Probably the least disruptive way to take an ETS peer out of service is to place the DHCP server on the other ETS into a "partner-down" state near the end of a work shift. The machine in the "partner-down" state will assume the full load for all future DHCP requests. At this point, notify all the users that are currently running on the machine that needs to be removed from service and tell them to reboot at the end of their shift. The other ETS will pick up the new session requests and run the workstations during the planned outage. The process can be reversed (move operational ETS from "partner-down" to "normal") at the end of the next work shift in order to place both machines back online and restore them to their normal roles.>>>

*caution* This is an untested procedure that is currently in the development stages.

Appendix J - Duplicating the ETS Systems

1. Duplicate the primary and secondary hard drives from the operational system.
2. Install the duplicate drives in the new servers. Don't plug them into the network. (avoids IP conflict with originals)
3. Boot the primary and reconfigure all required files for its new IP.
4. Shut down, plug the machine into the network and reboot.
5. Use dhcpd.conf.singleserver to get the machine running with the new IP addresses as a stand alone LTSP server.
6. Migrate the changes from dhcpd.conf.singleserver to dhcpd.conf.
7. Repeat steps 3 through 6 for the secondary.
8. Perform the ETS integration (step 5) with both boxes.

Appendix K - System Administration

For administering the ETS, the browser based Webmin was chosen for it's well designed and flexible architecture, comprehensive configuration coverage, wide third part support and ease of use. It's also a secure system that utilizes ssl for creating a secure channel and certificate verified links between the administrator and the ETS. While the documentation for Webmin is extensive, it's a largely self-explanatory system with a relatively short learning curve.

To install Webmin, download the package from here or here. The instructions are here but it's a simple matter to get it installed.  On an RPM based system, the command is: rpm -i webmin-1.030-1.noarch.rpm. After the installation, bring up a local browser and start configuring Webmin and interacting with the local ETS host by typing the url:

http://localhost:10000

If Webmin has been configured to use ssl, the url will be:

https://localhost:10000

If Webmin hasn't been configured to use ssl, set it to do so... link security is free! You may also want to consider generating a server certificate for each server you administer to increase security and confidence during remote logins. To log in remotely, just replace localhost in the example with the IP or name of the remote ETS:

https://10.104.75.9:10000

While Webmin uses SSL, don't place any ETS interface directly on the Internet. Internet access from an ETS should always be done via a well maintained and very secure external firewall.

OpenSSH is another invaluable remote admin tool that works well with and compliments Webmin.  In situations where you'll need to maintain ETS machines on the other side of a firewall, you can maintain a portal to portal security envelope by using SSH to port forward through the firewall to the internal ETS machine... then using the https connection to Webmin on the ETS host. For example, typing in the following command will set port 8080 on your local machine to listen, open a secure connection to remotefirewall and tell it to forward any traffic from port 8080 on the originating machine to port 10000 on the ETS at the internal IP address 10.104.75.9.

SSH -L 8080:10.104.75.9:10000 myuserid@SomeRemoteFirewallWithSSH.com

After you've established this connection, you can bring up a browser on your local machine and access Webmin on the remote ETS with the following URL:

https://localhost:8080

If you're a Linux expert or beginner, an excellent overall reference for administering Linux systems can be found here. The K12Linux Project also provides some excellent documentation for setting up and administering Linux systems.