opus:interactive | If These Servers Could Talk

The Goal:

We needed a way to protect a set of load balanced web servers that host public facing web sites for a client. The web servers each run a copy of the hosting control panel DirectAdmin on top of FreeBSD 6.2 Release.

The Problem:

DirectAdmin licensing requires a real public IP address on the server’s external interface to protect against using the license multiple times by hiding it on a private network. Many different flavors of control panel software have this requirement.

This, of course, eliminates the ability to take the easy way out and NAT the servers behind a firewall.

We would have simply purchased a firewall appliance for this installation (we implement GB-2000 firewall appliances by GTA Inc. however an additional requirement was to use existing 1U server hardware and not purchase an additional appliance.

Solution:

The end solution was to install FreeBSD 6.2 Release on the 1U server hardware and utilize packet filter (pf) as a transparent bridge to meet the IP addressing requirements.

Hardware load balancing was used to load balance HTTP traffic to the web servers but will not be discussed here.

Howto:

1.) Install FreeBSD 6.2.

We have a checklist list of tasks to perform to install and lock down our production servers. Follow your own best practices to get a basic install of FreeBSD 6.2 running and patched. Install the minimal amount of options and packages necessary.

You will need, or at least you will most likely want, a third NIC installed in the server. In a transparent bridge the WAN and LAN interfaces become “transparent” and no longer take an IP address. So without the third NIC installed and connected to your network you will have no way to remotely manage the server. A benefit of this though is that without an IP address to attack your transparent bridging firewall itself would be free from attack.

Pf is available in a default install by re-compiling the kernel with specific changes made, or by enabling pf via kernel loadable module.

We re-compiled the kernel. The options below were added at the end of the kernel source and the new kernel compiled:

# pf support
device pf # Packet Filter firewall
device pflog # PF logging facility
device pfsync # PF state syncing

# ALTQ support
options ALTQ
options ALTQ_CBQ
options ALTQ_RED
options ALTQ_RIO
options ALTQ_HFSC
options ALTQ_PRIQ

2.) Configure your third NIC with an IP address and verify you can remotely access your server.

In the /etc/rc.conf file we have the following definition for the management IF:

ifconfig_fxp0=”inet 123.123.123.2 netmask 255.255.255.0″

We will be building pf rules for this NIC as well to protect the firewall itself.

3.) Create the bridge between the two desired interfaces.

Use your favorite editor to edit /etc/rc.conf and enable the bridge

Add:

cloned_interfaces=”bridge0″
ifconfig_bridge0=”addm bge0 addm nfe0 up”
ifconfig_bge0=”up”
ifconfig_nfe0=”up”

In this case we are bridging the two interfaces bge0 and nfe0.

Use your favorite editor to edit /etc/sysctl.conf

Add:

net.link.bridge.pfil_onlyip=1
net.link.bridge.pfil_member=1
net.link.bridge.pfil_bridge=0

4.) Enable the use of pf on your server.

Use your favorite editor to edit /etc/rc.conf and enable the use of pf

Add:

pf_enable=”YES” # enable PF (load module if required)
pf_rules=”/etc/pf-bridge.conf” # rules definition file for pf
pf_flags=”" # additional flags for pfctl startup
pflog_enable=”YES” # start pflogd(8)
pflog_logfile=”/var/log/pflog” # where pflogd should store the logfile
pflog_flags=”" # additional flags for pflogd startup

5.) Build the firewall ruleset.

First make a copy of the default ruleset and designate it as a bridging ruleset.

# cp /etc/pf.conf /etc/pf-bridge.conf

Use your favorite editor to edit /etc/pf-bridge.conf. Place your ruleset within the pf-bridge.conf file and save the changes.

Here is the sample ruleset we used: pf-bridge_generic.txt

6.) Apply the rules and enable the firewall.

Finally to actually enable a new ruleset we need to tell pf to read the config file. This would also automatically happen upon reboot.

# pfctl –f /etc/pf-bridge.conf
# pfctl –e

That’s it! You will now need to go through and test the bridge and verify you can access what you intended to allow access to, and that what you wanted to block is now blocked. Hopefully you still have access to the management interface as well. The best test will be to perform some form of vulnerability testing against IPs behind your firewall and the firewall itself.

Some notes on the ruleset specifically:

Hurdles:

The first major hurdle we ran into had to do with Multiple MAC address tables on the switch (or lack thereof). We wanted to use a single switch to handle the connectivity for both the inside (LAN) and outside (WAN) of my transparent bridge. To do this we created two VLANs on the switch so that the WAN IF of the bridge connected to VLAN 1 and the LAN IF and the web servers connected to VLAN 2.

The problem is that because of the bridging, the MAC addresses of the web server network adapters were now appearing in both VLANs on the switch thoroughly confusing it.

It turns out that the switch we were using – an older HP Procurve – only had a single MAC address table.

The interim solution was to use two switches, one for each side of the transparent bridge. A better solution will be to use a switch that has more than one MAC address table so that we can use only a single switch for this solution.

The second hurdle is really only the fact that it can take time to perfect the pf ruleset. There are so many options and more than one way to do the same thing. Keep working till it works out correctly.

Some additional commands for managing pf:

# pfctl -s rules : list current parsed rules
# pfctl -f filename : reload the ruleset with the specified file
# pfctl -d : disable pf
# pfctl -e : enable pf
# pfctl -R /etc/pf.conf : enable rules from specified file
# pfctl -s rules -v : hit stats for each rule

View current log with TCPDump:

Log specific tcp packets to a different log file with a large snaplen
(useful with a log-all rule to dump complete sessions)

# pflogd -s 1600 -f suspicious.log port 80 and host evilhost

Display binary logs:
# tcpdump -n -e -ttt -v -r /var/log/pflog

Display the logs in real time (this does not interfere with the operationof pflogd):

# tcpdump -nexttti pflog0

Here are some key points to look for when shopping around for a provider.

If you explore the Internet, send or receive e-mails, or save files and programs to your computer, you run the risk of contracting a virus. And today, viruses do more than just destroy files and cripple computers. They gather confidential information and share it with criminals, and let uninvited guests commandeer your computer from remote locations. Protecting your system is vital if you value your computer, credit and privacy.

Viruses, worms and Trojan horses

Viruses come in three common strains: virus, worm and Trojan horse. A virus spreads through human interaction, often as an e-mail attachment. A worm, on the other hand, spreads independently by sending itself (in your or another’s name) to e-mail addresses found on your computer. The Trojan horse (also known as a “backdoor virus”) is typically downloaded by an unwitting computer user. Once the Trojan is installed, it searches for private information—including files, passwords, logins and credit card numbers—and ships it out a “back door” to a shadowy recipient.

Keeping Viruses at Bay

There’s no single solution to thwarting viruses. Rather, the solution lies in a matrix of protection. Because viruses are continuously updated to breach the day’s best security systems, nothing can guarantee absolute safety. That said, the following steps—used together—can help you stay virus-free.

Insist that your Internet service provider (ISP) offers spam and virus filtering of all e-mails. Good ISPs stay abreast of the latest trends, employing sophisticated content filtering to help prevent e-mail-borne viruses and worms from reaching your computer.

Don’t open an e-mail attachment unless you know what it is. When in doubt, call the sender to verify the attached file.

Update your computer operating system with the latest version. Viruses often exploit security holes in computer operating systems. The older your operating system, the greater your vulnerability.

Install and run antiviral software on your computer. These programs scan your system and quarantine potentially harmful files. Many of these programs automatically update via the Internet to protect users from the latest viruses. Leading products include Norton AntiVirus Corporate Edition (for business), Trend Micro PC-cillin, and AVG Anti Virus.

Install a firewall. Firewalls block viruses and prevent unauthorized access to your computer via the Internet. Firewalls are available as software (for protecting individual computers) and hardware (for standing guard in front of a network of computers). Some ISPs offer DSL routers with built-in firewalls, eliminating the need for purchasing additional software. If you subscribe to DSL, check with your ISP to see if your router doubles as a firewall.

Avoid “back alley” websites. Sites that feature “questionable” material are notorious for hosting viruses. If the site isn’t maintained by a reputable company or individual, go elsewhere.

Beware of “free” downloads. As with “back alley” sites, if a free download doesn’t come from a reputable company or individual, don’t click through and download it.

If you have further questions, or if you think your computer is infected with a virus, don’t hesitate to call your ISP. If your provider can’t help, it should happily point you toward a knowledgeable technician who can.

Most Internet shopping experiences are good. But some turn sour. Typically, these bad experiences involve a disagreement over price, product or delivery time—and sometimes they’re outright scams with little chance of resolution. The good news is that bad experiences aren’t random. They happen, predictably, to unvigilant shoppers. Common sense prevails. By exercising good judgment, the guidelines below will help you shop online with safety and confidence.