Institutt for Informatikk - Master Thesis - VoIP firewalling

Hmm ... looks like my dear fellow master students were even worse than me when it comes to updating their blogs. But hey, who cares about that anyway? Did I mention that the blog wave will vanish sooner than later - people just don't care enough anymore after some time. Like me.

So, this is the latest entry of my master desaster blog. The final theme is

Generating Synthetic VoIP Traffic for Testing Redundant OpenBSD-firewalls

and will be handed in tomorrow. After the final retouches (proofreading, fixing some poor-quality graphs etc.)

That's all folks - thanks for the attention! The oral presentation is on June 12th/13th.

So finally both machines are properly patched (but without the real-time interrupt patch, the homepage was unaccessible). Compiled libpcap-0.9.4, then tcpdump (installed in /usr/local), then plab, now working on Coral Reef (strictly speaking this would not need to be compiled with PF_RING since I will be analyzing dumps).

Very important hint: the sequence of the libraries is important - LIBS must first have -lpcap and then -lpfring, otherwise compilation errors occur.

Working on automating the tests for having reproducible results. This includes

- the script for generating traffic (pktgen, eventually yate)
- analyzing the tcpdumps

A short test with the following params
- no carp (using only fw1)
- pfctl disabled
- generating 89478 packets with size 1500 (error: should be 12000 Bits!)
show that pcount (with PF_RING) sees all packets that come in, but about 20,000 packets get lost in the firewall (double-checked with the switch).

May be on the lead for an upper limit of packets that can be processed ... and have not turned on pfctl yet ...

Questions: right packet size (VoIP?), and darn those counters from the switch are hard to get! (manual so far ...)

Mid-term presentation. Flu. Then - finally new firewalls, we have given up the old ones. HP ProLiant DL 140 with 3.6 GHz Intel Xeon - good luck with loading them ...

Examined the Linux pktgen kernel module for generating random UDP packets that could simulate RTP traffic. Enabling PF_RING on the receiver (waited for a patch from Deri/ntop). Wrote some script for logging usage (OpenBSD: kern.cp_time, Linux: Fabian's cpusage). Started on Perl for analyzing the dumped artificial traffic (checking sequence number etc.).

Thinking about how to query the switch for packet count...

And Knut's generosity for hinting me about yate, could be easy enough to use yate for generating real traffic - would be interesting to compare with the artificial model ...

Stay tuned.

What happened during the past week?

1. (Thu) meeting on how to fix the hw-problems, (Fri) moved up all equipment to FreeCode's new offices at Forskingsparken, got anSMC Networks SMC8648T Gigabit switch. Oh no, it uses Cisco CLI standards...


2. (Weekend) chill (not really, but pretending)


3. (Man) installed firwall 2 (missing PCI card, 2 Gigabit NICs, tested with firewall1's RAM - works, yet crashes when Linksys NICs are used)


4. (Tue) Last Supercomputing Day - yeahh! Actually a little sad, the course is cool.


5. (Wed-Fri) setting up stuff, looking into ntop and other tools to analyze traffic, etc.

A warm and hearty welcome to a new week! Fresh and with new ideas (ahm ... ): let's start. Some more background articles. (X-Ports is still not working, do I really have to change my password??)

What has happened:

• configured the Nortel switch


• configured the dell1 and dell2 machines (all are accessible through huldra now, by ssh, did proper routing stuff with an entry in /etc/network/interfaces for setting the LAN/WAN route: up /sbin/route add -net 10.0.{1 | 2}.0 netmask 255.255.255.0 gw 10.0.{1 | 2}.{1 | 2})

And, finally the firewall1 is configured as it should be:

• switched to the serial console at boot time (it's magic: put the line set tty com0 in the file /etc/boot.conf in the root directory of the flash card!)


• edited properly the hostname.whatever files (means I can log in from huldra!) HINT: the interfaces are numbered top-down or left-right. Means: sk0 is the upper network card (WAN - 192.168.1.1), sk1 the lower (LAN - 192.168.2.1), vr0 is the left built-in card (ADMIN - 192.168.1.31), vr1 is the PFSYNC (10.0.3.1, not in use yet).

... and what happens? KERNEL PANIC upon initializing the network (more specifically: when initializing the Gigabit-cards):

(more startup messages)

pf enabled

starting network

uvm_fault(0xd15ed9c0, 0xea6b2000, 0, 3) -> e

fatal page fault in supervisor mode

trap type 6 code 2 eip d02a755e cs 50 eflags 10282 cr2 ea6b2628 cpl a0

panic: trap type 6, code=2, pc=d02a755e

Starting stack trace...

panic(ea5be334,0,ea5be398,d0460b63,d067603b) at panic+0xa4

panic(d067603b,6,2,d02a755e,6) at panic+0xa4

trap() at trap+0x2b3

--- trap (number 6) ---

itimerdecr(ea6b2620,2710,a0,24) at itimerdecr+0x34

hardclock(ea5be450,d8c6066c,6bf594,a0) at hardclock+0xb5

clockintr(ea5be450) at clockintr+0x17

Xrecurse_legacy0() at Xrecurse_legacy0+0x82

--- interrupt ---

bzero(d2755000,ea5be4e8,d0100eea,ea5be4ec) at bzero+0x24

vr_intr(d2755000) at vr_intr+0x35

Xrecurse_legacy9() at Xrecurse_legacy9+0x86

I start to have a pretty bad magefølelse ... will the firewalls ever work as expected??

Pain and tears ... the fun part first: instead of burning a CD or creating a floppy with OpenBSD, I used my OS X iBook as TFTP/PXE server (see this macosxhints hint). Two catches: needed to authenticate as root in the NetInfo manager in order to be able to add new machines, and one needs to fill in the right MAC-address from the target device.

Took me a while to understand OpenBSD's disklabel stuff for my wd0-disk -- in other words, the first attempt failed since I only mounted the root slice and filled it up - restart the installation ... -- with a=whole slice [not true in my case, turned out to be the root slice], b=swap, c=entire disk [not true either: only the OpenBSD-partition], d=/usr, e=/tmp, f=/var. My NTFS, Linux and Linux swap did not show up at all. Who cares ... well maybe later.

Installed almost all sets (not bsd.mp, game38.tgz and xshare38.tgz).

Once running, I needed to configure my Netgear MA 401 manually with ifconfig wi0 nwid wlanname nwkey 0xWEP-CODE-HEX and dhclient wi0 (thanks, this blog was useful).

Added new user (group wheel, login class staff, visudo do allow wheel to sudo). Changed /etc/hostname.wi0 to contain the line dhcp nwkey wlanname nxkey 0xWEP-CODE-HEX, and hope for the best.

The moment for today - can I mount the flash disk? -- YESSSSSS!!!!!!!!!! Dump the card (=backup), start messing with the config tomorrow.

Natta.

Well. So I give up. Looks like Linux just cannot handle the FFS filesystem. Any mount command (like mount -t ufs -o ufstype={old | ufs | 44bsd},ro /dev/sda4 /tmp/freecode) just produces a kernel bug and crash. I was not in the mood for checking if the problem is fixed in the latest 2.6-kernel ...

So I decided to offer my FreeBSD-installation/partition and set up a fresh OpenBSD-installation. Do I really believe this??? Just for getting the darn card to read!!!

And by the way ... both FreeBSD and Mac OS X was unable to read the file system, too. Great compatibility, dear BSD-folks!!

Worked on the primitive what-might-google-find approach to dbl-check that I did not miss any important articles in ISI ... found some more (semi-academic) articles, good reading.

And ... is it really possible that I cannot mount the OpenBSD (FFS)-flashcard in any other operating system??! Linux, Windows (ffsdrv), Mac OS X ... all fail!! Do I have to install OpenBSD?? Dette er et syrt epple ...

Started looking for more articles that present VoIP firewall concepts, found like 5. They are becoming a part of the background-chapter ...

Worked more on the serial console problem ... now at least it works from dell1 to fw1, but only for the boot messages (if they are sent automatically to the console with a -h flag in boot.config - beware, this will disable any output on the screen! So you better have a working serial console before trying this!! (learned it the hard way)

And ... last but not least ... fw2 is missing RAM, no memory at all! Fire up some mail ...

Worked all friday in order to get the serial console to work (for remote booting the firewalls). First, minicom just did not show anything. Upon wanting to try the 2nd firewall by installing the Gigabit interfaces (excluding problem with fw2), turns out a PCI reiser card is needed to connect the NICs. Ødid not have any, so the question went up to FreeCode ... Claes looks around if they have one, or he will order one.

Finally done with including the parts of my survey that may be interesting to have in the thesis. The overganger are not smooth as slik yet, but this I manage later.

Tomorrow: configuring the Nortel, having a working network! Meeting with Hårek at 1215 @ Netlab.

Thursday as usual ... this week with practical assignments: editing Data Sets (DS) and their Members (DSM). Next week no lecture, yeah - a day for the thesis!

Got 2 Gigabit-interfaces for the firewall2. Since the firewalls have two gigabit-interfaces on purpose, the network topology needs to be changes. Big time - I need to trunk the VLANs through the optical port going to the Juniper. Meaning I have to talk to Sven/Jon Henrik/Gard for not messing up their setup.

And - FreeCode will move (this time it sounds serious). Upcoming weekend or something. Will see if I can use some office space there.

And yes. the never ending story about the master emner from previous studies continues ... have soon been (passively) waiting at Blindern.

Great book, soon done with it. Jumpstart lecture for the configuration of the two firewalls.