Bro-IDS - Signature Matching

Lately I have deployed a testing box on 30Mbps link by using Bro-IDS, apparently it is a small monster when running with default setting. Today I started to turn on the signatures matching engine. Guess what !!!!! The small monster starts to become hulk, let's see how it goes -

PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND
546 bro 1 -58 0 166M 164M bpf 80:40 81.42% bro

It seems that it is not a good idea to turn the signature matching engine on since it consumes too much processing power, I would rather having snort instance running for signatures matching and bro running as protocol analyzer indeed. Anyway it's up to you.

F34R teh Hulk!!!!!

Peace :]

Network Trace Files - Share it!???

I think people who work in Network Security should have chance to learn, and study the packet dump files, usually if we are following the Open Source Standard, libpcap is considered the most common format that widely been used everywhere including commercial companies.

However not much people want to share the network trace files, the critical and sensitive information yields many people stop doing that. I'm still looking forward to OpenPacket that soon will be launched, though I don't know when since Rich is busy with his stuffs. OpenPacket will serve a central repository for interesting network trace files. If you want to learn about protocol by studying the headers and payloads, you can check it out at,

http://wiki.wireshark.org/SampleCaptures

http://www.icir.org/enterprise-tracing/download.html

While you may wonder how you can share your network trace files, there are tools available to help you anonymizing the packet headers, I won't be showing how it can be done here but you can learn by reading the man page, or maybe waiting for my handbook that still in process. Here are the tools,

ipsumdump - http://www.cs.ucla.edu/~kohler/ipsumdump/

tcpdpriv - http://ita.ee.lbl.gov/html/contrib/tcpdpriv.html

tcpmkpub - http://www.icir.org/enterprise-tracing/tcpmkpub.html

There maybe other tools like netdude where it can edit network trace files on the fly. With those tools you can remove or modify the confidential data in the network trace files and share to the world.

P/S: For people wonder what I'm doing lately since not much updates in the blog, I'm still writing technical materials for the handbook that I plan to release after HITB conference.

Cheers :]

Aget - Flashget?

There's no open source flashget but there's a relatively good and fast http downloader which using multi threads to retrieve files from http server, though it is kinda old tool but I like it for the fact that it offers fast and consistent download speed, let's check out Aget. You may find the main site of aget at

http://www.enderunix.org/aget/

I run aget with the recommended -n 20 threads and use -f to force the usage of 20 threads, there it goes ----->>>>>

With my crap ISP home link, I can get roughly of 83Kb/s. This is not bad at all for my situation. The only thing that aget lacking would be support for ftp, however since aget is no longer in active development, I doubt that it will be updated with that functionability, you can use wget for that purpose as alternative.

Cheers :]

Netflow - One Useful Link

While digging the information regarding Netflow, I found a very good reference and useful link, I think I will read over it in details before jumping to other resources I found because this seems to be better and complete with the RFC reference as well.

http://netflow.caligare.com/

At the same time, I'm trying to learn about protocol tunnelling which I seldom get in touch with that used to evade IPS/IDS most of the time.

Cheers :)

Fluxbox 1.0 RC

Version jumping again from the project after Wireshark(ethereal), Fluxbox goes 1.0 RC after 0.9.15, here we see another open source project grows to be mature. Check it out at www.fluxbox.org.

Again I haven't been doing much blogging, real life sucks me out of it. I have been doing a lot of researches and studies on how one can use generic flow analysis to detect anomaly or malicious network activities.

Fosscar is around, for people who don't know about it, you may check it out at www.fosscar.com. Me and other OSS folks will be speaking and running workshops in the event. Hopefully I get a chance to have beer with them again.

Have fun :)

FreeBSD - IDS Sensor Tweaking

IDS used to suffer in high speed network where it need to sustain the heavy load traffics while detecting malicious traffic. Relying solely to the IDS software seems not to be a right idea, hence OS tweaking is supposed to be done in order to build a perfect Intrusion Detection System with commodity hardware, of course gigabit network card is preferred with lots of RAM. Here's my current testing configuration and I hope this is helpful to certain people who want to run IDS with comodity hardware and using either Bro-IDS or Snort. The OS I'm running is FreeBSD, you may find similar tweaking with Linux.

I added this to kernel config file in order to enable device polling,

options DEVICE_POLLING
options HZ=1000

After recompile the kernel and install it, I added those values below to the /etc/sysctl.conf

net.bpf.maxbufsize=8388608
net.bpf.bufsize=4194304
net.inet.tcp.sendspace=65536
net.inet.tcp.recvspace=65536
net.inet.tcp.rfc1323=1

Then I added this configuration to /etc/rc.conf for the network interface that used to capture network traffic and running IDS as well.

ifconfig_fxp1="polling promisc up"

I suggest if you are running IDS with commodity hardwares, you may need two network interface, one will be the management interface with access control enabled and another one just run as IDS interface, the configuration above applies to the IDS interface where IP is not needed and no other traffics inteference except the traffic you want to capture.

I'm currently testing how well this experimental IDS box in heavy load traffic, I run snort in sniffer mode,

shell>snort -i fxp1 -D

My snort PID is 738, since I have bpfstat installed, I try to run -

shell>bpfstat -i 3 -I fxp1 -p 738

You can view the result in the screenshot, 0 drop rate .....

If you happenned to know the better tweaking of OS or you are actually performing tweaking for your IDS box in different kind of OS as well, please do feedback or comment. I would like to learn more ways of building IDS boxen with commodity hardware.

Cheers (:])

FreeBSD - Google Earth

Since my friend told me the availabilities of Google Earth on Linux Platform with it's beta v4 release, I just browse to take a look there - http://earth.google.com/index.html, I downloaded the Linux version and think that it maybe fun to try to install it on FreeBSD, and with Linux ABI supports enabled, I then just go to directory that I have google earth downloaded and run sh ./GoogleEarthLinux.bin, the installation works flawlessly and you may check out the screenshots below.

Configure and Install .....

Installation Done ..... :)

Running Google Earth at the first time .....

Check out where I'm now .....

I'm kinda happy with Google Earth on FreeBSD, though it maybe slow because of soft emulation. Enjoy .....

Cheers :]

FreeBSD last.fm

I know you like radio station with cool musics don't you, and last.fm might be one of the popular fm these days, in fact last.fm been very Open Source Oriented, you can even download the player for various OS including Linux and FreeBSD, I have just downloaded the FreeBSD version and try installing it, tada!!!! It goes perfectly and I can now have fun with last.rm. Remember to register in last.fm.

Downloading it .....

Install with pkg_add after downloading it and start playing with last.fm.

That's all, folks.

Cheers :)

Session Data - Useful Links

I have been doing a lot of reading on netflow and session data collections and methodologies, and since I'm now moving to more systematic learning method, I always collect all the useful links and documentations before reading it in one shoot, there may be information overflows but I think that's more easy to make comparisons when reading and intepret. Since I find them useful, I might share the links as well, here you have it -

http://www.cs.dal.ca/~mchugh/netanalysis/slides/01-Introduction-2up.pdf

http://www.dynamicnetworks.us/netflow/index.html
http://www.networkuptime.com/tools/netflow/
http://www.hcs.ufl.edu/~park/tracearchive.html
http://events.ccc.de/congress/2005/fahrplan/attachments/560-Paper_IntrusionDetectionSystems.pdf

http://users.pandora.be/jurgen.kobierczynski/jkflow/eindwerk.pdf

www.educause.edu/ir/library/powerpoint/MWR0574A.pps

www.cert.org/flocon/2005/presentations/Trammell-Translator-FloCon2005.pdf

http://www.acsac.org/2005/case/wed-1030-yurcik-paper.pdf

http://cansecwest.com/core03/jhaile-cansec03.ppt


You can subscribe Argus Mail List at

https://lists.andrew.cmu.edu/mailman/listinfo/argus-info

Most of the links are presentation type so it should not take too much times to read. Hopefully you enjoy reading them.

Peace :]

Desktop Tips - icon

Again this is small tips for desktop users, if you happenned to have lovely icons that not available in your Open Source OS, you can actually convert it with the small util which is called iconconvert, just grab it via port/package will do,

shell>pkg_add -vr iconconvert

And if you have file with png format and you want to convert it to xpm which is loadable via fluxbox, you can use the small script that written by tenner via,

http://tenr.de/files/png2xpm.sh

There you will have tons of icons that you can use now.

Cheers :)

Xtra for ThinkPad X series Fluxbox users

If you are happenned to have Thinkpad X series and you are Fluxbox user, this is for you. I have few keys mapping work perfectly, here's how my configuration.

Here's my ~/.Xmodmap

keycode 92 = F13
keycode 111 = SunPrint_Screen
keycode 233 = XF86Forward
keycode 234 = XF86Back

Here's the keys file under ~/.fluxbox

Mod1 l :ExecCommand xlock -mode matrix -geometry 1x1 -enablesaver

None XF86Forward :NextWorkspace
None XF86Back :PrevWorkspace
None Print :ExecCommand scrot '%Y%m%d%R_$wx$h_scrot.png' -e 'mv $n ~/i-Screenshots/'

You can now jump to previous/next workspace with the mail forward and mail backwad key, and the printscreen will work too after you install scrot via package. Alternate + L will lock the machine if you install xlockmore.

Remember to add xmodmap ~/.Xmodmap at ~/.fluxbox/startup, this is important to get the key mapping works.

And guess WHAT?!!! Lenovo now turns their head again, check out the link below -

http://www.desktoplinux.com/news/NS5301096581.html

Again, we cheers :]

FreeBSD - Fluxbox + Gdm

I have been in Freenode #fluxbox channel for a while, it seems that many people are asking the same question regarding how to setup fluxbox on FreeBSD, previously I have written how to setup Fluxbox + Gdm in OpenBSD and I think I should write this one for FreeBSD. I will discard the X configuration part because it is similar to the previous OpenBSD Fluxbox post. Here's the quickies -

Installing fluxbox-devel and gdm, remember don't install the old fluxbox, many FreeBSD used to install the old stable version which is not actually stable compare to the recent devel version.

shell>pkg_add -vr fluxbox-devel gdm

Configure it to load through gdm,

shell>cd /usr/X11R6/share/gnome/xsessions

shell>touch Fluxbox.desktop

Adding the lines below to Fluxbox.desktop,

[Desktop Entry]
Encoding=UTF-8
Name=Fluxbox
Exec=/usr/X11R6/bin/startfluxbox
Icon=
Type=Application

Configuring ~/.xsession

Add this line,

exec startfluxbox

To add it into gdm session alternative,

shell>echo "exec /usr/X11R6/etc/gdm/Xsession \
/usr/X11R6/bin/startfluxbox" >> /usr/X11R6/etc/gdm/Xsession

Now you can find that you have fluxbox as alternative in your gdm menu when you login, just choose it if you want Gdm to lauch Fluxbox after login.

- Go Fluxy -

Cheers (:])

Bro-IDS - The learning process

Since I want to have more tools to provide valuable alert data for clues when accessing network traffic, I have installed bro-ids on my FreeBSD workstation. It is installed fine on FreeBSD, however when I try to run bro against pcap file, I get an error where bro.init not found, bro.init file is in policy directory and running bro in that directory works, so that must be path issue and it can actually be resolved easily by adding the following lines to your .bash_profile if you are using bash shell.

BROHOME=/usr/local/stow/bro-1.1
BROPATH=/usr/local/stow/bro-1.1/policy:/usr/local/stow/bro-1.1/site

export BROHOME BROPATH

That's it and now bro runs perfectly fine.

$BROHOME is your default Bro home directory and for your local config tweaking, you need to check for site directory under $BROHOME. Bro disables it's signatures detection capability by default, to turn it on, you just need to load the line below to the file - local.site.bro or one with your host.domain.bro,

@load brolite-sigs

Then restart Bro with the command /usr/local/etc/bro.rc checkpoint. In snort, those protocol decoders are defined as preprocessor, however in bro, it is called analyzer. Those analyzers are mainly the policy scripts that under $BROHOME/policy. You can write your own analyzer if you need one, that's pretty similar where you can write your own preprocessor for snort, especially version 2.6 now that you no longer need to patch snort to get unofficial/external preprocessors. You have dynamic preprocessor loading capability in snort 2.6!

I try to correlate the similarities of bro and snort so that I can take bro easily in my learning process, though Bro is developed for research purpose, it can be very powerful when comes to provide alert data. And those documentation and manual are comes with the source tarball when you downloaded Bro, so I read through the documentation and there are few chapters that pretty interesting such as Bulk Traces & Off-Line Analysis. Those mentioning how to analyse pcap file and using Bro to extract the packet payloads. I still feel adventureous with trying more stuffs with Bro and maybe getting a sensor running Bro to see how it goes.

That's all for Bro now, peace :]

Sguil Client - Quick && Easy

I remember I have problem installing Sguil Client on FreeBSD previously that push me to use source installation for one of the tcl libraries, however in FreeBSD 6.1, this is no more case, it is even rather easy to get sguil client works compared to other OS now. The steps are, should be the step is

shell>pkg_add -vr tcl tk tcllib tclX itcl itk iwidgets

Now just download sguil client from source forge, untar and run

shell>wish8.4 ./sguil.tk

Here's the screenshot,


By the way I'm now updating my snort to 2.6, hopefully I can play with it later.

Cheers :]

Multipurposes post :]

I have been out of posting due to some serious matters, anyway I think I should be writing some stuffs to keep me going. First of all, I'm pretty satisfied and happy that I have reached 200 blog posts where I never think of writing so much. Thanks for the comments and feedbacks along.

Few things I want to blog about are I will no longer be supporter of IBM ThinkPad after Lenovo bought over it's brand name, the bad design and idea that put the Thinkpad to the dead, what can I say, Lnv you sux big time - check the link below.

http://hardware.slashdot.org/article.pl?sid=06/06/04/0415221

I guess my main choices would be either Toshiba or HP now, seriously Lnv is a real ass hole. Bye beloved IBM ThinkPad. For people who haven't have chance to look at the new Z series, the design is utterly ugly.

I have fun playing with Bro-IDS under FreeBSD, it is installed fine on FreeBSD 6.1R, here's are the note when I install it. You need to install the package below first

shell>pkg_add -vr p5-Config-General adns

Then just run the usual configure, make and make install, since I'm not integrating bro as the tool to provide alert data, I prefer it to be on /nsm for management wise, so that's what I do, again I use stow for source installation management. I untar the bro-1.1 and start my installation process with

shell>mkdir /nsm/stow

shell>./configure --prefix=/nsm/stow/bro-1.1

shell>make && make install

shell>make install-brolite

It will ask you a series of questions for configuration settings.

shell>cd /nsm/stow && stow bro-1.1

Installation are done now and you can start bro with

[root@trinity /nsm/stow]# /nsm/stow/bro-1.1/etc/bro.rc --start
bro.rc: Running as non-root user bro
bro.rc: Starting ..........bro.rc: Failed to start Bro
/usr/local/stow/bro-1.1/bin/bro: problem with interface bge0 - pcap_open_live: (no devices found) /dev/bpf0: Permission denied
... FAILED

Since I get permission denied, I change the permission setting of bpf0

[root@trinity /nsm/stow]# ls -la /dev/bpf0
crw------- 1 root wheel 0, 115 Jun 6 07:28 /dev/bpf0
[root@trinity /nsm/stow]# chmod 604 /dev/bpf0
[root@trinity /nsm/stow]# /nsm/stow/bro-1.1/etc/bro.rc --start
bro.rc: Running as non-root user bro
bro.rc: Starting ............. SUCCESS

To check if it is running,

[root@trinity /nsm/stow]# ps auxww | grep bro
bro 17459 0.0 0.1 1760 1104 p3 I 10:29AM 0:00.03 /bin/sh /usr/local/stow/bro-1.1/etc/bro.rc --start
bro 17464 0.0 1.1 12716 11512 p3 R 10:29AM 0:04.76 /usr/local/stow/bro-1.1/bin/bro -W -i bge0 trinity.dissectible.org.bro
bro 17510 0.0 0.1 1760 1104 p3 I 10:29AM 0:00.00 /bin/sh /usr/local/stow/bro-1.1/etc/bro.rc --start
bro 17512 0.0 0.5 6836 5584 p3 S 10:29AM 0:00.12 /usr/local/stow/bro-1.1/bin/bro -W -i bge0 trinity.dissectible.org.bro print-filter.bro

Check if it adds the cron entry correctly,

[root@trinity /nsm/stow]# crontab -e
BROHOME=/nsm/stow/bro-1.1
# checkpoint Bro once a week
0 0 * * 1 /nsm/stow/bro-1.1/etc/bro.rc --checkpoint
10 00 * * * ( nice -n 19 /nsm/stow/bro-1.1/scripts/site-report.pl )
10 3 * * * (/nsm/stow/bro-1.1/scripts/mail_reports.sh /usr/local/stow/bro-1.1
/etc/bro.cfg)
0 3 * * * (/nsm/stow/bro-1.1/scripts/bro_log_compress.sh)
# If you are process logs on a front end host, add this:
#10 3 * * * (/nsm/stow/bro-1.1/scripts/push_logs.sh FrontendHost)

Bro suggests tweaking bpf buffer size and its max value, I tweak it manually, I'm thinking of testing this sysctl settings for my sguil sensor as well and guess it should be applicable.

shell>sysctl net.bpf.maxbufsize=8388608

shell>sysctl net.bpf.bufsize=4194304

To uninstall it cleanly, again we will make use of stow,

shell>cd /nsm/stow && stow -D bro-1.1

Go to the bro source directory and run

shell>make uninstall

shell>rm -rf /nsm/stow/bro-1.1

shell>make distclean

Now everything back to the previous state where you haven't installed bro-ids. Since bro-1.1 is installed cleanly, I supposed it should be easy to make into port/package, the FreeBSD package which is version which is version 0.8 is kinda dated, may need to email the porter for updates.

For sleuthkit on FreeBSD, you need to install the package below or else mactime won't work,

shell>pkg_add -vr p5-Date-Manip

Autopsy is not working when install via package, as it can't find Main.pm. Thus I install using port and it works now.

shell>cd /usr/ports/sysutils/autopsy && make && make install

Now what?!!! Of course snort, snort-2.6 Final is released, you may find out all interesting features and updates in the link below, go go snorting .....

http://www.snort.org/pub-bin/snortnews.cgi#445

Hopefully I can make to 300 blog posts !

Cheers (:])