Tuesday, June 17, 2008

HeX 021: Learning PCRE and its performance

PCRE stands for Perl Compatible Regular Expressions, it is mainly used for pattern matching. If you want to learn more about PCRE, take a good read of its manual -

shell>man pcre

shell>man pcrematching

shell>man pcrepartial

shell>man pcrepattern

shell>man pcreperform

So why do you need to learn regular expressions(regex), here's the answer -

http://geek00l.blogspot.com/2006/12/regex-magic-for-netsexcanalyst.html

Next look at the tool that comes with pcre - pcretest, as the name implies, you can use pcretest to test your regex. Lets go -

shell>pcre --help
Usage: pcretest [options] [input file [output file]]

Input and output default to stdin and stdout.
This version of pcretest is not linked with readline().

Options:
-b show compiled code (bytecode)
-C show PCRE compile-time options and exit
-d debug: show compiled code and information (-b and -i)
-dfa force DFA matching for all subjects
-help show usage information
-i show information about compiled patterns
-m output memory used information
-o set size of offsets vector to
-p use POSIX interface
-q quiet: do not output PCRE version number at start
-S set stack size to megabytes
-s output store (memory) used information
-t time compilation and execution
-t time compilation and execution, repeating times
-tm time execution (matching) only
-tm time execution (matching) only, repeating times

If you have already read the man pages above, you should be able to understand some of the options, I normally use the option -C to check the compiles-time option first -

shell>pcretest -C
PCRE version 7.7 2008-05-07
Compiled with
UTF-8 support
Unicode properties support
Newline sequence is LF
\R matches all Unicode newlines
Internal link size = 2
POSIX malloc threshold = 10
Default match limit = 10000000
Default recursion depth limit = 10000000
Match recursion uses stack

Other option I usually use is -t to test on the time compilation and execution of particular regex I write.

shell>pcretest -t
PCRE version 7.7 2008-05-07

re>

So you may see the prompt goes to interactive mode - re>, it is for you to define your regex, bear in mind that your regex must use forward slash as delimeter, for example -

re>/[a-z0-9]+/

This means your regex is [a-z0-9]+, once you enter you will see this -

Compile time 0.0028 milliseconds
data>

You may notice the compile time for this regex is 0.0028 milliseconds, now you try to put any data to see if they match the regex,

data>ABC

Once you hit the enter, you will see this -

Execute time 0.0008 milliseconds
No match

The execution time is 0.0008 milliseconds and there's no match, lets change the data -

data> abc
Execute time 0.0004 milliseconds
0: abc

We can now see the execution time is 0.0004 milliseconds and the data seems to match the regex.

You can also figure out multiple regex compile time on the fly by defining them in a file instead of using interactive mode. For example I write the lines below to a file - pcre-testing.txt

/\d{,10000}/

/([a-z0-9]+)?/i

Do remember that if you want to test multi regex at once, you have to split them with a blank line, you can't do like this and it will incur errors -

/\d{,10000}/
/([a-z0-9]+)?/i

Now we can run this -

shell>pcretest -t pcre-testing
PCRE version 7.7 2008-05-07

/\d{,10000}/
Compile time 0.0032 milliseconds

/([a-z0-9]+)?/i
Compile time 0.0054 milliseconds

There are other options that you may want to try out, but I think I have given you enough guide to carry on, you may be interested in reading some of my related posts here -

http://geek00l.blogspot.com/2007/11/regex-learning-tool-kregexpeditor.html

http://geek00l.blogspot.com/2007/07/visualregexp-nice-regex-learning-tool.html

I advocate pcretest because it comes with pcre and available in HeX, and you can evaluate the performance of the regex quickly.

Enjoy (;])

Wednesday, June 11, 2008

HeX 2.0: Sneak Peak

We bring you the HeX 2.0 quick preview(it's really just view)!!!!!

FreeBSD 7.0-STABLE, is it real?

Sguil Client 0.7 is here!

Where's the monkey, morphing into lobster?

Stop snorting, oink oink!!!!!

Don't you think it is sexy when shark is on the wire?

Ask for more? Be patient!!!!!

Cheers (;])

Monday, June 09, 2008

MSN IM -> Blogspot -> Pr0ning

I came across this seductive message, and it contains the link that I can't resist to click since it is asked by horny ladies, the link must be legitimate -

http://cux7850mdmk.blogspot.com

Once you click on it, that blog will bring you to another site which is -

http://66.111.45.170/cams/1/

You can see below what is loaded when you go to the blog that is setup with malicious purpose -

The cut-down zoom in version -

META http-equiv="refresh" content="0;URL=http://66.111.45.170/cams/1/"

I manually check http://66.111.45.170/cams, and you might enjoy the screenshot -


Lets see what is in http://66.111.45.170/cams/1/, the content location is actually at -

http://66.111.45.170/cams/1/index.htm

And the index.htm contains -

meta http-equiv="refresh" content="0; URL=http://www.xxxblackbook.com/?s=register&r=lc129795"

Now you should be happy to land at this page, and lets register as a member.


It's rather easy to get someone to click on "look legitimate" link than from the email spam these days. We see the use of meta http-equiv="refresh", and you can find the information about it here -

http://www.html-reference.com/META_httpequiv_refresh.htm

During discussion at freenode #rawpacket, my friend scholar pointed me out related information here -

http://spamtrackers.eu/wiki/index.php?title=Blogspot

Enjoy ;]

Sunday, June 01, 2008

Network Flow: Uni-Directional VS Bi-Directional

If you are working on network flow research, you should have heard about Uni-Directional and Bi-Directional Network Flow. I will try to explain what are they here. Lets take the quick look of what network flow is first -

Network Flow is the sequence of packets or a packet that belonged to certain network session(conversation) between two end points but delimited by the setting of flow generation tool. To cut it short, it provides network traffic summarization by metering or accounting certain attributes in the network session.

The endpoints here are defined as below -

Layer 2 Endpoint - Source Mac Address | Destination Mac Address
Layer 3 Endpoint - Source IP Address | Destination IP Address
Layer 4 Endpoint - Source Port | Destination Port

Before we dive into understanding of UniFlow and BiFlow, lets look at the definition of Uni and Bi here -

http://www.yourdictionary.com/uni-prefix

http://www.yourdictionary.com/bi-prefix

Uni - one; having or consisting of one only; regarded as a single entity

Bi - using two or both; joining two, combining or involving two

In the context of Uni/Bi Directional Flow, Uni means single, Bi means both. Now, let make it more clearer.

Uni-Directional = Single Directional

Bi-Direction = Both Directional

I put up the illustration in the diagram below.

Uni-Directional Flow


Bi-Directional Flow

Now I will make a simple example, host A sends 90 bytes to host B and host B replies with 120 bytes. Here's the output -

Uni-Directional Network Flow
Srcaddr Direction Dstaddr Total Bytes
Host A -> Host B 90
Host B -> Host A 120

Bi-Directional Network Flow
Srcaddr Direction Dstaddr Total Bytes Src Bytes Dst Bytes
Host A <-> Host B 210 90 120

The Srcaddr and Dstaddr are the endpoints here. In Uni-Directional Flow, you only see the total bytes that sent by Host A(attribute of Host A) but nothing about Host B in the first flow record. Then the next record shows Host B sends 120 bytes to Host A(attribute of Host B). The total bytes is accounted from single endpoint(either Host A or B) only. But in BiFlow, you can see that Host A sends 90 bytes(Source Bytes) and Host B replies with 120 bytes(Destination Bytes). The total bytes is the accumulation of source and destination bytes. To summarize them -

Uni-Directional Network Flow Model - One direction at a time, every flow record contains the attribute of single endpoint only.

Bi-Directional Network Flow Model - Both direction at a time, every flow record contains the attribute of both endpoints.

Theory is tough sometime, here's the practical sample -

Cisco NetFlow uses Uni-Directional model for flow generation

Argus uses Bi-Directional model for flow generation

To draw good picture of Uni-Directional and Bi-Directional Network Flow, it's best to do comparison of them.

1. Network Flow data which is generated by Argus 3 natively
2. Network Flow data which is generated by Cisco NetFlow version 5

The flow records below are generated from the same network session. You can examine closely by clicking on them.

Cisco NetFlow(UniFlow):


Argus(BiFlow):


Flow record property:
SrcAddr = Source Address
Sport = Source Port
Dir = Direction
DstAddr = Destination Address
Dport = Destination Port
SrcPkts = Source Packets
DstPkets = Destination Packets
TotPkts = Total Packets
SrcBytes = Source Bytes
DstBytes = Destination Bytes
TotBytes = Total Bytes

Sometimes I like to think that UniFlow is stateless and BiFlow is stateful.

I will continue writing this Network Flow series, and I hope you enjoy it. Stay tuned for the next one - Traffic Matrix. And of course the HeX 021 series too.

Argus 3 Tip:
You can convert Argus BiFlow to UniFlow by using -M rmon option.

Peace (;])

Friday, May 30, 2008

Network Flow: TopN

There are a lot of questions popping up on and off in argus mailing list regarding how to generate TopN output from argus data, but frequently you may find the questions are too rough to give complete answer.

I'm going to discuss about TopN this time, TopN is the technique that widely used in many industries, what is it for?

TopN is used to retrieve the first N records from the data based on certain object and ordered by its property. Since I'm talking about Network Flow, I would like to make the example using it.

Data: Network Flow Record
Object: Protocol, Network, IP(host), Port, etc
Object Property: Packet Count, Byte Count, etc

Bear in mind that I'm avoiding the use of Flow terminology but layman one so that this example can be understood easily.

If you want to use TopN technique to generate information from the network flow data, first you need to know what you are looking for. Lets go with a simple one -

I want to find out Top 5 IP ordered by Total Packet Count

Total Packet Per IP(host) = (packet send + packet receive) Per IP(host)

Now you run the argus client command to parse the data and generate exactly the result which looks like this -

shell>racluster -M rmon -m saddr -nr testing.arg3 -w - | \
rasort -m pkts -w - | \
ra -L0 -N 5 -s saddr pkts

SrcAddr TotPkts
172.16.1.108 993
193.231.236.41 824
211.185.125.124 178
172.16.1.103 56
211.180.229.190 36

The command above is to generate Top 5 IP ordered by Packet Count. Don't ask me about the command line, it looks complicated for now but that's not my point here, look at the output instead. Host 172.16.1.108 sends or receives 993 pakcets, followed by 193.231.236.41 and so forth.

Now if you want to locate Top 5 IP ordered by Byte Count. You can just run -

shell>racluster -M rmon -m saddr -nr testing.arg3 -w - | \
rasort -m bytes -w - | \
ra -L0 -N 5 -s saddr bytes

SrcAddr TotBytes
172.16.1.108 599949
193.231.236.41 579050
211.185.125.124 18901
172.16.1.103 4964
216.168.224.69 3458

You want to use TopN, you should draft out the TopN output you are looking for, I have seen questions like this -

1. Which is the most active network?
2. Who is the most active sender?
3. Who is the most active receiver(got ddos?)

Or worse,

How can I find out the top talkers?

These kind of questions are too loose, you should at least specify the property, such as most active sender that is ordered by packet count, or most active network that is ordered by byte count and so forth. You have to bear in mind that packet and byte are not going inline, you can have one host sending many small size packets which won't hit TopN byte count at all.

With this kind of idea in mind, you can build the list of TopN which can draw you a good picture of network activeness to solve different issues.

For the next round, I will introduce Traffic Matrix, stay tuned!

Enjoy (;])

Thursday, May 29, 2008

Laptop: Alternatives For Security Road Warrior

Which laptop model are you using? This is my question today. I have been looking for laptop of choice for myself, as a security road warrior, I prefer it to be -

1. Lightweight(small/medium size and compact)

2. Black and solid look

3. Miminum 150G of Disk and 4G of Rams

4. Great keyboard touch build(old thinkpad?)

5. 12-14inch display

6. Good battery life

7. FreeBSD/Linux Compatible or I will use VMware/VirtualBox

Currently there are 3 laptop models I have in my mind -

1. Macbook Black

2. Thinkpad T61

3. Dell Xps m1330

What do you think and what's your favorite laptop if you work in security industry, and you need to travel frequently? I'm looking forward for any good suggestion and sharing. Thanks!

Peace ;]

Wednesday, May 28, 2008

HeX 021 Series

I will start this HeX Zero To One(021) Series in my blog while HeX 2.0 is in active development, and all of them will be imported to HeX Handbook. In future you will see my post with the title prefix of HeX 021: belongs to the series.

Enjoy ;]

HeX: From Zero To One

These days, I have encountered questions like this,

1. How can I be an efficient network security analyst?

2. Is there a quick path or short cut to be one?

3. I'm just system administrator/programmer and don't know crap about security, I'm interested in it but don't know where to start?

4. There are so many resources in the internet, what's the specific knowledge required to be network security analyst so that I can be more focusing on particular subjects?

If you are a student, or just starting to work as network security analyst, I hope this post will shade some lights for you -

I would like to point out 3 posts that I have written -

http://geek00l.blogspot.com/2008/05/hex-handbook.html

http://geek00l.blogspot.com/2007/07/hex-livecd-analogy.html

http://geek00l.blogspot.com/2007/03/netsecanalyst-handbook.html

The HeX System that we are developing is the key to answer almost all your questions, but you might not be able to know where to look at if I throw you HeX liveCD without giving you hints or tips. The HeX Handbook which is derived from my own Network Security Analyst Handbook is actually designed to lead you to the right path.

If you read my post about HeX liveCD analogy, I mentioned this -

The HeX liveCD can only make up to this part(see below) -

Obtain Network Based Data -> Utilizing NSM Based Tools -> Generate Output

The rest depends on how analyst able to perform it -

Output Interpretation -> Output Analysis -> Output Summarization -> Report

Clearly enough, the HeX itself can't do everything for you, you have to help yourself starting from Output Interpretation process.

If you apply the reverse thinking, what are the obstacle you have encountered during Output Interpretation? You have used the tools to generate the output for you, for example - snort, bro-ids or even simple tcpdump. Apparently if you find yourself can't understand those output, you can't interpret them correctly. Now the important question is "Why can't you understand those output?" There are few answers to it -

1. You may not have enough network protocol knowledge.

2. You may not familiar with the tools because different tools tend to generate the output in different ways or results.

3. You may not update yourself with current security trends(follow bug traq, cve and so forth)

4. You are being lazy

Now I flash back again to my Network Security Analyst Handbook post, I have put the book into four sections -

Sec 1 - Net Sec Analyst: The RoadMap
Sec 2 - Net Sec Analyst: The Workflows
Sec 3 - Net Sec Analyst: The Tools
Sec 4 - Net Sec Analyst: The Case Study

For the Section 1 and Section 2, I have elaborated them as -

Network Security Analyst: The RoadMap
What are good foundations and technical knowledge that should be acquired to become good network security analyst? I hope The RoadMap can answer question like that, until now I haven 't seen any books discussing about this topic yet.

Network Security Analyst: The WorkFlows
What are the methodologies and mechanisms that are used by network security analyst to handle their tasks? The routine daily tasks, the automated and manual process of performing analysis, situation handling and so forth.
This is more of how to think or work like a network security analyst. I will try to standardize the common work flows but you are free to extend it to your own way.

If you have gone through Section 1 and 2, you should be able to do this -

Output Interpretation -> Output Analysis -> Output Summarization -> Report

Unfortunately we don't offer these in HeX version 1.x, but this is going to change, we are currently working on integrating things that are discussed in Section 1 and 2 into HeX version 2.x which will be released sometime around June. As Section 3 is already integrated into HeX, you should be able to complete Section 1-3 with HeX, all you need is discipline!

This is not a myth, the HeX Handbook will guide you to complete Section 1-3 using HeX System itself, you don't need more.

For the Section 4, I already have other plan and maybe you can see them in HeX 3.x, who knows.

If you want to learn to be a competent network security analyst, you can start with HeX. It will take you from 0 to 1.

Now I start to think that University should offer this kind of course for students, as far as I know country like Philippine has their Universities offering malware analysis course and therefore you can see a lot of them working in AntiVirus Industry, if our country want to produce competent network security analyst, they should offer security related courses in University. Not wait until they are out of school and busy with works.

Enjoy ;]

Monday, May 26, 2008

Training: Practical Network Flow Analysis

This time, me and spoonfork will bring you our new and upcoming training which is -

Understanding Network Conversations:
Practical Network Flow Analysis


Here's the description of our training -

Network Flow data represents a summary of conversation between two end points. It provides valuable information to assist investigation and analysis of network and security issues. Unlike deep packet inspection, flow data does not rely on packet payloads. Instead the analyst relies on information gathered from packet headers and its associated metrics. This provides the analyst a neutral view of network traffic flow by tracking network sessions between multiple endpoints simultaneously. In addition, having network flow data will provide a better visibility of network events without having the need to perform payload analysis.

With the implementation and deployment of Network Flow technologies, an analyst can discover different types and classes of network activities, be it normal or abnormal. In this training we will show you how to interpret Network Flow data and perform practical Network Flow Analysis.

While high level theory explanations are extremely useful, hands-on exercises are even better. Each chapter is accompanied by practical hands-on exercises such as exporting network flow data from Unix and Cisco-based routers, performing simple operations such as IP accounting, network baselining, and identifying different kinds of network anomalies and attacks.

Who should attend?

Network Security Analyst
Network Administrator
ISP Network Architect
System Administrator

Bonus

+ First 10 registrants get free seat for HITB Conference Kuala Lumpur in October 2008

+ Human Resources Development Fund(HRDF) Claimable

For more information, check it out at -

http://training.hitb.org/flowanalysis/

Cheers (;])

Friday, May 23, 2008

HeX: Handbook

While we are in the active development of HeX 2.0, we will start a side project mainly for documentation purpose. We call it HeX Handbook, the link is here -

https://trac.security.org.my/hex/wiki/HeXHandbook

Currently there's nothing yet, but I will import all the contents from my incomplete Network Security Analyst Handbook to there, and I'm now trying to design the standard template so that whoever want to contribute can follow the template.

If you are using HeX, and you know different way of doing analysis using the tools in HeX, we would like to hear from you. By the way, if you are good in language translation, please do let me know.

Thanks to scholar who always gives me very fruitful input!

Cheers (;])

Como: Installation on Ubuntu

I found this without intention while searching for tool that can convert pcap format to netflow v5 format, its name is Como, the project which is developed by Intel people. If you want to know more about Como, check out their publications here -

http://como.sourceforge.net/publications.php

Here's the quick way to get it installed on Ubuntu -

shell>sudo apt-get install cmake

shell>wget \
http://como.sourceforge.net/download/como-1.5.tar.gz

shell>tar xvzf como-1.5.tar.gz

shell>mkdir build-como

shell>cd build-como/

shell>cmake -DCMAKE_INSTALL_PREFIX=/usr/local/stow/como-1.5 ../como-1.5

shell>make

shell>sudo make install

Once you have it installed, you can run it via command line interface but make sure you have configured its paths and modules. The configuration file is como.conf which can be found under directory /usr/local/stow/como-1.5/etc/como, you can also enable its module in run time. Here's the available modules -

shell>ls -la /usr/local/stow/como-1.5/libexec/como-1.0/
total 596
drwxr-xr-x 2 root root 4096 2008-05-23 10:34 .
drwxr-xr-x 3 root root 4096 2008-05-23 10:34 ..
-rw-r--r-- 1 root root 21281 2008-05-23 10:34 apps.so
-rw-r--r-- 1 root root 18778 2008-05-23 10:34 assoc.so
-rw-r--r-- 1 root root 23131 2008-05-23 10:34 autofocus.so
-rw-r--r-- 1 root root 15151 2008-05-23 10:34 dhcp.so
-rw-r--r-- 1 root root 21425 2008-05-23 10:34 ethtypes.so
-rw-r--r-- 1 root root 25640 2008-05-23 10:34 ewma.so
-rw-r--r-- 1 root root 16434 2008-05-23 10:34 flowcount.so
-rw-r--r-- 1 root root 27356 2008-05-23 10:34 flow-reassembly.so
-rw-r--r-- 1 root root 16169 2008-05-23 10:34 frames.so
-rw-r--r-- 1 root root 20629 2008-05-23 10:34 hwtm.so
-rw-r--r-- 1 root root 14416 2008-05-23 10:34 ipssi.so
-rw-r--r-- 1 root root 14736 2008-05-23 10:34 macssi.so
-rw-r--r-- 1 root root 13290 2008-05-23 10:34 nfexlist.so
-rw-r--r-- 1 root root 16080 2008-05-23 10:34 pattern-search.so
-rw-r--r-- 1 root root 20117 2008-05-23 10:34 protocol.so
-rw-r--r-- 1 root root 27098 2008-05-23 10:34 scanner-detector.so
-rw-r--r-- 1 root root 17405 2008-05-23 10:34 ssid.so
-rw-r--r-- 1 root root 21645 2008-05-23 10:34 superaddr.so
-rw-r--r-- 1 root root 25659 2008-05-23 10:34 topaddr.so
-rw-r--r-- 1 root root 25433 2008-05-23 10:34 tophwaddr.so
-rw-r--r-- 1 root root 21516 2008-05-23 10:34 topports.so
-rw-r--r-- 1 root root 15122 2008-05-23 10:34 trace.so
-rw-r--r-- 1 root root 19080 2008-05-23 10:34 traffic.so
-rw-r--r-- 1 root root 27192 2008-05-23 10:34 tuple.so
-rw-r--r-- 1 root root 20970 2008-05-23 10:34 unknown-ports.so
-rw-r--r-- 1 root root 39547 2008-05-23 10:34 worm-signature.so

To enable any of module at run time, you can just execute -

shell>/usr/local/stow/como-1.5/bin/como topports

It also provides http access to the data but I keep getting the error below while connect to http://localhost:44444 -

Module "" not found in the current configuration

I believe I have the modules configured properly but I can't get rid of the error, anyway that's all for Como and I will write more about it after some testings.

Peace ;]

Tuesday, May 20, 2008

Argus 3 Release

This is nothing new, argus 3 is finally released after long testing period. Thanks to everyone who involves in the argus 3 development and testing cycle especially Carter. You can download it at -

- http://qosient.com/argus/downloads.htm

If you are using FreeBSD, good new is argus 3 port is available now and you can check out the information about it here -

- http://www.freshports.org/net-mgmt/argus3/
- http://www.freshports.org/net-mgmt/argus3-clients/

I will try out the argus 3 ports on FreeBSD and see how it goes, have fun!

Enjoy ;]

HeX: Hardware Compability List

We started this list a while ago, and if you have tried out HeX and for the basic part it works properly, please update the list or at least email me about it.

https://trac.security.org.my/hex/wiki/Hardwares

So what are the basic things that must work in order to be included in the list, here you go -

- boot properly
- display properly
- ethernet adapter is supported

Please help out to improve the list, thanks!!!!!

Cheers ;]

Sunday, May 11, 2008

SecurityDistro: Interview

Thanks to Dakrone who has committed HeX to SecurityDistro which I don't know myself, and interestingly Josh from SecurityDistro sent me the interview questions via email and here's the interview result -

http://securitydistro.com/articles/407/Interview-with-CS-Lee-creator-of-HeX.php

Thanks to Josh for his kindness and free promotion from SecurityDistro.

I would like to thank to all the team members for progressive HeX development, and feel great to have you guys working together with me.

Cheers ;]

Network Flow Analysis: The Tools

I need to keep track of all the network flow analysis tools and study their offerings, this link contains many tools which may be useful for that purpose -

http://www.switch.ch/network/projects/completed/TF-NGN/floma/software.html

Enjoy ;]

Blog: Quick Update

I haven't been updating my blog lately, here's the quick one.

- I'm not in the mood of blogging but learning.

- I'm preparing myself for many things now which I can't tell yet.

- I'm learning network protocols that I'm not familiar with.

- I'm learning the advance usage of wireshark, and I'm glad the presentation slides of Sharkfest are available online here.

- I will spend 2 months of my free time on non-tech stuffs soon, which means I will still online but more for casual browsing, email checking and light reading. I need to be more focus!

- I will still blog even though the mood is not with me.

Cheers ;]

Thursday, May 01, 2008

CERT: Vulnerability Analysis Blog

CERT has launched its Vulnerability Analysis Blog which you can find at -

http://www.cert.org/blogs/vuls/

Another useful resource for security professional.

Cheers ;]

Wednesday, April 30, 2008

Dnstop: Statistical Tool for DNS Traffics

I learn about dnstop when reading rolland's presentation slide, thanks to his comment in my blog.

You can find his presentation at -

http://homepage.mac.com/roland.dobbins/FileSharing5


The presentation title is Listening to the Network: Utilizing Telemetry to Detect and Classify Network Traffic, I enjoy reading it as I have learned some new tricks or two from his presentation. One of the tool he has mentioned in the presentation is dnstop and I figure it is available in either Ubuntu packages repository or FreeBSD ports, I decide to try it out.

Just like the top command in Unix based system, it can run as real time monitor for your dns traffic by listening to the network device or it can also do post processing by reading the pcap file. The command option for dnstop is pretty straight forward and you can find them in man dnstop. It also provides a set of run time option to show different results from the statistical analysis output. To perform real time monitoring of dns traffic, you can listen to the device -

shell>sudo dnstop -4 lnc0

It looks pretty straightforward, you can also read the pcap file by specifying them -

shell>dnstop -4 -b ip -l 9 testing.pcap
Queries: 1005 new, 34812 total Wed Apr 30 18:10:46 2008

Sources Count %
-------------- --------- ------
192.168.42.149 3644 10.5
192.168.42.56 1965 5.6
192.168.42.33 1791 5.1
192.168.42.78 1790 5.1
192.168.42.163 1530 4.4
.....

I truncated the output here, then if you press 1, it will show the first level query names -

Queries: 44 new, 34856 total Wed Apr 30 18:11:55 2008

Query Name Count %
------------ --------- ------
com 13200 37.9
in-addr.arpa 8843 25.4
my 5950 17.1
org 3145 9.0
net 2630 7.5
biz 223 0.6
de 92 0.3
.....

If you press 3, it shows third level query names -

Queries: 0 new, 34856 total Wed Apr 30 18:13:10 2008

Query Name Count %
----------------------------- --------- ------
waumail.com 2805 8.0
208.218.in-addr.arpa 1883 5.4
excitedd.com.my 1388 4.0
sharishit.com.my 1112 3.2
sbl-xbl.spamhaus.org 938 2.7
.....

If you press !, it will show the sources that are performing first level query names -

Queries: 0 new, 34856 total Wed Apr 30 18:15:14 2008

Source Query Name Count %
-------------- ------------ --------- ------
192.168.42.149 in-addr.arpa 3007 8.6
192.168.42.78 in-addr.arpa 1009 2.9
192.168.42.56 in-addr.arpa 845 2.4
192.168.60.253 com 833 2.4
192.168.125.201 my 697 2.0
.....

There are many other options which you can figure it out yourself and this is really great tool for dns traffic analysis. For more information, you can find at its home page -

http://dns.measurement-factory.com/tools/dnstop/


Btw, I like the use of the word 'telemetry' here and enjoy some dns voodoo .....

Cheers (;])

Thinking: Enumerating Goodness & Security Through Obscurity

I read about this and this.

Enumerating Goodness has its own weakness though, the mimic attacks that look legit will create false negative and be categorized as Goodness.

Security through obscurity is not a great idea and to certain people it sounds dumb as if the application/software that you are trying to protect is vulnerable, you will be eventually exploited, however it does asist for defensive side because it helps to prevent against automated tools and also requires offensive side to perform more steps to achieve what they are trying to do and this leaves more footprints to be examined and traced by the defensive side. It can be considered as early warning of its kind if you know how to make use of it.

This is just my personal thought and feel free to discuss about the topic. There's no perfect model or principle.

Monday, April 28, 2008

Export Cisco NetFlow in Single Screenshot

I'm digging into network flow analysis for a while. The single screenshot below is to show how to export Cisco Netflow version 5 to the flow monitor at 172.16.1.55, simple and straightforward. The router model is Cisco 7200 series.


Note to myself.

Cheers ;]

Tuesday, April 22, 2008

Malaysia Tourism, visit the malwares!!!!!

If you are security.org.my reader, you may find the latest post about Malaysia Tourism Website is serving malwares to everyone. This is not big surprise as Malaysia Internetwork is largely contributing to malwares distribution globally according to Google.

So this "Malaysia Can" spirit continues, we are distributing malwares to people around the world efficiently. I'm wondering how much our lovely government has spent to setup and maintain the web sites, let us know if you have clue.

Enough crap, lets look at the term of usage in the Tourism Malaysia website -

All the information contained herein is correct at the time of publication. Whilst every effort has been made to ensure the accuracy of the contents, Tourism Malaysia shall not be held liable for any errors, omissions or inaccuracies which may occur.

That's great, don't blame them as they have thrown the best effort!

Now time to do some analysis by looking at these two scripts -

http://www.852599.cn/mp3/list.htm
http://www.852599.cn/mp3/MZ.htm

The MZ.htm is the java script, here's the content -

This looks like the black hat SEO technique that quite commonly used by China and Russia hackers which we have seen to increase the counter stat, and of course the charset does tell the story here. On the other hand, the file list.htm doesn't look to be so straight forward to understand -

I truncated the line t="68,105,109....." so that it can be read properly here. The list.htm is VBScript and that's why it won't work on other browsers except Internet Explorer. If you look at the last line, it calls MZ.htm using iframe technique. In order to know what this VBScript, we have to de-obfuscate it and the easiest way is to replace the function execute to msgbox so it looks like this -

msgbox(rechange(t))

Then load the file to Internet Explorer and viola, check out the screenshot below -


Now we can find the location of the malware -

http://www.852599.cn/mp3/setup.exe

I downloaded the executable file and tried to scan it using ClamAV with no result, therefore I uploaded it to Virustotal to scan through different Antivirus Softwares. The result is decent, out of 31 types of Antivirus Software, only 12 recognize it as malware. If any of users who are using those Antivirus Softwares that still don't detect it, congratulations! The result of scanning is here.

I would like to thank to my friend - tehtb who is the decent Windows Programmer, I was silly to tell him maybe we can use print function and he pointed me out msgbox function works well here.

Enjoy (;])

Sunday, April 20, 2008

Google Http Redirection

Thanks to Google, you can now redirect anyone to yahoo.com from google.com. Click below -

Google to Yahoo

Basically you can redirect to any site you want, the http request looks like this -

http://www.google.com/pagead/iclk?sa=l&ai=FnzbwS&num=67575&adurl=
http://www.yahoo.com

I think Google doesn't verify ai= or num= field, you can just put whatever input to it and still get it parsed properly, for example -

http://www.google.com/pagead/iclk?sa=l&ai=abcdef&num=12345&adurl=whatverURL

This is the trick from one of the spam I received today which redirects me to download malwares and it still works now, hopefully Google fixes it as soon as possible.

Peace ;]

HITB Dubai 2008 Wrap Up

First of all, thanks to HITB crews, sponsors, speakers and all the attendees to make this happen again. We also thank to Dhillon(HITB Founder) for giving us(me and spoonfork) opportunity to present this time. Lets talk about some craps first -

1. Dubai hotel is expensive!
2. The hotel foods are always the same!
3. Pizza express is nice!
4. We always have great support from Army!
5. Dubai Gold Souk has too much golds!
6. Rufio and Niresh are upset because of .....!
7. I met all the cool people there!
8. The conference party is real fun because we get to hang up with beers!

Here's the sum up of all the presentations that I have attended -

Bruce Schneier
Schneier On Security
Bruce talks about the distance between the feeling and reality of security. I'm not really a guy who emphasizes on conceptual or semantic, however I think people will never get it right between the believe and reality, the line is blur because you can never educate the masses about the technologies that keep evolving from time to time.

Jeremiah Grossman
Hacks Happen
Jeremiah is statistics man, and of course the numbers tell you the changes of threat landscape and that's not really big surprise. He concluded all the quantitative data from different sources to give you macro view of current vulnerabilities and threats state. The web attack is popular as it becomes important business platform, remember the popularity brings paparazzi. And again we can't really defend ourselves from "just surface or 1st day' attack(I don't call it 0 days as the technique is already used) because we don't understand them. The summary is pretty simple, hacks happen .... all the time. That guarantees we won't be jobless too ;P

Christopher J.Rouland
Advances in Intrusion Detection and Prevention
I don't know Christopher personally, I attended his talk because the topic has something to do with intrusion detecion, seriously there's nothing really new or advance if you are following cutting edge intrusion detection technologies for long time. Most of the time Christopher is just explaining what IBM ISS offers and they are better than other vendors in comparison. Their system also emphasizes on session data by importing NetFlow data and we can't consider it as new stuff(think NSM). We are seeing more and more vendors taking session data seriously now.

Alessio 'mayhem' Pennasilico
VoIP (in)Security - Italians Do It Better
Alessio talked about his experience of making fun with voip system during CCC in Italy, I like his term of spitting, you should check out his presentation slide once it is up online. I would say fear the embassy who can reach you anytime, that's really entertaining talk from Mayhem. He also demonstrated the device which can change the voice at the end of the talk.

Marc Weber Tobias
How We Cracked Their Codes: A Case Study in Compromising the Most Popular High Security Lock in America - Medeco m3 and Biaxial
I'm not into lock domain or I know nothing about it at all, however I'm really impressed with Marc's work to crack the lock, and respect his 40 years passion in the same domain. I'm just 5 years old in network security industry and hope I can keep my passionate to network security field like he does.

Cesar Ceurrudo
Token Kidnapping
Cesar is the CEO of ArgenISS who sponsored our conference party, he showed us the weakness of design flaw in latest windows technology by kidnapping the token of particular process threads and you can gain the access priviledge to do anything you want. This is not something that can be fixed easily and therefore he has informed Microsoft before the presentation. If you want to know more about it, feel free to check out his presentation slides.

Walter Goulet, Viviana Basso and Benjamin Hagen
Real World Attacks Against 3G Networks Using Subscriber Devices
This is the talk from Motorola guys, they discuss their experience of penetration testing for 3G Networks, their conclusion is pretty simple, most of the security issues derive from the deployment phase where the party that setting up the network has to meet the timeline and leave most of the devices with default setting. As long as they are connected to the IP networks, you can basically probe them easily.

Alexander Kornbrust
Practical Oracle Forensics
I'm not database expert and don't know anything about Oracle at all(I use MySQL or PostgreSQL), I like Alexander's formalized approach to perform forensics process on Oracle Database System, he classifies different kind of malicious attackers with different kind of purposes and attentions, by knowing what kind of malicious attackers you are dealing with, you know what kind of sql statement will be queried by them, and that's really good lead to figure things out. Alex is really a bright guy and he shares his stuffs, he has written the scripts to retrieve all the necessary information to find culprits and evidences from the database system. I'm wondering if similar approach can be applied to MySQL and PostgreSQL and maybe should start looking up on this.

Petko D Petkov
For My Next Trick… Client-Side Hacking
Adrian 'pagvac' Pastor
Cracking into Embedded Devices and Beyond!
Pdp and Adrian are both thinktankers from Gnucitizen, I'm one of their blog readers(as always you can't forget about offensive security techniques while you are in defensive side), they are concluding their works of client side hackings and embedded devices cracking. If you are their blog reader, you should be familiar with all the stuffs they are sharing during the presentation, I like their single slide explanation where they only show the meat of particular hacking trick so that you can understand them easily. I think they are trying to deliver the mindset of "hacking through simplicity" and it will just work most of the time.

Mel(Spoonfork) and I presented Defensive Network Security, this is the first time we deliver our talk in Dubai, I hope whoever have attended our talk enjoy it and thanks to the 22 years old Amy for not sparing even 5 minutes.

I miss some of the presentations as both presentation tracks were running at the same time, anyway I will grab their presentation slide once they are online. Overall I enjoy the HITB conference in Dubai this time(congrates to Rufio as the new CTF overlord this time, hopefully you will suffer for another few years).

This conference party was organized on the cruise, and I bet everyone was enjoying the alcohols after "long fasting". See you guys in next HITB Conference again!!!!!

Cheers (;])

Tuesday, April 15, 2008

HeX is 1 year old now

Me and Chfl4gs_ decided to start the development of HeX System since mid of April 2007, and it's much improved with the involvement of other active members. Today it turns one year old.

Happy birthday to HeX, and thanks to the raWPacket team members to make it happen.

Happy packetysis!!!!!

Enjoy (;])

Saturday, April 12, 2008

Tuesday, April 01, 2008

Sguil: Excitement!!!!!

As the active member of NSM community, I got the first hand information about the acquisition of Sguil by Cisco. My friend Hanashi hits it first in his blog. Here's the full announcement -

Cisco Announces Agreement to Acquire SguilTM Open Source Monitoring Project Acquisition Furthers Cisco’s Vision for Integrated Security Products

SAN JOSE, Calif., and LONGMONT, Color., April 1st, 2008 – Cisco and the SguilTM project today announced an agreement for Cisco to acquire the SguilTM project, a leading Open Source network security solution. With hundreds of installations world-wide, SguilTM is the de facto reference implementation for the Network Security Monitoring (NSM) model. SguilTM-based NSM will enable Cisco’s customer base to more efficiently collect and analyze security-related information as it traverses their enterprise networks. This acquisition will help Cisco to cement its reputation as a leader in the Open Source movement while at the same time furthering its long-held vision of integrating security into the network infrastructure.

Under terms of the transaction, Cisco has acquired the SguilTM project and related trademarks, as well as the copyrights held by the five principal members of the SguilTM team, including project founder Robert "Bamm" Visscher. Cisco will assume control of the open source SguilTM project including the Sguil.net domain, web site and web site content and the SguilTM Sourceforge project page. In addition, the SguilTM team will remain dedicated to the project as Cisco employees, continuing their management of the project on a day-to-day basis.

To date, SguilTM has been developed primarily in the Tcl scripting language, support for which is already present inside many of Cisco’s routers and switches. The new product, to be known as “Cisco Embedded Monitoring Solution (CEMS)”, will be made available first in Cisco’s carrier-grade products in 3Q08, with support being phased into the rest of the Cisco product line by 4Q09. Linksys-branded device will follow thereafter, though the exact deployment schedule has yet to be announced.

“We’re extremely pleased to announce this deal,” said Cisco’s Chief Security Product Manager Cletus F. Simmons. “For some time, our customers have told us that our existing security monitoring products did not extend far enough into their network infrastructure layer. Not only was it sometimes difficult to intercept and monitor the traffic, but there were often political problems at the customer site with deploying our Intrusion Detection Systems, as management had heard several years ago that they were ‘dead’. Now, with SguilTM integrated into all their network devices, they’ll have no choice!”

Although the financial details of the agreement have not been announced, SguilTM developer Robert Visscher will become the new VP of Cisco Rapid Analysis Products for Security. “This deal means a lot to the SguilTM project and to me personally,” Visscher explains. “Previously, we had to be content with simply being the best technical solution to enable intrusion analysts to collect and analyze large amounts of data in an extraordinarily efficient manner. But now, we’ll have the additional advantage of the world’s largest manufacturer of networking gear shoving it down their customers’ throats! We will no longer have to concern ourselves with mere technical excellence. Instead, I can worry more about which tropical island to visit next, and which flavor daiquiri to order. You know, the important things.”

About Cisco Systems
Cisco, (NASDAQ: CSCO), is the worldwide leader in networking that transforms how people connect, communicate and collaborate. Information about Cisco can be found at http://www.cisco.com. For ongoing news, please go to http://newsroom.cisco.com.

About SguilTM
SguilTM is the leading Network Security Monitoring (NSM) framework. It is built for network security analysts by network security analysts. Sguil’s main component is an intuitive GUI that provides access to a wide variety of security related information, including real-time IDS alerts, network session database and full packet captures. SguilTM was written by Robert “Bamm” Visscher, who was apparently too cheap to buy a book on Java or C.

With this deal, Sguil will go into active and sharp development very soon and kudos to the Sguil team, I would say Cisco has made the best decision ever!!!!!

I bet the stubborn of using TCL works this time .....

Enjoy (;])

Saturday, March 29, 2008

Pads: Nginx Sig

I read about misuse of nginx web/proxy server in Emerging Threats here. Jonkman has added the snort signature for it but I think I can write the Pads signature for it. After some testing it seems I have the signature written and works pretty well. Here it is -

www,v/Nginx/$1//,\x0d\x0aServer: nginx\/([\S]+)\x0d\x0a

I added this signature to pads-signature-list, then run Pads -

shell>sudo pads -i le0 -n 192.168.1.0/24 -c pads.conf
pads - Passive Asset Detection System
v1.2 - 06/17/05
Matt Shelton

[-] Filter: (null)
[-] Listening on interface eth1

[*] Asset Found: IP Address - 192.168.1.100 / MAC Address - 0:1B:77:5B:F4:3F
[*] Asset Found: Port - 80 / Host - 192.168.1.100 / Service - www / Application - nginx/0.5.26

It able to track the version of nginx that I run as well, this is pretty useful if you discover there's nginx server running in your network. One interesting thing I have figured is that nginx supports gzip encoding so you can't see the web contents in plain text and this maybe one of the reason why it is used by the blackhats.

Cheers (;])

Friday, March 28, 2008

Sancp: Pcap Index

We have been having discussion in #snort-gui regarding pcap indexing lately which will allow us to retrieve the network data based on the index quickly. I have learned about this feature during the use of time machine which will be integrated with Bro-IDS, I especially like their time machine framework which is illustrated here.

Sancp 1.6.2 beta has quite a few new features, one of them is actually in discussion I mentioned above - pcap index. We are actually looking forward for pcap index so that we can retrieve the pcap data based on index instead of doing it manually with tcpdump or other tools. Hopefully the future sguil will support this so we can just right click and obtain the pcap from sancp record.

I was not so sured how to do it with sancp when I took the first look at 1.6.2 beta, after emails exchange with my friend John, I finally have the idea of how to do it. Here's the simple howto -

shell>mkdir sancp-output

shell>echo "default index log" > sancp-pcapindex.conf

shell>echo "format index delimiter=| sancp_id output_filename start_pos stop_pos src_ip_dotted dst_ip_dotted ip_proto src_port dst_port" >> sancp-pcapindex.conf

shell>sancp -i eth0 -c sancp-pcapindex.conf -d sancp-output

The index file is created in sancp-output along with the pcap data. Basically the index file looks like this -

5177639571328195257|sancp-output/pcap.eth1.1205513154|24|114|
192.168.1.101|121.35.19.76|6|51570|80
5177639571328196206|sancp-output/pcap.eth1.1205513154|114|204|
192.168.1.101|86.1.191.36|6|50397|13322
5177639571328204971|sancp-output/pcap.eth1.1205513154|204|294|
192.168.1.101|219.77.117.51|6|35860|6881
5177639571328205574|sancp-output/pcap.eth1.1205513154|294|384|
192.168.1.101|219.90.206.228|6|53729|55265
5177639571328206646|sancp-output/pcap.eth1.1205513154|384|474|
192.168.1.101|76.122.33.223|6|45014|22100
5177639571328208237|sancp-output/pcap.eth1.1205513154|474|1984|
220.140.112.45|192.168.1.101|6|20315|60025
Output truncated ...

To extract the pcap based on its index, you can use getpcapfromsancpindex.pl which is also written by Jlcurry. Download it first -

shell>wget http://metre.net/file/getpcapfromsancpindex.pl

Now you can grep the index and extract its corresponded data by executing -

shell>fgrep "5177639571328208237|sancp-output/pcap.eth1.1205513154|474|
1984|220.140.112.45|192.168.1.101|6|20315|60025" sancp-output/index | ./getpcapfromsancpindex.pl 220-140-112-45


Lets examine the file -

shell>file 220-140-112-45
220-140-112-45: tcpdump capture file (little-endian) - version 2.4 (Ethernet, capture length 65535)

Read it with tcpdump -

shell>tcpdump -ttttnnr 220-140-112-45
reading from file 220-140-112-45, link-type EN10MB (Ethernet)
2008-03-15 00:45:54.396653 IP 220.140.112.45.20315 > 192.168.1.101.60025: . 4279573104:4279574532(1428) ack 1270973978 win 65535

John(Jlcurry) has put up his own note of how to use getpcapfromsancpindex.pl here -

http://metre.net/getpcapfromsancpindex.html

You can also read the interesting interview of me with John about Sancp here.

Enjoy (;])

FreeBSD: Updating ports with csup

I learned about csup during the development of HeX, thanks to chfl4gs_ for that. Now we no longer need to use cvsup to update the source or software packages as csup is the replacement of cvsup which is written in C. In fact, I feel that it is much faster after testing it.

To update the FreeBSD ports, I just need to do the following

shell>cp /usr/share/examples/cvsup/ports-supfile /usr

Edit the ports-supfile by adding the fastest mirror for you(remember nearest is not always the fastest), I choose Taiwan mirror here -

*default host=cvsup.tw.FreeBSD.org

Now run the csup to update the ports

shell>csup -L 2 /usr/ports-supfile

Once the update is done, you will read Finished successfully at the end of message.

Peace ;]

USM: Industry Talk

This Monday, I was glad to be able to present at University Science Malaysia. I always have good feeling when back to Penang and thanks everyone for coming. Our main audience is student so we try to present it in light manner.

I presented about -

- Dive Into Infosec Industry
- HeX System Introductory

And Mel(Spoonfork) presented

- Honeynet Foundation
- Malaysia Distributed Honeynet Project

Mel was with me and hopefully he enjoyed the trip to the North. Thanks to Sara as he linked us up with the USM. We also thank to CW(one of my friend) who has responsible for the arrangement here.

On the other hand, we just knew that this week is ICT week in USM.

Cheers ;]

Wednesday, March 26, 2008

Sguil 0.7.0 Unleashed

After longggggg period, finally Sguil 0.7.0 is here! Check it out at

- http://sguil.sourceforge.net/news.html

Thanks to the hard work of all the #snort-gui NSM gang!

Enjoy (;])

Sunday, March 23, 2008

Ubuntu: Unicornscan Installation

I hate to remember simple thing these days, and the only way to prevent myself to remember simple thing is to write it down so I can refer later. This is Unicornscan installation steps on Ubuntu 7.10, I hope it helps someone if he/she want to try out Unicornscan on Ubuntu. If you don't know what Unicornscan is, here's the description from the Unicorn site -

Unicornscan is a new information gathering and correlation engine built for and by members of the security research and testing communities. It was designed to provide an engine that is Scalable, Accurate, Flexible, and Efficient.

Lets start the installation -

shell>sudo apt-get install libpcap0.8-dev

shell>sudo apt-get install libgeoip-dev

shell>sudo apt-get install libltdl3-dev

shell>wget \
http://www.unicornscan.org/releases/unicornscan-0.4.7-2.tar.bz2

shell>tar xvjf unicornscan-0.4.7-2.tar.bz2

shell>cd unicornscan-0.4.7-2

shell>./configure --prefix=/usr/local/stow/unicornscan-0.4.7-2

shell>make

shell>sudo make install

Simple as it is, time to dive into Unicornscan. In fact they have blog now.

Peace ;]

SinFP: Reading the Signatures

I learned about SinFP from when I first read it in taosecurity blog here, then today I read that it is integrated into Nessus in tenable blog here. Therefore it raises my interest to look at it. Basically SinFP is available in HeX, instead of trying its features, I'm particularly curious about its signature database. I download the latest signature database from its home site -

shell>wget http://www.gomor.org/files/sinfp-latest.db

My habit is using file command to identify what file type it is before openning, and I'm surprised it is not in plain text format like other tools, it's the SQLite 3.x database file.

shell>file sinfp-latest.db
sinfp-latest.db: SQLite 3.x database

Now this is killing me, I'm worse than beginner and totally have no experience in SQLite. I go to its main site and figure there are many big companies using it and it tells how popular it is here. So I need to try it out, the manual really helps but I don't want to read it for casual use, so I need to write it down for my reference in my blog. Below are what I have gone through.

To show the current default setting value -

shell>sqlite3 sinfp-latest.db ".show"
echo: off
explain: off
headers: off
mode: list
nullvalue: ""
output: stdout
separator: "|"
width:

To list the name and file of the database -

shell>sqlite3 sinfp-latest.db ".databases"
seq name file
--- --------------- ----------------------------------------------------------
0 main /home/geek00l/Desktop/sinfp-latest.db

To show all the tables in the database -

shell>sqlite3 sinfp-latest.db ".table"
IpVersion OsVersionFamily PatternTcpOptions Vendor
Os PatternBinary PatternTcpWindow
OsVersion PatternTcpFlags Signature
OsVersionChildren PatternTcpMss SystemClass

Then select everything from the table Os, you can see what OS is available -

shell>sqlite3 sinfp-latest.db "select * from Os"
1|FreeBSD
2|Linux
3|NetBSD
4|OpenBSD
5|SunOS
6|Windows
7|IOS
Output truncated ...

You can also select everything from multiple tables at the same time with comma, I pipe it to less for navigation purpose -

shell>sqlite3 sinfp-latest.db "select * from Os ,OsVersionFamily, Vendor, PatternBinary, Signature;" | less

You can also dump the signature into raw file and examine it later since it is in plain text form, here I dump it to sinfp-raw.sig file -

shell>sqlite3 sinfp-latest.db "select * from signature" > sinfp-raw.sig

The sqlite3 command line is quite flexible, you can turn on or off its setting in command line directly, as I want to see the header(title of the column) and table in column form, I add -header and -column when running it, for the sql syntax, you can see I use limit 5, this is quite basic stuff for experience SQL database administrator -

shell>sqlite3 -header -column sinfp-latest.db "select * from Os limit 5"
idOs os
---------- ----------
1 FreeBSD
2 Linux
3 NetBSD
4 OpenBSD
5 SunOS

You can also turn it into CSV format using -separator ',' -

shell>sqlite3 -header -separator ',' sinfp-latest.db "select * from Os, Signature limit 5"
idOs,os,idSignature,trusted,idIpVersion,idSystemClass,idVendor,idOs,
idOsVersion,idOsVersionFamily,idP1PatternBinary,idP1PatternTcpFlags,
idP1PatternTcpWindow,idP1PatternTcpOptions,idP1PatternTcpMss,
idP2PatternBinary,idP2PatternTcpFlags,idP2PatternTcpWindow,
idP2PatternTcpOptions,idP2PatternTcpMss,idP3PatternBinary,
idP3PatternTcpFlags,idP3PatternTcpWindow,idP3PatternTcpOptions,
idP3PatternTcpMss
1,FreeBSD,1,1,2,1,1,1,1,1,1,1,1,1,1,1,1,1,2,1,2,2,2,3,2
1,FreeBSD,2,1,2,1,1,1,3,2,1,1,1,1,1,1,1,1,4,1,2,2,2,3,2
1,FreeBSD,3,1,2,2,1,2,5,3,3,1,3,1,1,3,1,4,5,1,2,2,2,3,2
1,FreeBSD,4,1,2,2,1,2,7,4,3,1,3,1,1,3,1,4,6,1,2,2,2,3,2
1,FreeBSD,5,1,2,1,1,3,9,5,3,1,5,1,3,3,1,5,7,3,4,2,2,3,2

If you want to learn how the data is inserted to the database, using .dump to dump them in SQL text format. I figure this is useful especially to beginner like me -

shell>sqlite3 sinfp-latest.db ".dump Os"

shell>sqlite3 sinfp-latest.db ".dump Signature"

Of course this is not something I'm really good at, but learning something new is always fun, I think if you are SQL database administrator or developer, you can take advantage of SQLite quickly. I learn MySQL while using Sguil and now I take a brief look at SQLite while trying out Sinfp, all I can say is you always learn something else.

Enjoy ;]

Thursday, March 20, 2008

raWPacket: More Papers

While discussing with Hol about fl0p, I came out the idea of writing this paper - Basic Fl0p Signature Writing Guide to compliment his paper which I will put up soon. If you don't know what is fl0p, check out its description from Michal Zalewski himself -

Fl0p is a passive L7 flow fingerprinter that does not examine packet payloads, only their relative sizes, the sequence of client-server traffic, and its timing. The tool can be thus used to peek into encrypted tunnels, automatically tell users from robots, and far more.

On the other hand, dakrone has also published his first paper - An Introduction To NSM Console. The paper will cover the underlying concept of NSM Console and its functionalities.

I hope all the raWPacket team members can contribute more papers so we have richer documentation to share with the world as we have benefit from others papers too.

Enjoy ;]

Wednesday, March 19, 2008

Argus 3: Tips & Tricks

I have labeled all my argus 3 blog post to make it easy for people who are interested in it. You can find them at -

http://geek00l.blogspot.com/search/label/Argus3

If you are using argus, and would like to know how to use certain client tools and you think it might be useful, feel free to email me as well and maybe I can put up the tips and tricks for it.

Peace ;]

Monday, March 17, 2008

HITB Conference and Training in Dubai

Again this time, me and mel(spoonfork) will be going to conduct Structured Network Threat Analysis and Forensics training in HITB Conference Dubai. The training is more of how you can perform packet examination, analysis and threat detection using various Open Source tools with effective methodologies, we are going to use HeX liveCD for the training if you ask me.

Besides I hope to meet speakers and UAE folks who are involved in network security operation, that's going to be fun.

I'm now thinking of developing another training contents, most properly about network flow analysis and how can you apply it in real world network deployment. What do you think?

Cheers ;]

Sunday, March 16, 2008

Argus 3: Audit Trail with HGWL

I have discussed about the Harimau Global Watchlist here, initially one of the reason Spoonfork developes HGWL is to put all the malicious IP addresses in one place so that it can be used with argus. In fact it can be used with any network flow technology but I'm going to show how you can use HGWL and argus 3 to perform trail auditing.

Obtaining the Harimau Global Watchlist -

shell>wget http://watchlist.security.org.my/all.txt

Generate IP addresses list from argus data(shit.arg3) -

shell>racluster -M rmon -m saddr -r shit.arg3 -s saddr - ip > IP.list

I use -M rmon to convert argus biflow data to uniflow, the source and destination doesn't matter now, with that I just need to specify it to show source address by using -s saddr and I will be able to obtain all the IP address. The -m saddr is used here to cluster source address field only so the same IP is only shown once.

Filtering out suspected IP addresses -

shell>for host in `cat IP.list`; \
do egrep $host all.txt >> Suspect.list; done

61.187.72.70, http://www.dshield.org/ipsascii.html, Dshield: Top IPs, dshield-top-ips, 2008/03/16 00:00:08
61.193.128.171, http://www.dshield.org/ipsascii.html, Dshield: Top IPs, dshield-top-ips, 2008/03/16 00:00:08
65.160.238.180, http://www.dshield.org/ipsascii.html, Dshield: Top IPs, dshield-top-ips, 2008/03/16 00:00:08
66.45.254.244, www.emergingthreats.net/rules/bleeding-rbn.rules, ET RBN Known Russian Business Network Monitored Domains (15), rbn, 2008/03/16 00:03:31
195.161.8.1,http://www.senderbase.org/home/detail_spam_source,senderbase-spam-source,2008/03/16 00:28:34
202.58.85.15, http://www.dshield.org/ipsascii.html, Dshield: Top IPs, dshield-top-ips, 2008/03/16 00:00:08
217.175.134.217, http://www.dshield.org/ipsascii.html, Dshield: Top IPs, dshield-top-ips, 2008/03/16 00:00:08
218.30.19.40, http://www.dshield.org/ipsascii.html, Dshield: Top IPs, dshield-top-ips, 2008/03/16 00:00:08
221.6.4.67, http://www.dshield.org/ipsascii.html, Dshield: Top IPs, dshield-top-ips, 2008/03/16 00:00:08
221.237.161.186, http://www.dshield.org/ipsascii.html, Dshield: Top IPs, dshield-top-ips, 2008/03/16 00:00:08

Most of the time we start doing audit trail when IDS is giving us pointer. Now we can start querying the IP addresses in Suspect.list from argus data when there's matching and tracing its network activities. For example -

shell>ra -nr data.arg3 - host 195.161.8.1
StartTime Flgs Proto SrcAddr Sport Dir DstAddr Dport TotPkts TotBytes State
13:00:58.918872 e tcp 195.161.8.1.10424 -> 192.168.42.109.25 16 1559 RST
13:00:59.275570 e tcp 192.168.42.109.28457 -> 195.161.8.1.113 1 60 REQ
13:02:03.314641 e d tcp 195.161.8.1.24113 -> 192.168.42.109.25 8 698 RST
13:02:03.815631 e tcp 192.168.42.109.28493 -> 195.161.8.1.113 1 60 REQ
Output truncated ....

End for this.

The new client tool - rastream is included in argus 3 lately as argus data stream block processor, I'm thinking of using it to perform HGWL matching operation automatically once the argus data is generated so that we can catch near real time suspected host to watch possible ongoing malicious actions, maybe I will discuss about rastream in future blog post.

Updates: My friend scholar01 has shown me useful trick, instead of using for host in `cat IP.list`; do egrep $host all.txt >> Suspect.list; done, it is better to use fgrep -f IP.list all.txt as fgrep has advance engine for matching static strings at once. Thanks.

Enjoy (;])

Wednesday, March 12, 2008

Message to network device vendors

These days command line interface(CLI) becomes not so popular to manage network devices such as routers, switches and etc anymore, the replacement is the "not so shiny" web graphical user interface(GUI).

Basically I have no complaint about the usability of web GUI, it provides really easy to use interface for new users and good kickstart to manage the network devices especially when you are not familiar with specific network devices, but I really have one rant here. There are many vendors producing web GUI that only works well on certain web browser especially Internet Explorer, and there's no access through CLI, this is annoying to network administrators that are using open source softwares.

To the vendors, if you can't produce the web GUI that works across multiple web browser platforms, then just give us CLI(ssh preferred) and we can work it out instead of seeing your ugly GUI which is non-productive.

Thanks ;]

Sunday, March 09, 2008

raWPacket: New Members

We are now welcome two new members joining raWPacket team. They are -

- Hol(Rob Floodeen)

- Saravanan

Hol is contributing few interesting papers which I will put it up in rawpacket site soon. Saravanan is the maintainer for our own version of fl0p and he will also involve in Ular FVS project which lead by Zarul.

On the other hand, Guti who is our web master since day 1, will be involving in honeynet and HeX sensor project.

Kudos (;])

Saturday, March 08, 2008

FreeBSD ldconfig

In orde to configure the shared library cache for ELF format on FreeBSD, you can add all the directories that contain the shared libraries to /etc/ld-elf.so.conf.

shell>cat /etc/ld-elf.so.conf
/lib
/usr/lib
/usr/lib/compat
/usr/local/lib
/usr/local/lib/compat
/usr/local/lib/compat/pkg
/usr/local/lib/graphviz
/usr/local/lib/mysql
/usr/local/lib/nss
/usr/local/lib/pth
/usr/local/share

Then run -

shell>ldconfig -elf /etc/ld-elf.so.conf

Now you can check the hints file -

shell>strings /var/run/ld-elf.so.hints
Ehnt
/lib:/usr/lib:/usr/lib/compat:/usr/local/lib:/usr/local/lib/compat:
/usr/local/lib/compat/pkg:/usr/local/lib/graphviz:
/usr/local/lib/mysql:/usr/local/lib/nss:
/usr/local/lib/pth:/usr/local/share


All of them are added now. If you have shared object missing or not found error when you launch certain application next time, please add the directories that containing its shared library in /etc/ld-elf.so.conf and follow the steps above will do.

Peace ;]

Friday, March 07, 2008

HeX BitTorrent

Enhanced is kind enough to put up HeX in his Security Torrent Depot which you can find at -

- http://www.redsphereglobal.com:88/

Instead of using http/ftp mirror, now you can download HeX using BitTorrent. If you want to contribute to HeX project, the easiest way is seeding ;).

Enjoy ;]


Sunday, March 02, 2008

Updates

Idle is bad for a blogger, here's the update for most of the things I have been following -

- FreeBSD 7 is released

- Wireshark 0.99.8 is released

- Ourmon 2.8 is released

You can find the FreeBSD 7 release note here and interesting interview here to study what are the features and fixes offered by them. For wireshark, read this one. I'm particularly interested in the ntar(next generation pcap) and rpcap support. Last but not least, check out the new features that are included in Ourmon 2.8 here! It is now supporting threads and people with multiple cpus(quad core!) most likely will benefit from it, I'm also looking into blacklisting feature and maybe it can integrate with our Harimau Watchlist.

I'm busy with my personal thingy so I won't be very active until mid of March, anyway with the release of FreeBSD 7, HeX 2.x engineering begins!

Cheers ;]