Best for Last .....

Today is a busy day as I'm packing everything and finally moving out from the Penang Island. I'm glad to meet my former colleague who just back from US, and playing games with my friends. I can't consider it as relaxing day for me as I'm rushing here and there to finish all the stuffs possible. I will be heading to Kuala Lumpur tomorrow and taking flight to this event -

http://conf.vnsecurity.net/

Guess what, with this post I'm hitting 35 in a month which means more than a blog post in a day. Insane!!!!!

Peace ;]

HeX liveCD: CD Label

Our HeX liveCD great graphic designer - Vickson has sent me the CD label that he has just redesigned and asking my opinion about it. This is almost the final version of the CD label for HeX liveCD version 1.0 after some discussion and modification ongoing between me and some of team members, I would like to put it up here and sharing with everyone. To protect his artworks, I have told him to register under Creative Common License.For better view, click on the picture to zoom in, let us know what do you think about it?

Awesome works, Vickson!

Enjoy (;])

Bro: It's not just NIDS

Thanks to Bro team which has published the workshop materials online so that we can learn more about Bro internal. While I have been using Bro in Operational Network, I haven't actually learned about Bro Scripting Language. Therefore I think those materials will be ice-breaker for me because it really makes thing clear and I plan to finish all the exercises to further sharpen my skills in operating Bro.

http://www.bro-ids.org/bro-workshop-2007/agenda.html

There are few things I have noted in the materials provided, in the slide 6 of Bro Overview -

Much of the system is policy-neutral
- i.e. no presumption of "good" or "bad"

This is exactly similar to NSM concept where we don't assume any alert events provided by IDS as intrusion or extrusion without further confirmation and verification with the subsystem given.

From the slide 4 of Bro Conclusion and Outlook -

The Bro Cluster
- A set of PCs running Bro jointly analyze large network streams
- A central manager system

And 5th slide -

Multi-Core Support
- Going to turn Bro into multi-threaded application
- Will fully exploit the multi-core potential of modern CPUs

I would love to see all of them integrated to Bro in near future as it will benefit us with both low or high end hardwares.

Again from the slide 4 of Bro Conclusion and Outlook -

New Functionality
- Time Machine Interface
- Netflow Analyzer

I have written the setup of time machine here previously and time machine will soon be integrated with Bro to provide full content data via its indexing system. If you want to learn more about it, check out its main site -

http://www.net.t-labs.tu-berlin.de/research/tm/

With Netflow Analyzer added that is plus point as most of the companies having Cisco router deployed in their network, or you can use fprobe if running *nixes network appliance.

Think again about the approach, is Bro a Network Intrusion Detection System?

Again I have to emphasize, NSM analyzt is da future!

Peace ;]

Malaysia Honeynet Community

Me and Mel have discussed about running non-profit honeynet group to research on internet threats and attacks trend for a while, and finally Mel launched the website about the group after long delay. We are currently running low level honeypots to capture malwares and plan to deploy high level honeynet once we have found the sponsors because more costs intact in order to operate them.

We welcome any contributors, here's our list -

http://my-honeynet.org/index.php?/archives/Contributions-to-the-Malaysia-Honeynet-Project.html

Current members in the group -

http://my-honeynet.org/index.php?/archives/Malaysia-Honeynet-Project-Members.html

And that includes raWPacket team members too.

Of course if you are interested in the project(researches, contributions, sponsorships), feel free to email us, you can either contact me or Mel.

mel at hackinthebox dot org

geek00l at gmail dot com

We hope this can be the long term R & D with the supports of various parties, and the outcome value is definitely worthwhile.

Cheers ;]

Bro-NIDS: Pairing with Ourmon

Beside Snort, Bro-NIDS is another one of the most powerful open source NIDS that developed by Vern Paxson and his team with community supports.

I figure out the lacking part of Bro, the detail explanation of analysis flow definitely. So what's the problem with that? Let's take a look at its logs -

shell>ls -la logs
-rw-r--r-- 1 bro wheel 196714 Jul 26 12:17 alarm.hostname.07-07-23_00.00.01
-rw-r--r-- 1 bro wheel 406878 Jul 23 17:33 alarm.hostname.07-07-23_00.00.01-07-07-23
-rw-r--r-- 1 bro wheel 142239 Jul 24 21:39 alarm.hostname.07-07-23_00.00.01-07-07-24
-rw-r--r-- 1 bro wheel 585678 Jul 25 22:39 alarm.hostname.07-07-23_00.00.01-07-07-25
-rw-r--r-- 1 bro wheel 258391385 Jul 23 00:00 conn.hostname.07-07-18_15.25.41-07-07-22
-rw-r--r-- 1 bro wheel 130996 Jul 23 00:00 conn.hostname.07-07-18_15.25.41-07-07-23
-rw-r--r-- 1 bro wheel 35500032 Jul 26 12:17 conn.hostname.07-07-23
-rw-r--r-- 1 bro wheel 100144755 Jul 24 00:00 conn.hostname.07-07-23_00.00.01-07-07-23
-rw-r--r-- 1 bro wheel 76778138 Jul 25 00:00 conn.hostname.07-07-23_00.00.01-07-07-24
-rw-r--r-- 1 bro wheel 83297924 Jul 26 00:00 conn.hostname.07-07-23_00.00.01-07-07-25
-rw-r--r-- 1 bro wheel 856239 Jul 23 00:00 ftp.hostname.07-07-18_15.25.41-07-07-22
-rw-r--r-- 1 bro wheel 603 Jul 23 00:00 ftp.hostname.07-07-18_15.25.41-07-07-23
-rw-r--r-- 1 bro wheel 765952 Jul 26 12:17 ftp.hostname.07-07-23
-rw-r--r-- 1 bro wheel 2371772 Jul 24 00:00 ftp.hostname.07-07-23_00.00.01-07-07-23
-rw-r--r-- 1 bro wheel 2529580 Jul 25 00:00 ftp.hostname.07-07-23_00.00.01-07-07-24
-rw-r--r-- 1 bro wheel 1646301 Jul 26 00:00 ftp.hostname.07-07-23_00.00.01-07-07-25
-rw-r--r-- 1 bro wheel 179461957 Jul 23 00:00 http.hostname.07-07-18_15.25.41-07-07-22
-rw-r--r-- 1 bro wheel 47554 Jul 23 00:00 http.hostname.07-07-18_15.25.41-07-07-23
-rw-r--r-- 1 bro wheel 77824 Jul 26 11:49 http.hostname.07-07-23_00.00.01
-rw-r--r-- 1 bro wheel 642726 Jul 24 00:00 http.hostname.07-07-23_00.00.01-07-07-23
-rw-r--r-- 1 bro wheel 627456 Jul 25 00:00 http.hostname.07-07-23_00.00.01-07-07-24
-rw-r--r-- 1 bro wheel 202697 Jul 26 00:00 http.hostname.07-07-23_00.00.01-07-07-25
Output truncated .....

Ouch! There are so many logs here and how should I start? Just like sguil where you must speak mysql pretty well to perform better analysis, you must speak shell scripting enough to perform smooth analysis flow on Bro logs. The power of Bro lies in it's protocol anomaly detection and therefore having different protocols logging in different files(prefix with its protocol) do make things clear. However you can't consider its logs as alert and indicator of malicious event(different approach comparing with snort) even though alarm, notice and weird logs do give some clues about suspected network event. Another problem should be when the file grows larger in a day, it is pretty hard to crawl the file to check on malicious event. For example look at one of the file -

shell>du -h \
conn.hostname.07-07-18_15.25.41-07-07-19_00.00.00
510M conn.hostname.07-07-18_15.25.41-07-07-19_00.00.00

It could be deep pain if you want to read them, don't you? Therefore it becomes unclear when one first learn about Bro and want to proceed to examine the logs but don't know where to start. Here I will give you some insight of how I perform analysis using Bro but I need another powerful tool to assist me, I vote ourmon!

Bro only generates network statistical data in its daily report therefore we don't have real time view of current network statistic, therefore ourmon plays the main role for you to access to the network statistical data via its well crafted graph. But where's our flow data to understand each connection? Bro does store connections flow data, all its connections flow stored in the file with the prefix of conn.*. The only data we don't have is full content(pcap), you can either use tcpdump, daemonlogger or Bro alone to do the job for you however I would like to ignore this part for the moment because generally you can have idea of each connections because most of the widely used and popular protocols such as http, smtp and so forth are decoded and stored in its corresponding file such as http.*, smtp.* and so forth, other protocols that are not decoded(provided no decoder for it) still can be seen in conn.* to understand its nature. Here's the example when you find unconfirmed or suspected malicious activity in certain time frame via notice log -

notice.sguard.07-07-23_00.00.01-07-07-25_00.00.00:t=1185314110.984062 no=ProtocolFound na=NOTICE_FILE sa=1.2.3.4 sp=33071/tcp da=10.0.0.1 dp=443/tcp num=14 msg=1.2.3.4/33071\ >\ 10.0.0.1/https\ FlashCom\ (via\ HTTP)\ on\ port\ 443/tcp sub=FlashCom\ (via\ HTTP) tag=@2113

Interesting connection from 1.2.3.4 to 10.0.0.1, you have the tag number @2113 for this connection flow, now we can grep the flow from connections log -

shell>egrep '@2113' \
conn.sguard.07-07-23_00.00.01-07-07-25_00.00.00 | cf
Jul 25 05:55:05 122.708121 1.2.3.4 10.0.0.1 https 33071 443 tcp 13384 12298 RSTR L %332 @2113 HTTP

You may see I pipe to cf, cf is small utility to translate unix epoch time format to human readable form, therefore it's more easy for you to read and correlate the event based on time frame. The format of connections flow should be read in this way -

| Start | Duration | Local IP | Remote IP | Service | Local PORT | Remote PORT | Protocol | Orig Bytes Sent | Res Bytes Sent | State | Flags | Tag |

The only part which is not clear is the connection state - RSTR, you can check out here -

RSTR Established, responder aborted.

While L means connection is initiated locally.

There are two tags, %332 and @2113, so now we know about the connection flows which is http, we should search the tag %332 in http log -

shell>egrep '%332' \
http.hostname.07-07-23_00.00.01-07-07-25_00.00.00

1185314105.742038 %332 (200 "OK" [10])
1185314107.122409 %332 POST /idle/315397832/2 (200 "OK" [67] ic.flashcom.peopleim.userplane.com:443)
1185314107.750136 %332 POST /send/315397832/4 (200 "OK" [1260] ic.flashcom.peopleim.userplane.com:443)
Output truncated .....

Here you go, you are now having full understanding of the network connections(all those http requests with same tag number), this is kind of batch analysis to identify network events. I don't explain how I make use of ourmon here or maybe I will make it for future post. Ourmon generates the graph by plotting such as TCP flags ratio, network errors, irc stats and some other graphs that you can correlate with the data provided by Bro to fully understand your network and readily countering any network threats.

Below I demonstrate some of the tips I use before I examine Bro logs, it's good catch if we want to read the separate kind of logs in timely manner, for example I want to check out the alarm log only, I just execute -

shell>ls -lac alarm*
-rw-r--r-- 1 bro wheel 0 Jul 23 00:00 alarm.hostname.07-07-18_15.25.41
-rw-r--r-- 1 bro wheel 3999 Jul 19 00:00 alarm.hostname.07-07-18_15.25.41-07-07-18_15.25.50
-rw-r--r-- 1 bro wheel 4515 Jul 20 00:00 alarm.hostname.07-07-18_15.25.41-07-07-19_00.00.00
-rw-r--r-- 1 bro wheel 4525 Jul 21 00:00 alarm.hostname.07-07-18_15.25.41-07-07-20_00.00.00
-rw-r--r-- 1 bro wheel 1551 Jul 22 00:00 alarm.hostname.07-07-18_15.25.41-07-07-21_00.00.00
-rw-r--r-- 1 bro wheel 1363 Jul 23 00:00 alarm.hostname.07-07-18_15.25.41-07-07-22_00.00.00
-rw-r--r-- 1 bro wheel 0 Jul 23 00:00 alarm.hostname.07-07-18_15.25.41-07-07-23_00.00.00
-rw-r--r-- 1 bro wheel 206776 Jul 26 13:50 alarm.hostname.07-07-23_00.00.01
-rw-r--r-- 1 bro wheel 406878 Jul 24 00:00 alarm.hostname.07-07-23_00.00.01-07-07-23_00.00.04
-rw-r--r-- 1 bro wheel 142239 Jul 25 00:00 alarm.hostname.07-07-23_00.00.01-07-07-24_00.00.00
-rw-r--r-- 1 bro wheel 585678 Jul 26 00:00 alarm.hostname.07-07-23_00.00.01-07-07-25_00.00.00

I use ls with -c switch so that I can sort them by file creation time, that way making us easy to check out any daily log we want by date especially searching for connections tag. In the alarm log, you may see the time stamp of every network event started with t=, that render cf tool unable to convert the time stamp from unix epoch time to human readable form correctly -

shell>head -2 alarm.hostname.07-07-23_00.00.01
t=1185379996.560573 no=ProtocolViolation na=NOTICE_ALARM_ALWAYS sa=1.2.3.4 sp=33762/tcp da=2.3.4.5 dp=25/tcp msg=1.2.3.4/33762\ >\ 2.3.4.5/smtp\ analyzer\ SMTP\ disabled\ due\ to\ protocol\ violation sub=reply\ code\ out\ of\ range tag=@4363
t=1185380009.544433 no=PortScan na=NOTICE_ALARM_ALWAYS sa=1.2.3.4 da=2.3.4.5 dp=443/tcp msg=64.86.142.65\ has\ scanned\ 50\ ports\ of\ 10.14.42.154 tag=@4364

You can use sed to remedy this issue -

shell>sed 's/^t=//g' alarm.hostname.07-07-23_00.00.01 | cf

Just discard t= and parse it to cf. Bingo! Piping to less command is also great because you can perform certain function such as search by regular expression, easy log navigation and so forth(vi style functions).

Both Bro and Ourmon provide you great context for most of anomaly network events, I have spent my time to figure out the better way to utilize them and hopefully this help anyone who uses them because I did face difficulty when first dealing with Bro that it seems to require more man power to examine them. One more tip, remember to turn on dynamic protocol detection(DPD) when dealing with stealthy attackers.

Some of you may wonder why not examining the application protocol log(http.*, ftp.*) and instead of digging the connections log first, the answer is pretty simple, if you examine the connections log and found no data transfers(Orig Bytes Sent | Res Bytes Sent), it's pointless to look at the application protocol log and may indicate some kind of network probing or scanning activities. However this doesn't apply to every condition and sometimes you need to tinker of how to perform analysis effectively. I draw the simple diagram for better illustration -

I use color depth to indicate the understanding level of the network event. You may see the color in the entity becomes lighter and lighter from left to right when you have better understanding of network event through out the structural analysis process.

What's the lacking in Bro? Clearly enough that it really needs better way to organize and manage its logs and perhaps OSSEC can fill the gaps with additional log parser. Using this mechanism with the deployment of Snort NIDS, I'm pretty confirmed that you can identify known and unknown(0 days) network attacks.

Enjoy (;])

High Quality Spam, please .....

As usual I login to my webmail after I woke up in the morning, checking the email and there's one with zip attachment, sounds malicious that it may contain executable file? Here's how the email looks like, I discard the sender part as it can be any originator -

I like the Note message, it looks so harmless. Anyway I just unzip it and it appears to be a pdf file -

shell>file 2685.pdf
2685.pdf: PDF document, version 1.2

shell>hexdump -C 2685.pdf
255044462d312e32 200d0a312030206f |%PDF-1.2 ..1 0 o|
626a0d0a3c3c0d0a 2f54797065202f43 |bj..<<../Type /C|
6174616c6f670d0a 2f50616765732033 |atalog../Pages 3|
203020520d0a2f50 6167654d6f646520 | 0 R../PageMode |
2f5573654e6f6e65 0d0a2f506167654c |/UseNone../PageL|
Output truncated .....

Therefore I use xpdf to open the file and here's how it looks like .....

You can click on the image to zoom in, but the image quality is so bad until you can't really read, I know you spammer want it to be small size(68K) only but please deliver with better quality image so that I can read or else how I gonna invest?

I don't see much of small gif file from spamming activities lately, maybe the trend is changing again .....

Enjoy ;]

400th Post: The New Milestone

This 400th blog post means a lot to me because I have never thought of myself able to blog that much continuously but I really make it. On the other hand, today it's my last half day(Saturday) working in Exabytes Network and I'm officially resigned starting from 1200 21/07/2007. Without me noticing, I have already worked in this company for 2 years, time really flies .....

Thanks to Mr. Chan who has given me opportunity to join the company, I greatly thanks to everyone who has been working closely with me especially Andy, Jackie, Albert and Guan who has been very supportive(sorry to disclose your name and don't get mad if your name is not listed :P). At the same time, I also greet the whole technical team which dedicated a lot of times and efforts in maintaining the server farms, you guys are just awesome!

I have been involved in many network security operations such as incident response, vulnerability assessment, network forensics and so forth. This is greatly improving my technical knowledge to face new challenge in my next career.

It is undoubtedly that I have mix feeling because I have to leave Pulau Penang and moving to somewhere soon, I must say I love this island very much, great people and delicious foods here. Hopefully the new place won't be disappointing me.

Seriously I don't know what to say, I'm rarely expressing my real feeling in this technical blog, anyway -

good bye to all my friends here and good luck to ya all!!!!! My memory is larger than RAM so I will carry them with me no matter where I go .....

Peace ;]

P/S: Stop being geek, you fool ;-)

HeX liveCD: Ntop

We have included Ntop in our liveCD and here's the simple how to use it to perform offline processing on pcap data, it's another useful to generate network statistic besides tshark and tcpdstat. I always use honeynet-scan18.pcap as example as it is publically available here. Credit to honeypot team for making the trace available.

shell>sudo ntop -u analyzt -M -n \ -f ./honeynet-scan18.pcap -m 172.16.1.0/24 \ -O ntop-output/ -w 192.168.0.50:3000 -W 0 -g -c -a -q

shell>sockstat -4 | grep ntop
analyzt ntop 3054 12 tcp4 *:3000 *:*

Now we have access to port 3000, just point our browser to it -

http://192.168.0.50:3000

Major Protocols Distribution

Application Protocols Distribution

Per Host Information

The down side about ntop is that it will purge the old data therefore you only view the latest data displayed, if anyone of you(experience Ntop user) know how to disable it, please share it with me because I would prefer to use it to process the pcap offline.

Enjoy (;])

Blogmania

I just realized that I have turned planet.foss.org.my to geek00l.blogspot.com except that the white background doesn't turn into black. Thanks to my blogmania!!!!! Kindly checkout the screenshots .....

It's temporarily renamed to planet geek00l .....

Cheers ;]

HeX liveCD: Chaosreader

I have included chaosreader in the HeXtra so that you can use with HeX liveCD, you can use chaosreader to parse the pcap data and it will generate the html report to ease the process of analysis thus you can quickly learn about all the network activities. Just follow the screenshot below -

Once the process is done, you will find index.html and all the files extracted in the rp-chaosreader directory, just open index.html using your internet browser and you will come to the screen below -

You can even download the compressed file(tar.gz) from the session or examine the raw data -

If you would like to have high level understanding of the network data, chaosreader is definitely a good choice.

Enjoy ;]

Malaysia: HITB SEC CONF 2007

Hack In The Box Security Conference is around the corner, if you are interested in attending one of the most interesting security conference on earth and meeting the hackers around the world, feel free to check it out here -

http://conference.hitb.org/hitbsecconf2007kl/

Enjoy ;]

Fl0p - Passive L7 Flow Fingerprinter

Checking back my old posts and I just figured out I have this post in my saved draft and never be posted online. It's all about identifying the flow by fingerprinting the application bytes in packets exchange of the connection stream. Thanks to Michal Zalewski who writes this tool called Fl0p, from the description -

fl0p is a passive L7 flow fingerprinter that examines TCP/UDP/ICMP packet sequences, can peek into cryptographic tunnels, can tell human beings and robots apart, and performs a couple of other infosec-related tricks.

If you are on FreeBSD, you can just install it via package/port system, but for gentoo users, you will have to install via source -

shell>wget http://lcamtuf.coredump.cx/soft/fl0p-devel.tgz

shell>tar xvzf fl0p-devel.tgz

shell>cd fl0p

shell>make
./Build all
Your system type is: FreeBSD

Please help with p0f 2:
http://lcamtuf.coredump.cx/p0f-help/

GNU make not found; failing back to regular (BSD?) make.
gcc -g -ggdb -Wall -DUSE_BPF=\"net/bpf.h\" -I/usr/include/pcap -I/usr/local/include/pcap -I/usr/local/include -o fl0p fl0p.c crc32.c -lpcap
strip fl0p 2>/dev/null || true

Running fl0p -

shell>./fl0p -h

Usage: ./fl0p [ -f file ] [ -i device ] [ -s file ] [ -o file ]
[ -u user ] [ -e ms ] [ -T ms ] [ -FUKrqvpdtl ] [ 'filter rule' ]
-f file - read fingerprints from file
-i device - listen on this device
-s file - read packets from tcpdump snapshot
-o file - write to this logfile (implies -t)
-u user - chroot and setuid to this user
-e ms - pcap capture timeout in milliseconds (1)
-q ms - packet timing threshold in milliseconds (400)
-F - disable fuzzy matching on all signatures
-U - display fingerprints for unidentified streams
-K - do not display known signatures (implies -U)
-r - resolve host names (not recommended)
-q - be quiet - no banner
-v - enable support for 802.1Q VLAN frames
-p - switch card to promiscuous mode
-d - daemon mode (fork into background)
-t - add timestamps to every entry
-l - output concise 1-line output

'Filter rule' is an optional pcap-style BPF expression (man tcpdump).

To automatically generate signatures for certain traffics, I decided to run the commands below -

shell>./fl0p -i sk0 -o /nsm/fl0p-logs/smtp-gather -t -l -U -d 'port 25'

shell>./fl0p -i sk0 -o /nsm/fl0p-logs/http-gather -t -l -U -d 'port 80'

shell>./fl0p -i sk0 -o /nsm/fl0p-logs/ssh-gather -t -l -U -d 'port 22'

I'm currently having a lot of signatures collected but having no time to examine about them yet, guess I need some spare time for that.

Argus: Development Testing

I think many of you have heard about argus from my previous blog post. Here's how I maintained my argus installation in my testing environment on FreeBSD platform.

Installing argus server -

shell>wget ftp://qosient.com/dev/argus-3.0/argus-3.0.0.tar.gz

shell>tar xvzf argus-3.0.0.tar.gz

shell>cd argus-3.0.0

shell>./configure --prefix=/usr/local/stow/argus3

shell>make && make install

shell>mv /usr/local/stow/argus3/bin/argusbug /usr/local/stow/argus3/

Installing argus client -

shell>wget ftp://qosient.com/dev/argus-3.0/argus-clients-3.0.0.rc.45.tar.gz

shell>tar xvzf argus-clients-3.0.0.rc.45.tar.gz

shell>cd argus-clients-3.0.0.rc.45

shell>./configure --prefix=/usr/local/stow/argusc-3rc45

shell>make && make install

I have mentioned about the usage of stow previously, you can easily handle source installation with it.

shell>cd /usr/local/stow

shell>stow argus3

shell>stow argusc-3rc45

If there's new version of argus released, you can just unstow and remove all the argus file from /usr/local/stow, then install the new version using the same steps above and you are done. If you want to try out argus 3, just download HeXtra here and load it to HeX liveCD will do.

I'm reading .....

Phrack is alive -

http://www.phrack.org/issues.html?issue=64

Faster PF implementation -

http://www.openbsd.org/papers/cuug2007/

RFC - Real Time Streaming Protocol

http://www.ietf.org/rfc/rfc2326.txt

Fast Flux Service Network

http://www.honeynet.org/papers/ff/index.html

Network Forensics

Link 1

Link 2

I plan to relearn DNS protocol because I found it interesting and may pick up RFC and reread again if I have time. Anyway I'm still reading .....

Enjoy ;]

What is reality?

I have heard many people told me that they have Firewall and IPS in place therefore they don't need monitoring, but can't you think logically what is reality? The simple fact -

Humans > System created by Humans

So you want to argue again, or you are actually humiliating yourself! And if you ask me what's the best IDS again, I will beat your ass and say -

Humans

Don't get mad, this is reality .....

Peace (;])

Network Based Forensics?

Talking about Network Based Forensics, are you going to perform batch analysis on each files that you obtain via network data through network forensics mechanism?

Even if you able to examine each file transfers over network, can you really dig into the details of every single code? Remember malicious contents can be hidden inside picture file(stenography), simple backdoor can be injected into the normal application and so forth, and how do you able to examine every single binaries?

Do you have enough resources to perform this kind of operation?

How do you examine encrypted connections, any trail?

Therefore it still falls back to the NSM concept. I have seen expensive commercial system that can extract all the files and categorize them reliably, however I swear I don't want to examine those files one by one, it's too costly and exhaustive. I do agree those data can be stored as historical purpose but doing data mining on them requires better mechanism, and if you know Network Security Monitoring well, I think you will get what I mean as better mechanism.

NSM allows you to research on different area if you ask me, you can study on Network Statistical Analysis, Network Flow analysis, IDS log analysis or even data mining on raw network data(usually pcap). So you say you want to learn Network Based Forensics, then you should again rethink about NSM!!!!!

In fact, we don't need IDS analyzt but NSM analyzt.

Cheers ;]

KDE Applications List

I have been using a lot of KDE based(QT) applications while I'm still using Fluxbox as my window manager. I find them pretty stable and works out of the box and here's the list -

- Koffice

- Kivio

- Ktorrent

- Kopete

- Amarok

I don't know what else is good but they all work pretty well. If you have any QT based application that you think is great, just share it with me. Thanks!

Enjoy ;]

HeX liveCD: Using tshark

We have included Wireshark in our liveCD, therefore it is pretty straight forward if you want to generate network statistic using wireshark command line tool - tshark(or previously called tethereal), just checkout the screenshot below -

There you will have protocol hierarchy displayed, or you can also use tcpdstat as well to compliment the output.

Cheers ;]

Smart NTFS-3G

Again I need to use my large disk and I try to mount it once I have connected it, but this time it has minor issue -

shell>sudo ntfs-3g /dev/sdb1 ~/i-Mnt
$LogFile indicates unclean shutdown (0, 1)
Failed to mount '/dev/sdb1': Operation not supported
Mount is denied because NTFS is marked to be in use. Choose one action:

Choice 1: If you have Windows then disconnect the external devices by clicking on the 'Safely Remove Hardware' icon in the Windows taskbar then shutdown Windows cleanly.

Choice 2: If you don't have Windows then you can use the 'force' option for your own responsibility. For example type on the command line:

mount -t ntfs-3g /dev/sdb1 /home/geek00l/i-Mnt -o force

Or add the option to the relevant row in the /etc/fstab file:

/dev/sdb1 /home/geek00l/i-Mnt ntfs-3g defaults,force 0 0

Smart enough, remind me that I have just passed the disk to my friend and he has actually disconnected the disk uncleanly. Therefore I just need to follow the hint given -

shell>sudo mount -t ntfs-3g /dev/sdb1 ~/i-Mnt -o force
$LogFile indicates unclean shutdown (0, 1)
WARNING: Forced mount, reset $LogFile.
shell>cd ~/i-Mnt shell>ls -la
total 20
drwxrwxrwx 1 root root 4096 Jun 11 15:16 .
drwxr-xr-x 86 geek00l geek00l 12288 Jun 11 16:07 ..
drwxrwxrwx 1 root root 4096 Jun 11 15:40 System Volume Information

It's all working now!

Enjoy ;]

OpenBSD: Wireshark Port

Thanks to Nikns again for sending me this information and I think it would be great to share with everyone especially OpenBSD users who need Wireshark for life.

In 5th of July, new wireshark version 0.99.6 is released. Many of us, who uses OpenBSD has need to use wireshark. However, ethereal (now wireshark), has been removed from OpenBSD ports tree long ago due to the bad security record where many vulnerabilities have been discovered in dissectors code.

Nikns has created unofficial OpenBSD port for Wireshark where you can find here -

http://secure.lv/~nikns/stuff/ports/wireshark-0.99.6_4.1.tar

Some of details about the port -

From pkg/DESCR:
SECURITY MEASURES:

If run with root privileges, wireshark, tshark and dumpcap will drop privileges to unprivileged user "_wireshark" after opening live capture device or dump file.

So, like OpenBSD's tcpdump written previously here...

http://geek00l.blogspot.com/2007/04/tcpdump-privilege-dropping-passive-os.html

If run as root it drops privileges after opening capture device or dump file. This is why it is recommended to start it as root first. The disadvantage for this privilege dropping is that once privileges are dropped wireshark must be restarted to start now capture...

For me I think it's great to have Wireshark on OpenBSD not for the purpose of network sniffing/logging instead of using it for network analysis, therefore thanks to Nikns putting the effort in maintaining the unofficial port himself.

Enjoy ;]

Gentoo: Mounting Large Disk via USB

I got 2 Western Digital 500 Gigs hard drives that formatted as NTFS file system at hand but not able to mount it, I choose to recompile my kernel as I have been using 2.6.19 for a while and it's time to move to 2.6.20. I read thread from the forum that I have to disable this feature in kernel in order to get it mounted properly -

CONFIG_USB_EHCI_HCD=m

I choosed to compile it as module so that I may be able to enable it if I need it. Then once I have finished compiling my kernel and reboot, I mounted the hard drive to my external enclusure, connect it to the laptop via usb and switched on the power , I can view the information via dmesg -

Jun 21 11:14:42 trinity sdb: Write Protect is off
Jun 21 11:14:42 trinity sdb: Mode Sense: 23 00 00 00
Jun 21 11:14:42 trinity sdb: assuming drive cache: write through
Jun 21 11:14:42 trinity sdb:<7>usb-storage: queuecommand called
Jun 21 11:14:42 trinity usb-storage: *** thread awakened.
Jun 21 11:14:42 trinity usb-storage: Command READ_10 (10 bytes)
Jun 21 11:14:42 trinity usb-storage: 28 00 00 00 00 00 00 00 08 00
Jun 21 11:14:42 trinity usb-storage: Bulk Command S 0x43425355 T 0xd L 4096 F 12
8 Trg 0 LUN 0 CL 10
Jun 21 11:14:42 trinity usb-storage: usb_stor_bulk_transfer_buf: xfer 31 bytes
Jun 21 11:14:42 trinity usb-storage: Status code 0; transferred 31/31
Jun 21 11:14:42 trinity usb-storage: -- transfer complete
Jun 21 11:14:42 trinity usb-storage: Bulk command transfer result=0
Jun 21 11:14:42 trinity usb-storage: usb_stor_bulk_transfer_sglist: xfer 4096 by
tes, 1 entries
Jun 21 11:14:42 trinity usb-storage: Status code 0; transferred 4096/4096
Jun 21 11:14:42 trinity usb-storage: -- transfer complete
Jun 21 11:14:42 trinity usb-storage: Bulk data transfer result 0x0
Jun 21 11:14:42 trinity usb-storage: Attempting to get CSW...
Jun 21 11:14:42 trinity usb-storage: usb_stor_bulk_transfer_buf: xfer 13 bytes
Jun 21 11:14:42 trinity usb-storage: Status code 0; transferred 13/13
Jun 21 11:14:42 trinity usb-storage: -- transfer complete
Jun 21 11:14:42 trinity usb-storage: Bulk status result = 0
Jun 21 11:14:42 trinity usb-storage: Bulk Status S 0x53425355 T 0xd R 0 Stat 0x0
Jun 21 11:14:42 trinity usb-storage: scsi cmd done, result=0x0
Jun 21 11:14:42 trinity usb-storage: *** thread sleeping.
Jun 21 11:14:42 trinity sdb1
Jun 21 11:14:42 trinity sd 2:0:0:0: Attached scsi removable disk sdb
....truncated

It is ntfs file system, therefore I mount it with ntfs-3g,

shell>sudo ntfs-3g /dev/sdb1 /mnt

shell>mount
/dev/sdb1 on /mnt type fuseblk (rw,nosuid,nodev,noatime,allow_other,blksize=4096)

I see this message in /var/log/messages -

Jun 21 11:26:49 trinity ntfs-3g[4715]: Mounted /dev/sdb1 (Read-Write, label "Sto
rage1", NTFS 3.1)

shell>df -h
/dev/sdb1 466G 101M 466G 1% /mnt

We all love big storage, don't you?

Cheers ;]

TCPdump: Understanding the Output

Tcpdump is definitely a TOOL for most of network engineers to debug or examine their network pcap data. However not many actually read it with full understanding, let's go back to basic and take a simple look at this tcpdump output -

2007-04-18 20:09:17.334010 IP 192.168.0.24.49971 > 210.171.226.46.21: P 49:54(5) ack 543 win 14
0x0000: 0016 b681 3b0e 000a e435 ea8e 0800 4500 ....;....5....E.
0x0010: 0039 9430 4000 4006 30f4 c0a8 0018 d2ab .9.0@.@.0.......
0x0020: e22e c333 0015 3ed0 d5bd 4304 5fb9 8018 ...3..>...C._...
0x0030: 000e 75c6 0000 0101 080a 0150 492e 6828 ..u........PI.h(
0x0040: 2f7c 5057 440d 0a /|PWD..

I have highlighted the hex number in the most left column, but how can we make use of them when examining the data? Here's the conversation I have with my friend and he said I should blog about it and I think that's not bad idea too since I haven't read anything about it online yet. Our conversation went like this -

(10:19:03) me: u need to make use of the left side hex number too
(10:19:11) me: 0x0000
(10:19:13) me: 0x0010
(10:19:15) me: 0x0020
(10:19:17) me: :)
(10:19:27) friend: and that represents for?
(10:19:38) me: it tells you exactly the start of the byte offset in that row
(10:19:49) me: 0x0000 = starts from 0 byte
(10:19:54) friend: ohhhhhhh
(10:19:58) me: 0x0010 = starts from 16 byte
(10:20:03) me: 16th byte
(10:20:06) me: hehe
(10:20:24) me: 0x0020 = starts from 32nd byte
(10:20:43) me: and that's for whole frame offset instead of per layer/header offset
(10:20:53) me: but it's good indication
(10:21:12) me: remember for all layer, it counts starting from 0 byte offset.
(10:21:20) me: i think i should blog about this
bla bla bla bla ...... truncated

This is definitely better than couting the byte manually especially useful when you have a large size packet, but keep in mind if you want to decode per layer/header, you will have to examine initial header length in IP header and offset/header length in TCP header which I highlighted in the output below -

2007-04-18 20:09:17.334010 IP 192.168.0.24.49971 > 210.171.226.46.21: P 49:54(5) ack 543 win 14
0x0000: 0016 b681 3b0e 000a e435 ea8e 0800 4500 ....;....5....E.
0x0010: 0039 9430 4000 4006 30f4 c0a8 0018 d2ab .9.0@.@.0.......
0x0020: e22e c333 0015 3ed0 d5bd 4304 5fb9 8018 ...3..>...C._...
0x0030: 000e 75c6 0000 0101 080a 0150 492e 6828 ..u........PI.h(
0x0040: 2f7c 5057 440d 0a /|PWD..

2 hex number = 1 byte, 5 x 4 = 20 bytes and here's your IP header -

2007-04-18 20:09:17.334010 IP 192.168.0.24.49971 > 210.171.226.46.21: P 49:54(5) ack 543 win 14
0x0000: 0016 b681 3b0e 000a e435 ea8e 0800 4500 ....;....5....E.
0x0010: 0039 9430 4000 4006 30f4 c0a8 0018 d2ab .9.0@.@.0.......
0x0020: e22e c333 0015 3ed0 d5bd 4304 5fb9 8018 ...3..>...C._...
0x0030: 000e 75c6 0000 0101 080a 0150 492e 6828 ..u........PI.h(
0x0040: 2f7c 5057 440d 0a /|PWD..

Following by the IP header should be your TCP header(Check the 9th byte offset and that's 06), just now you see I have highlighted 8 and that's the length of TCP header - 8 x 4 = 32 bytes

2007-04-18 20:09:17.334010 IP 192.168.0.24.49971 > 210.171.226.46.21: P 49:54(5) ack 543 win 14
0x0000: 0016 b681 3b0e 000a e435 ea8e 0800 4500 ....;....5....E.
0x0010: 0039 9430 4000 4006 30f4 c0a8 0018 d2ab .9.0@.@.0.......
0x0020: e22e c333 0015 3ed0 d5bd 4304 5fb9 8018 ...3..>...C._...
0x0030: 000e 75c6 0000 0101 080a 0150 492e 6828 ..u........PI.h(
0x0040: 2f7c 5057 440d 0a /|PWD..

You may notice I didn't highlight the ascii on the most right(3rd) column, because it is totally meaningless to examine the ascii data when reading the layer2, 3 and 4 header in most circumstances. What about the remaining 5 bytes - 5057 440d 0a?

2007-04-18 20:09:17.334010 IP 192.168.0.24.49971 > 210.171.226.46.21: P 49:54(5) ack 543 win 14
0x0000: 0016 b681 3b0e 000a e435 ea8e 0800 4500 ....;....5....E.
0x0010: 0039 9430 4000 4006 30f4 c0a8 0018 d2ab .9.0@.@.0.......
0x0020: e22e c333 0015 3ed0 d5bd 4304 5fb9 8018 ...3..>...C._...
0x0030: 000e 75c6 0000 0101 080a 0150 492e 6828 ..u........PI.h(
0x0040: 2f7c 5057 440d 0a /|PWD..

This is definitely FTP traffic(notice port 21 highlighted) for this case, and the reason why I highlight the ascii now is because it is the application data that you should examine, PWD is print working directory when you check the RFC here, 0d is carriage return and 0a is newline or usually be recognized as \r\n.

Hopefully here I have clear the myth of tcpdump output for some of newcomers who want to learn about reading tcpdump output, it's not so hard to decode them when you understand. I don't explain every single field in the header yet but maybe I will make it next time if I'm in mood. I know some of you prefer to use wireshark as it generates more human readable output by decoding every single field correctly but I would say it is meaningless if you generally have no idea about networking therefore it's always good to go back to basic.

Peace (;])

Linksys WRT54GL: OpenWRT

Thanks to my friend who works in network hardware line so that I can obtain this device locally. I'm lucky enough to get this linksys wireless router model WRT54GL of which I can install linux(OpenWRT) on it flawlessly, all I need is just upload the firmware that provided by OpenWRT and update . I follow the instructions here to get it install. I'm planning to install other applications such as fprobe and kismet for the fun of it. I just learn that IPKG is nifty packaging system for embedded device.

By default it's LAN interface has IP address 192.168.1.1, I login to the web interface via firefox browser that you are seeing now, you can setup the password and the default user name is root. Since I'm the subscriber of the Best ISP here for their Screamyx package, I just need to setup PPPOE for the WAN connection, configuring the user and password for the pap authentication and I'm done.

If you find yourself *nix head, then you can ssh into the router box as well, of course the main user is root and you can change the password from here as well, everything can be configured via CLI console.

This is what I want, I have bought another box mainly for war driving purpose, the changeable antenna is definitely a plus point. If you want to know the prize, it is about RM220.

Enjoy (;])

VisualRegexp: Nice Regex Learning Tool

I always emphasize on how important and helpful regular expression is as you can construct powerful pattern matching using them. Anyway I have found this good learning tool for people who would like to learn about regex, it is called VisualRegexp where you can find here -

http://laurent.riesterer.free.fr/regexp/

I install it via gentoo portage, the GUI interface is pretty simple and straight forward, what you need to do is just type in the keyword you want to match in the lower input pane, then construct the regex pattern matching in the upper input pane, then click on go button and you will see the part that is matching will turn red color, this can help you to understand how regex works and also create more accurate regex pattern.

Once you have mastered regular expression, you will be more handy in using some of the shell tools such as egrep, sed, awk and more proficient when you need to write signatures for analysis tools such as ngrep, snort, bro-ids and so forth.

If you want to learn more about it, check out this very simple tutorial and I guess you will love it -

http://zez.org/article/articleprint/11/

Cheers ;]

Outlook Email Forensics

I have done this previously and can't recall everything, however I would like to share here about what I have done before I'm out of memory. I myself don't use outlook mail client therefore I need to convert it to unix mbox mail format so that I can examine them, I found libpst that can do the job for me and install it via FreeBSD port -

shell>pkg_add -vr libpst

To extract all the emails from outlook pst file, run -

shell>readpst Monitoring.pst -o Email-Forensic

shell>ls -la Email-Forensic
-rw-r--r-- 1 geek00l geek00l 5104848 Mar 21 17:52 Backup and remove
-rw-r--r-- 1 geek00l geek00l 5693288 Mar 21 17:52 Deleted Items
-rw-r--r-- 1 geek00l geek00l 691007 Mar 21 17:52 Noc
-rw-r--r-- 1 geek00l geek00l 201746 Mar 21 17:52 Root
-rw-r--r-- 1 geek00l geek00l 441450 Mar 21 17:52 Junk
-rw-r--r-- 1 geek00l geek00l 10521973 Mar 21 17:52 Alert

shell>file *
Backup and remove: ASCII mail text
Deleted Items: ASCII mail text, with very long lines
Noc: ASCII mail text
Root: ASCII mail text
Junk: ASCII mail text
Alert: ASCII mail text

That's possible to retrieve Outlook 2003 email but it requires more works and the pst file size must less than 2G or you will have to split it before converting it back to older pst format so that it can be parsed by readpst.

I should have read Real Digital Forensics so that I won't need to google around because it is introduced in Real Digital Forensics book as well, anyway have fun.

Cheers ;]

Argus: Anonymize the flows

After l have read the interesting example in secviz.org, I decide to try the argus graphing example using afterglow and graphviz tool, however I'm lazy to change the IPs in my pcap file that contains p2p traffic to protect the privacy, hence I decide to use one of the tool bundled in argus suite called ranonymize(the name tells the story), I converted my pcap file to argus file format so that it can be parsed by argus suite.

shell>argus -r p2p-suspect.pcap -w p2p-suspect.arg

Now I just need to use ranonymize to read the file and pipe it to other tools to generate the graph, I have afterglow source under ~/i-Apps and below is the full command line I use -

shell>ranonymize -r p2p-suspect.arg -w - | \
racluster -r - -m saddr daddr proto dport -c, -s saddr daddr | \ ~/i-Apps/afterglow/src/perl/graph/afterglow.pl -a -t -e 2 -c \ ~/i-Apps/afterglow/src/perl/parsers/color.properties | \
neato -Tgif -o p2p-anonymize.gif

Now you will have p2p-anonymize.gif file in the current directory and here's the simple graph. You can see all the IPs are already been anonymized but you get the idea of the flows.

Instead of reading pcap raw data, sometimes graph helps.

Enjoy ;]

HeX liveCD 1.0R: The Progress

What we have done so far for HeX liveCD to reach 1.0 Solid Release, here are the lists -

- bash odd screen buffer fixed

- metasploit 3 updated by chfl4gs_ since no respond from the porter

- normal user can mount vfs

- non-root user can sniff on bpf interface as long as they are in wheel user group

- bro-ids porting in progress

- bsd installer porting

- fluxbox menu redesign

If you have any thought about the liveCD, just let us know!

Cheers ;]

Ourmon 2.7 Web Interface Configuration

I have previously installed ourmon 2.7 on FreeBSD that you can find here, since ourmon is great with all its meaningful graphs(of which I think it makes sense to network security analyzt instead of those I have seen in commercial systems(don't ask me which one) where all the graphs are generated on behalf of your management that has little or no value to us.

To get what I mean, you can check out the demo here -

http://jerry.cat.pdx.edu/ourmon/

After the installation via unofficial FreeBSD port that offered by Jim Binkley, we can now moving to setup and configure our web server so that we can view our statistical data via web interface, here's the step by step -

Installing apache 1.3 using FreeBSD port -

shell>pkg_add -vr apache

Add these two lines to /etc/rc.conf -

apache_enable="YES"
apache_flags=""

Then edit apache configuration file which is /usr/local/etc/apache/httpd.conf, you have to change the values below to where ourmon data is stored -

DocumentRoot "/usr/local/www/data/ourmon"



AllowOverride All

Since those data are sensitive, therefore we need at least basic authentication to read the data, I use .htaccess for this matter, just create .htaccess file and put in /usr/local/www/data/ourmon, the file shoud look like this -

AuthName "Access Denied!"
AuthType Basic
AuthUserFile /usr/local/mrourmon/.htpasswd
Require valid-user

Last I generate .htpasswd with the command -

shell>htpasswd -c -b /usr/local/mrourmon/.htpasswd trinity matrix

The user I have just created is trinity and the password is matrix. By now we can start apache -

shell>apachectl start

Just launch the web browser and point it to the box that running ourmon will do. You will see the similar web interface like the demo above. Please keep in mind this is not the best deployment but quick example to get it work.

Enjoy ;]

The Better Way to Promote Ubuntu Linux

Sometimes you are bored of what you are doing and you start to find something amusing, here's the interesting one -

http://lynstatic.doubleukay.com/uploads//av-114386.jpg

I bet that will attract more users, what a niche way to increase Ubuntu Linux user base!!!!!

Enjoy ;]

HeXtra 1.0 Beta

There are many NSM based tools that we don't install because it is not available via FreeBSD port system, don't blame us of not creating the package for it since we are all busy and some of them are not yet into production release but very useful in many conditions, here we have included extra NSM based tools that you can download, decompress and run it! We call it as HeXtra(HeX Extra) and its version is 1.0 Beta(similar to the HeX liveCD version) so it won't create confusion for future release. Here are the tools that we have included -

- Afterglow

- Argus3 RC

- Bro-Nids

- Chaosreader

- Sguil Client(CVS)

It is pretty easy to get them running, just do the following -

shell>mkdir ~/rp-Mnt

Mount your usb thumb drive to rp-Mnt -

shell>sudo sysctl vfs.usermount=1

shell>sudo chmod 777 /dev/da0s1 (it's da0s1 in my case but it may be differ)

shell>mount -t msdosfs /dev/da0s1 ~/rp-Mnt

shell>cd ~/rp-Mnt

shell>wget \
http://rawpacket.org/anonymous/projects/HeXtra-1.0B.tar.bz2

shell>tar xvjf HeXtra-1.0B.tar.bz2

All the tools reside in its specified directory and you can run them any time by now. On the other hand, we have also included clamav signature database to make it easy for user who want to use clamAV, snort signatures are not included but you can easily fetch it using oinkmaster, I will write up the howto later. I have also added the script call NSM-Offline.sh which you just need to run it against the network data(pcap) and it will generate NSM output for examination. Credit goes to Niklas who has initially written this script. I just modified it to run flawlessly using this liveCD. Make sure you have snort signatures in place in order to run this script as well.

Cheers ;]

HeX liveCD: Download Mirrors

Thanks to Dr. J who is kind enough to provide the bandwidth for our liveCD download mirrors, if you are in the other part of the world than Asia Pacific, you are encouraged to download from those mirrors instead.

- Mirror 1 (http)

- Mirror 2 (http)

- Mirror 3 (https)

For https mirror, if you want to download using wget, you can do so with -

shell>wget --no-check-certificate \
https://secure.redsphereglobal.com/data/tools/security/live/hex-i386-1.0beta.iso

Enjoy (;])

Tech Training: Structured Network Threat Analysis and Forensics

Last year, me and mel have conducted the training at HITB conference. The training is about performing network traffic analysis using NSM concept but more on threat centric. We are also adding the part Network Based Forensics and discarding Host Based Forensics this year to make it more compatible to the topic.

Again this year, we will be conducting the training at HITB conference again, however all the students who attend to the training no longer need to load the VMware image in order to use all the analysis tools but utilizing the HeX liveCD if possible(in case if the laptop brought by the students not FreeBSD compatible, he or she can still use VMware/Qemu to load the liveCD but we are trying to avoid this condition so that you can make full use of your computer resources.

There will be updates for our training contents as well to cope with the whole network security scene. If you are interested in the training, feel free to check it out here.

Peace ;]

HeX liveCD: The Analogy

Most of the people have this problem with liveCD, after booting it up, play around and forget about it .....

I think problem with the liveCD is its customized environment, for example if I'm familiar with linux but not bsd, then I have to dig into google to get some setup done because of different approach in setting up stuffs(sys v and bsd style for example), only who develop the liveCD can make use of it efficiently because he or she knows the environment entirely and familiar with the operating system.

Different liveCD is developed for different purpose, many prefer end user based liveCD, security liveCD(penetration testing and hacking), forensic liveCD and thus far, I have never seen any liveCD that mainly develop to perform Network Security Monitoring operation. I do know Knoppix-NSM and NST, but their design is more for real time monitoring with NSM Based tools but HeX is more emphasizing on reactive NSM operation and Network Based Forensics. For me I prefer to call this as Network Data Analysis Centric liveCD and it can be learning tool as well if you are interested in NSM.

First of all, I must admit I love two specific liveCDs - Backtrack and Helix. Both present really good idea to serve their purpose. Guess I don't have to speak about Backtrack anymore as I guess most of the people in security industry find it useful, on the other hand Helix is liveCD mainly developed to perform computer forensics in Incident Response operation, you can easily create the case and duplicate the data with Helix and it offers wide range of Forensic tools as well to do the job. And they are not those throw in new logo and install all the tools without customization liveCD which I hate the most.

Yes, our liveCD development team never aim for wide range of audience when creating this liveCD, as stated officially this liveCD is designed for network security analyzt, and not only we offer wide range of NSM based tools but also concentrating on the work flows. We believe tools are as good as how Network Security Analyzt can utilize it. You may have already read this before using this liveCD. We are following this logic -

Obtain Network Based Data -> Utilizing NSM Based Tools -> Generate Output -> Output Interpretation -> Output Analysis -> Output Summarization -> Report

I would like to draw a beautiful diagram for this but I just want to show the simple quickies. As a analyzt(especially in reactive NSM operation), we need to obtain network based data first, then using all the necessary tools to generate the output(I prefer to call it output because it means nothing if you don't understand them), then interpret the output(this part is pretty dynamic based on the skills level, however the more you understand about each field that presented by the output and the more you practice, you are getting better and efficient). To analyze the output, it is very dependent based on your experience, knowledge(TCP/IP, Programming) and how efficient you can make use of internet resources, Once finished the analysis part, you will have to conclude everything you have studied, and summarize the output. At the end, write the report with hostility but keep in mind the report does present how well you understand the output and translate it to concrete form.

The liveCD can only make up to this part(see below) -

Obtain Network Based Data -> Utilizing NSM Based Tools -> Generate Output

The rest depends on how analyzt able to perform it -

Output Interpretation -> Output Analysis -> Output Summarization -> Report

To further improve the usage of the liveCD and share it with the community, great documentation is a must. Therefore I will start writing a series of how-to guideline so that people who are interested can make full use of this liveCD and learning tips and tricks on using NSM based tools. Hopefully it can fill the gaps and you all love it.

Have I mentioned my Network Security Analyzt Handbook? It's in the progress now and hopefully you can use it inline with this liveCD soon.

Enjoy (;])

HeX liveCD 1.0 Beta: ReadyBoost?

I haven't tried out Windows Vista but some of my friends are using them, and they are showing off this feature to me call ReadyBoost where you can just plug in your flash drive and it will be the add on "Memory" to boost the performance. Sorry to say that I'm not impressed.

I started to use the HeX liveCD in my production environment so that I can find more problems to fix, it seems I have to use snort today to analyze the network data. After I launched the liveCD, I decided to download snort rules which is not distributed together with the liveCD using oinkmaster, you need to register at snort main site in order to download VRT certified snort rules by acquiring the oink code or you can use bleedingsnort rules. I use the former one and what I need is just uncomment one liner and put in my oink code -

Copy the sample oinkmaster configuration file to user directory -

shell>cp /usr/local/etc/oinkmaster.conf.sample ~/oinkmaster.conf

Uncomment this line and replace oinkcode with the code you obtain from snort site -

url = http://www.snort.org/pub-bin/oinkmaster.cgi/oinkcode/snortrules-snapshot-CURRENT.tar.gz

I started to fetch the ruleset to the snort-rules directory -

shell>mkdir snort-rules

shell>oinkmaster -C ~/oinkmaster.conf -o snort-rules

Once I have all the rules downloaded, I copy snort configuration file to snort-rules directory as well -

shell>cp /usr/local/etc/snort/snort.conf-sample snort-rules/snort.conf

I commented this line -

# include $RULE_PATH/local.rules

Then I started to run snort with the pcap file given -

shell>mkdir snort-output

shell>snort -c ~/snort-rules/snort.conf -ybr malicious.pcap -l ~/snort-output
output truncated .....
Killed ...

I check my /var/log/messages and I got this -
output truncated ....
Jul 4 14:33:33 raWPacket kernel: pid 2079 (snort), uid 1000, was killed: out of swap space
Jul 4 14:34:50 raWPacket kernel: pid 2080 (snort), uid 1000, was killed: out of swap space

It is out of swap space, but I'm running this liveCD and requires no disk. What can I do to fix this? Can I do ReadyBoost? Of course I can't as that's Vista technology!!!!! But wait, I can use something call Memory Disk for swap, I plugged in my 2G USB thumb drive which formatted as msdos file system so that I can use it flawlessly with other OS, and tried to mount it -

shell>dmesg
Output truncated .....
umass0: vendor 0x13fe USB DISK Pro, rev 2.00/1.10, addr 2
da0 at umass-sim0 bus 0 target 0 lun 0
da0: <> Removable Direct Access SCSI-0 device
da0: 1.000MB/s transfers
da0: 1959MB (4012032 512 byte sectors: 255H 63S/T 249C)
da1 at umass-sim0 bus 0 target 0 lun 1
da1: <> Removable Direct Access SCSI-0 device
da1: 1.000MB/s transfers
da1: 1MB (2880 512 byte sectors: 64H 32S/T 1C)

shell>sudo sysctl vfs.usermount=1

shell>sudo chmod 777 /dev/da0s1

shell>mkdir ~/rp-mnt

shell>mount -t msdosfs /dev/da0s1 ~/rp-mnt

shell>mount
/dev/iso9660/raWPacket on / (cd9660, local, read-only)
devfs on /dev (devfs, local, multilabel)
/dev/md0.uzip on /usr (ufs, local, read-only)
/dev/md1 on /etc (ufs, local)
/dev/md2 on /usr/home (ufs, local)
/dev/md3 on /var (ufs, local)
/dev/md4 on /tmp (ufs, local)
/dev/da0s1 on /usr/home/analyzt/rp-mnt (msdosfs, local)

Then I created 1G disk image in the USB drive -

shell>dd if=/dev/zero of=~/rp-mnt/SwapBoost bs=1M count=1024

As you know we are using memory disk for our liveCD, therefore they are already existed in the device list -

shell>ls -la /dev/md*
crw-r----- 1 root operator 0, 92 Jul 4 17:31 /dev/md0
crw-r----- 1 root operator 0, 93 Jul 4 17:31 /dev/md0.uzip
crw-r----- 1 root operator 0, 94 Jul 4 17:31 /dev/md1
crw-r----- 1 root operator 0, 95 Jul 4 17:31 /dev/md2
crw-r----- 1 root operator 0, 96 Jul 4 17:31 /dev/md3
crw-r----- 1 root operator 0, 98 Jul 4 17:31 /dev/md4
crw------- 1 root wheel 0, 81 Jul 4 17:31 /dev/mdctl

You can see memory disk 0-4 are occupied, thus we can create md5

shell>sudo mdconfig -a -t vnode -f ./swap0 -u 5

Then enable the swap -

shell>sudo swapon /dev/md5

To check how much swap space is used -

shell>pstat -s -h
Device 1K-blocks Used Avail Capacity

/dev/md5 1048576 23M 1001M 5%

I run snort again now and it has no crash at all but takes long time to finish processing the pcap as swap space are used heavily. To get snort running smoother, you can uncomment this line in snort.conf -

config detection: search-method lowmem

ReadyBoost????? We have SwapBoost since long time ago!!!!!

Enjoy (;])

HeX liveCD 1.0 Beta

Today is great day for us, finally we reach 1.0 beta milestone!!!!! Thanks to all members that have put a lot of hard works to get it done, and I also feel thankful to my friends who have contributed some of their configuration files.

We are out of schedule but consider ourself meeting the timeline(though we plan to release the first beta by end of June but now is just early July!).

You can find all the information regarding the liveCD here -

http://www.rawpacket.org/projects/hex-livecd

Of course we do plan for future release and here's our current 1.0 idea -

Todo list:
1. Import BSD Installer
2. Shiny Fluxbox Menu(New idea already in my mind)
3. Add on analysis scripts that either written by raWPacket team members or any contributors
4. Extra fl0p, tcpXtract and pads signatures addon
5. Fix unknown bugs that reported by users

As usual, any valuable feedbacks(suggestions or critics) are welcomed, however keep in mind that make love no war.

Enjoy (;])