HowTo sniff Wifi packets in Ubuntu 12.04

Main assumption:
I am using the Wifi adaptor with Atheros AR9271 chip.

Steps:

1. Make sure you have a Wifi adaptor which supports monitor mode.
   You may check it in linuxwireless.org. I prefer the one with Atheros AR9271 chip (eg. TP-LINK TL-WN721N). I don't recommend the one with Realtek chip (likely no monitor mode) or Ralink chip (seems recent version of the chip requires a latest version of kernel)
2. Compile and install aircrack-ng (Guide)
3. To start the monitor mode by executing (assume your wifi device is wlan0)

sudo airmon-ng start wlan0 [optional channel no.]

A monitor device (usually with name monX, where X=[0...9]) will be created and you can do sniffing on that device

4. Then you can sniff packets using Wireshark and monitor device mon0
5. To stop the monitor mode, execute

sudo airmon-ng stop mon0
sudo airmon-ng stop wlan0

FAQ
Question: Got error message "SIOCSIFFLAGS: Operation not possible due to RF-kill"
Solution: The Wifi adaptor is turned off somehow. Please either (i) turn it on in NetworkManager, or (ii) execute "sudo rfkill unblock wifi"

Makefile for GNUPlot source file

SOURCES=$(wildcard *.dem)
TARGETS=$(SOURCES:.dem=.eps)

all: $(TARGETS)

%.eps: %.dem
gnuplot < $<

Cloud in your box (Devstack, VMware Player)

Updated@29-02-2012: please use the commit fca0b641a733ff3f1b3697ca04ebffb65e6c20fa for horizon
Updated: devstack no longer support Ubuntu 11.04. more on [vpsee]


Recently, cloud computing is a buzzword for marketing. You can see it in everywhere: anti-virus software (namely the one called Kav...), Office 360+5, etc.
You may feel confused. Actually, cloud computing itself is a collective noun, and means nothing. The professional will classifier the "cloud" into three different types: IaaS(Infrastructure as a Service), PaaS(Platform as a Service) and SaaS(Service as a Service). In this article, we focus on the first one and build our IaaS cloud in our home computer.

Building a "cloud" does not require a powerful, in fact. You need a computer with VT-x / AMD SVM feature. Basically, a Core-i3 is ok.

First of all, you need to install VMware Player from VMware. Make sure you are install the version 5. Otherwise, the performance of the cloud is not acceptable.

Then, get a Ubuntu 11.04 server CD from Ubuntu. Install it with the help of VMware Player. It should not be difficult task as the Player will guide you in installation.

Finally, you need to install Devstack, a one-click-installed cloud testbed for Openstack (an open source cloud platform middleware) by:

sudo apt-get update
sudo apt-get install git
git clone git://github.com/cloudbuilders/devstack.git
cd devstack; ./stack.sh

Now you should got the management interface URL (eg. http://172.16.142.129/) and the corresponding username/password.

Happy cloud computing!!

SHA1 with SSSE3 acceleration

Just for recall, SHA1 include 3 steps:
1. initialize the hash value
2. a for-loop sha1_update, which each add 512-bit to the hash pool
3. finally do padding, appending extra 1 bit and appending the length of the message

Someone from Intel did SSSE3 acceleration Improving the Performance of the Secure Hash Algorithm (SHA-1) , which improves about 50%. For my workstation (i3-2100) , according to my OpenSSL CVS, the speed of SHA1 is around ~600MB.

In order to use original patch from the article (nearly the same mirror), just replace your step (2) with

void sha1_update_intel(int *hash, const char* input, size_t num_blocks );

Then compile the attached Assembly source using

nasm -f elf64 -o sha1sum sha1sum.c #OR
yasm -f elf64 -o sha1sum sha1sum.c

and link the .o to your program


Actually it ported to OpenSSL and was porting to kernel crypto:
OpenSSL CVS - Check-in [20937]
[PATCH v3 0/2] crypto, x86: assembler implementation of SHA1


Little benchmark using "openssl speed sha1"

The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
i3-2100, 3.1G, Ubuntu 11.04-amd64
sha1-noasm 41509.80k   102830.17k   192025.60k   246080.17k   272615.51k
sha1-0.9.8    42963.77k   119563.82k   245913.69k   347327.44k   380286.29k
sha1-cvs       92259.92k   222029.44k   416805.07k   537374.38k   586825.73k
i5-760, 2.8G, CentOS 5.6-amd64
sha1-0.9.8    34189.42k   102558.96k   227312.78k   326800.38k   373721.22k
sha1-cvs       86948.28k   212469.37k   382940.36k   478658.76k   515970.81k
Xeon W3520, 2.67G, Ubuntu 10.10-amd64
sha1-0.9.8    37745.02k   107145.20k   227622.96k   315530.95k   358102.73k
sha1-cvs       83978.67k   199786.96k   365259.25k   459923.91k   493331.01k
Core2 Quad, CentOS 5.5-amd64
sha1-0.9.8    33646.27k   100451.23k   227755.21k   332412.51k   385859.58k
sha1-cvs       74688.70k   173850.00k   317574.97k   401208.30k   436905.76k
Core2 Duo E7400, CentOS 5.6 i386
sha1-0.9.8    26966.17k    84344.87k   202065.04k   312690.22k   368893.56k
sha1-cvs      54377.91k   141416.44k   271153.15k   353743.53k   390078.93k
C7-M 1.2G,  Ubuntu 10.04
sha1-98k         4767.11k    13917.20k    30128.64k    42823.41k    48589.48k
sha1-cvs         9774.71k    22950.89k    40949.16k    51051.52k    55009.28k
sha1-98b_p     3616.33k    13747.16k    47283.20k   121266.52k   224901.91k
Nano 1.6G, src
sha1-no           20057.60k    51514.05k    99721.39k   130167.81k   142811.14k
sha1-padlock  37713.77k   114562.71k   259637.33k   379907.41k   438818.13k


sha1-noasm, sha1-cvs: OpenSSL Snapshot 20110918
sha1-0.9.8, Ubuntu 11.04: OpenSSL 0.9.8o 01 Jun 2010
sha1-0.9.8, CentOS 5.6: OpenSSL 0.9.8e-fips-rhel5
sha1-98b_p,  OpenSSL 0.9.8b with patch

Zip Unicode Filename 3

ZIP File Format Specification
Changelog
6.3.0 -Added option for Unicode filename storage
General purpose bit flag bit 11:
Language encoding flag (EFS). If this bit is set, the filename and comment fields for this file must be encoded using UTF-8. (see APPENDIX D)

6.3.2 -Documented InfoZIP "Extra Field" values for UTF-8 file name and file comment storage

APPENDIX D - Language Encoding (EFS)
------------------------------------

The ZIP format has historically supported only the original IBM PC character
encoding set, commonly referred to as IBM Code Page 437.  This limits storing
file name characters to only those within the original MS-DOS range of values
and does not properly support file names in other character encodings, or
languages. To address this limitation, this specification will support the
following change.

If general purpose bit 11 is unset, the file name and comment should conform
to the original ZIP character encoding.  If general purpose bit 11 is set, the
filename and comment must support The Unicode Standard, Version 4.1.0 or
greater using the character encoding form defined by the UTF-8 storage
specification.  The Unicode Standard is published by the The Unicode
Consortium (www.unicode.org).  UTF-8 encoded data stored within ZIP files
is expected to not include a byte order mark (BOM).

Applications may choose to supplement this file name storage through the use
of the 0x0008 Extra Field.  Storage for this optional field is currently
undefined, however it will be used to allow storing extended information
on source or target encoding that may further assist applications with file
name, or file content encoding tasks.  Please contact PKWARE with any
requirements on how this field should be used.

The 0x0008 Extra Field storage may be used with either setting for general
purpose bit 11.  Examples of the intended usage for this field is to store
whether "modified-UTF-8" (JAVA) is used, or UTF-8-MAC.  Similarly, other
commonly used character encoding (code page) designations can be indicated
through this field.  Formalized values for use of the 0x0008 record remain
undefined at this time.  The definition for the layout of the 0x0008 field
will be published when available.  Use of the 0x0008 Extra Field provides
for storing data within a ZIP file in an encoding other than IBM Code
Page 437 or UTF-8.

General purpose bit 11 will not imply any encoding of file content or
password.  Values defining character encoding for file content or
password must be stored within the 0x0008 Extended Language Encoding
Extra Field.

Ed Gordon of the Info-ZIP group has defined a pair of "extra field" records
that can be used to store UTF-8 file name and file comment fields.  These
records can be used for cases when the general purpose bit 11 method
for storing UTF-8 data in the standard file name and comment fields is
not desirable.  A common case for this alternate method is if backward
compatibility with older programs is required.

Definitions for the record structure of these fields are included above
in the section on 3rd party mappings for "extra field" records.  These
records are identified by Header ID's 0x6375 (Info-ZIP Unicode Comment
Extra Field) and 0x7075 (Info-ZIP Unicode Path Extra Field).

The choice of which storage method to use when writing a ZIP file is left
to the implementation.  Developers should expect that a ZIP file may
contain either method and should provide support for reading data in
either format. Use of general purpose bit 11 reduces storage requirements
for file name data by not requiring additional "extra field" data for
each file, but can result in older ZIP programs not being able to extract
files.  Use of the 0x6375 and 0x7075 records will result in a ZIP file
that should always be readable by older ZIP programs, but requires more
storage per file to write file name and/or file comment fields.

Zip Unicode Filename 2

ZIP (file format) - Wikipedia, the free encyclopedia

6.3.0: Documented Unicode (UTF-8) filename storage. Expanded list of supported hash, compression, encryption algorithms.

SourceForge.net: Files

Quick list of major changes in Zip 3.0:
- Unicode
If Unicode support is enabled and supported on the system Zip is run on, Zip now can read paths not in the current character set and store those paths in portable UTF-8 format. These Unicode paths can then be used to partially or fully recreate the paths on other systems depending on the character set support provided by the unzip on the receiving system. In particular, this allows portability of paths between Windows and Unix. Unicode comments are also supported on systems where UTF-8 is the current character set. Unicode comment support for other systems is expected in Zip 3.1.

結果近期的zip spec添加了UTF-8 Filename支援
各大解壓軟件也跟隨
(e.g. WinZip 11.2, WinRAR 3.8, 7-Zip 4.58 beta)

Zip Unicode Filenames??

簡單來說, 就是zip檔不支援unicode檔名,
要unicode檔名就要改用rar或是7z

SUMMARY: Zip/Unzip and encoding problem (Was: Re: How to detect the encoding of a string?)
How to detect the encoding of a string?
Zip Unicode Filenames