Chapter 3 Tools used in the project

Chapter 2 Tools used in the project

This chapter introduces the tools that will be used in this project and their source codes.

2.1 Blast Test Code (By Prof. Bob Russell, UDP version is by Chaoxing Lin)

Most of the performance experiments will use blast test tools. Here is how it works. Client side sends a number (iteration times, specified from command line) of packets of given request size (specified from command line) on end and close the connection. Server side keeps receiving packets and swallows it. Then when client closes connection it sends 1 byte back. For the complete blast test code, please refer to http://www.cs.unh.edu/cnrg/lin/linuxProject/phase3/nettest

2.2 Packet Dropper Code (By Glenn Herrin, Modified by Chaoxing Lin)

The first impairment we introduce to the router kernel is packet dropper module. This is a kernel module. It randomly drops packet with a specified destination address at a given rate. Code are inserted and executed on virtual DEVICE layer.

On receiving each packet, the Dropper checks the destination IP address. If the destination is the target IP, we get a 16-bit random number, if the random number is less than the dropping threshold value which is rate*65535, we drop this packet, otherwise, process it normally.

/************************packet_dropper*****************

This is what dev_queue_xmit will call while this module is installed ****/

int packet_dropper(struct sk_buff *skb)

{

unsigned short t;

if (skb->nh.iph->daddr == target) {

/* the following code is modified by lin: begin (*/

t = getUnsignedShortRandom();

if (t < cutoff)

{

number_of_packet_dropped++;

return 1; /* drop this packet */

}

/* modified by lin : end ) */

}

return 0; /* continue with normal routine */

} /* packet_dropper */

The random number is generated in this way. This way is only good for Intel processor because the assembly instruction “rdtsc” is only in Intel processor.

inline unsigned short getUnsignedShortRandom()

{

unsigned l, h;

unsigned short low;

unsigned char * lp;

unsigned char * hp;

unsigned char ldata;

/* get a CPU cycle. Only good for Intel processor */

__asm__ volatile("rdtsc": "=a" (l), "=d" (h));

/* get the lower 16 bits */

low = (unsigned short) l & 0xFFFF;

/****Swap lower byte with the higher byte *******/

hp=(unsigned char*)(&low);/*point to high byte */

lp=hp+1; /* point to low byte */

/* swap the higher byte with the lower byte */

ldata = *lp;

*lp = *hp;

*hp = ldata;

return low;

}

The packet dropper module also exports symbols that will be used in proc file system:

unsigned short cutoff;           /* drop threshold value */

float rate;                    /* drop percentage */

unsigned number_of_packet_dropped;/* #of packet dropped */

_u32 target = 0x0202A8C0;      /* address 192.168.2.2 */

Kernel source changed:

/usr/src/linux/net/core/dev.c

at line: 942 Add: int (*xmit_test_function)( struct sk_buff * ) = 0;

See the context:

.....

919 #ifdef CONFIG_HIGHMEM

920 /* Actually, we should eliminate this check as soon as we know, that:

921 * 1. IOMMU is present and allows to map all the memory.

922 * 2. No high memory really exists on this machine.

923 */

924

925 static inline int

926 illegal_highdma(struct net_device*dev,struct sk_buff *skb)

927 {

928 int i;

929

930 if (dev->features&NETIF_F_HIGHDMA)

931 return 0;

932

933 for (i=0; i<skb_shinfo(skb)->nr_frags; i++)

934 if (skb_shinfo(skb)->frags[i].page >=highmem_start_page)

935 return 1;

936

937 return 0;

938 }

939 #else

940 #define illegal_highdma(dev, skb) (0)

941 #endif

942

943 /* ADDED BY LIN to test packet-dropper:begin ( */

944

945 int (*xmit_test_function)( struct sk_buff * ) = 0;

946 /* ADDED BY LIN to test packet-dropper:end ) */

/**

* dev_queue_xmit - transmit a buffer

* @skb: buffer to transmit

* Queue a buffer for transmission to a network device. The caller must

* have set the device and priority and built the buffer *before calling this function. The function can be called *from an interrupt.

* A negative errno code is returned on a failure. A success does not

* guarantee the frame will be transmitted as it may be dropped due

* to congestion or traffic shaping.

........

In function: int dev_queue_xmit(struct sk_buff *skb)

At the very beginning, add:

if ( xmit_test_function && ( *xmit_test_function )(skb) )

{

kfree_skb( skb );

return 0;

}

See the context:

...........

int dev_queue_xmit(struct sk_buff *skb)

{

struct net_device *dev = skb->dev;

struct Qdisc *q;

/* ADDED BY LIN TO TEST PACKET-DROPPER : BEGIN ( */

if ( xmit_test_function && ( *xmit_test_function )(skb) )

{

kfree_skb( skb );

return 0;

}

/* ADDED BY LIN TO TEST PACKET-DROPPER : END ) */

if (skb_shinfo(skb)->frag_list &&

!(dev->features&NETIF_F_FRAGLIST) &&

skb_linearize(skb, GFP_ATOMIC) != 0) {

kfree_skb(skb);

return -ENOMEM;

}

...........

/usr/src/linux/net/netsyms.c

at line: 570 Add:

extern int ( *xmit_test_function ) ( struct sk_buff * );

EXPORT_SYMBOL_NOVERS( xmit_test_function );

After changing kernel source code, don’t forget to re-compile the kernel.

For the complete packet_dropper code, please refer to http://www.cs.unh.edu/cnrg/lin/linuxProject/phase3/random_dropper/

2.3 Proc File System Operation Code

In order to check the exact number of packets dropped by the packet dropper and to easily change dropping rate and destination IP address, we develop a kernel module to do this job. It dynamically probes current dropping rate, destination packet, number of packet dropped. It can also set these values.

static struct proc_dir_entry *dropperInfo, *fileEntry;

static int proc_reset_num(struct file *file,const char *

buffer,unsigned long count,void *data);

static int proc_read_num(char *buf, char **start,

off_t off, int count,int *eof, void *data);

static int proc_read_dropper(char *buf, char **start,

off_t off, int count,int *eof, void *data);

static int proc_write_dropper(struct file *file,const char

*buffer,unsigned long count,void *data);

int init_module()

{

int rv = 0;

EXPORT_NO_SYMBOLS;

/* create directory */

dropperInfo = proc_mkdir(“dropperInfo”, NULL);

if(dropperInfo == NULL) {

printk("<1> dropperInfo failed\n");

rv = -ENOMEM;

goto out;

}

dropperInfo->owner = THIS_MODULE;

/* create “dropper” and "numDropped" files */

fileEntry = create_proc_entry("dropper", 0644, dropperInfo);

if(fileEntry == NULL) {

rv = -ENOMEM;

goto error;

}

fileEntry->read_proc = proc_read_dropper;

fileEntry->write_proc = proc_write_dropper;

fileEntry->owner = THIS_MODULE;

fileEntry = create_proc_entry("numDropped", 0644, dropperInfo);

if(fileEntry == NULL) {

rv = -ENOMEM;

goto error;

}

fileEntry->read_proc = proc_read_num;

fileEntry->write_proc = proc_reset_num;

fileEntry->owner = THIS_MODULE;

/* everything OK */

printk(KERN_INFO "%s initialized\n", MODULE_NAME );

return 0;

error:

remove_proc_entry(MODULE_NAME, NULL);

out: return rv;

}

Inserting this module to kernel will create a directory /proc/dropperInfo. In this directory there will be 2 files:

dropper:

Use “cat /proc/dropperInfo/dropper” to see the rate, ip, # of packet dropped.

Use “input rate ip_addr” to reset rate and ip_addr dynamically.

numDropped:

Use “cat /proc/dropperInfo/numDropped” to see # of packet dropped since last

reset.

Use “reset” to set “# of packet dropped” to 0

For the complete proc operation code,

please refer to http://www.cs.unh.edu/cnrg/lin/linuxProject/phase3/proc/

For the proc file system handling, see http://www.cs.unh.edu/cnrg/lin/linuxProject/resource/proc.pdf

2.4 Tool used to synchronize router and source host

For some the experiments, as far as the control is on the same machine, we can write a shell script to let the experiments run again and again. But it’s almost impossible to write a script to control remote machines. In here, it’s really hard to write script to synchronize the router and the sending host. We need to run blast test for a few times at a given dropping rate. Then on router side, we need to set a new dropping rate and let experiments go on.

Our goal is to reduce the human interactions to the minimum when we do these experiments. The automation tool will also make it easy to repeat the experiments with the same settings sometime later.

Based on this, we develop a simple tool to synchronize the router and sending host. This tool contains 2 applications

“dublin” is to be run as server on dublin.cs.unh.edu:5678.

“madrid” is to be run as client on madrid.cs.unh.edu.

Step 1. Dublin installs “packet dropper” module.

Step 2. Dublin installs “proc file system manipulation” module.

Step 3. Dublin opens server port 5678 and waits for connection. And after connection is

created, it waits for control message from Madrid and do as instructed. Control

messages are:

SET_RATE rate: Dublin sets dropping rate to rate. It also creates a directory

with the name rate. Number of packet loss at this rate will be put under this

directory.

RECORD_LOSS: Dublin checks the proc file system, get the number of

packets dropped since last reset. And then it resets the number_of_packet to 0.

SET_ITERATION times: (used only in regular dropping test with dropping rate

1/6 ). Dublin creates a directory with the name times. Number_of_packet loss

will be put under this directory.

Step 4 Prague opens “blast test server”( using port 1026 because it’s hard coded in this

tools )

Step 5.Madrid opens tool client. It sends control message to Dublin to set rate. After it

gets acknowledgement, it does experiment(blastclient 1026 192.168.2.2 100000

1448). After each experiment, Madrid sends control message to Dublin to record

the packet loss. For each specific rate, Madrid does 10 times experiments and

then increment the rate and set it to Dublin.

Note: Don’t forget to redirect the result to a file. We will use this file to do statistics.

Step 6 On Dublin, run script “collectRawDropNum” whose content is:

cd dirName

for d in `ls`;

cd $d

echo $d >> /home/lin/Tools/rawDrop

for f in `ls`;

cat $f >> /home/lin/Tools/rawDrop

done

cd ..

done

cd ..

(Suppose that dirName is the directory created by “dublin”) we will get a file with the number of packets dropped for each experiment.

Run “getNum rawDrop dropStat”, we will get the statistical result of “number of packet dropped during each experiment”

Step 7. On Madrid, Suppose that we redirect the experiment result to a file “rawElapse“,

run “getElapse rawElapse elapseStat” we can get the statistical result. By this

result we can run gnuplot to draw the graph.

For the complete AutoTools code,

please refer to http://www.cs.unh.edu/cnrg/lin/linuxProject/phase3/AutoTools/