test on Oct 6


Dublin and prague are running kernel 2.4.10
(NOTE: This time, the tulip card driver is built into kernel directly )
madrid is running kernel 2.2.16

Strange things:

When I boot kernel 2.2.16-22, the ethernet interfaces information:

3Com:eth0	tulip1: eth1	tulip2: eth2

When I boot kernel 2.4.10, the ethernet interfaces information:

tulip1:eth0	tulip2: eth1	3Com:eth2


What's going on here?????? How can I bind 3Com to eth0 


In my experiments, if not otherwise specified, the network infrastrue is :
http://pubpages.unh.edu/~lin/linuxProject/diagram.jpg

And experiment 1 is always:

 Madrid:eth2( 192.168.3.1 )  ======>  Prague:eth2( 192.168.3.2 )

experiment 2 is always:
 
 Madrid:eth1( 192.168.1.2 ) ====>Dublin:eth2 ====>Dublin:eth1 
 ====>Prague:eth1( 192.168.2.2)

experiment 3 is always:

 Madrid:eth0( 132.177.8.27 ) ===>Switch .... ===>Prague:eth0( 132.177.8.29 )









=================================================================
Experiment 1:

[lin@prague ttcp]$ ttcp -r -s
ttcp-r: buflen=8192, nbuf=2048, align=16384/0, port=5001  tcp
ttcp-r: socket
ttcp-r: accept from 192.168.3.1
ttcp-r: 16777216 bytes in 1.44 real seconds = 11370.92 KB/sec +++
ttcp-r: 12285 I/O calls, msec/call = 0.12, calls/sec = 8526.11
ttcp-r: 0.0user 0.2sys 0:01real 18% 0i+0d 0maxrss 0+1pf 0+0csw


[lin@madrid lin]$ ttcp/ttcp -t -s 192.168.3.2 < nttcp-1.47.tar.gz 
ttcp-t: buflen=8192, nbuf=2048, align=16384/0, port=5001  tcp  -> 192.168.3.2
ttcp-t: socket
ttcp-t: connect
ttcp-t: 16777216 bytes in 1.43 real seconds = 11460.60 KB/sec +++
ttcp-t: 2048 I/O calls, msec/call = 0.71, calls/sec = 1432.58
ttcp-t: 0.0user 0.0sys 0:01real 4% 0i+0d 0maxrss 0+2pf 0+0csw



=================================================================
Experiment 2:


[lin@prague ttcp]$ ttcp -r -s
ttcp-r: buflen=8192, nbuf=2048, align=16384/0, port=5001  tcp
ttcp-r: socket
ttcp-r: accept from 192.168.1.2
ttcp-r: 16777216 bytes in 2.19 real seconds = 7469.57 KB/sec +++
ttcp-r: 12285 I/O calls, msec/call = 0.18, calls/sec = 5600.81
ttcp-r: 0.0user 0.1sys 0:02real 9% 0i+0d 0maxrss 0+1pf 0+0csw



[lin@madrid lin]$ ttcp/ttcp -t -s 192.168.2.2 < nttcp-1.47.tar.gz 
ttcp-t: buflen=8192, nbuf=2048, align=16384/0, port=5001  tcp  -> 192.168.2.2
ttcp-t: socket
ttcp-t: connect
ttcp-t: 16777216 bytes in 2.18 real seconds = 7506.90 KB/sec +++
ttcp-t: 2048 I/O calls, msec/call = 1.09, calls/sec = 938.36
ttcp-t: 0.0user 0.1sys 0:02real 4% 0i+0d 0maxrss 0+2pf 0+0csw



=================================================================
Experiment 3. USING SWITCH

[lin@prague ttcp]$ ttcp -r -s
ttcp-r: buflen=8192, nbuf=2048, align=16384/0, port=5001  tcp
ttcp-r: socket
ttcp-r: accept from 132.177.8.27
ttcp-r: 16777216 bytes in 1.96 real seconds = 8374.19 KB/sec +++
ttcp-r: 12246 I/O calls, msec/call = 0.16, calls/sec = 6259.18
ttcp-r: 0.0user 0.2sys 0:01real 11% 0i+0d 0maxrss 0+2pf 0+0csw


[lin@madrid lin]$ ttcp/ttcp -t -s 132.177.8.29 < nttcp-1.47.tar.gz 
ttcp-t: buflen=8192, nbuf=2048, align=16384/0, port=5001  tcp  -> 132.177.8.29
ttcp-t: socket
ttcp-t: connect
ttcp-t: 16777216 bytes in 1.95 real seconds = 8407.53 KB/sec +++
ttcp-t: 2048 I/O calls, msec/call = 0.97, calls/sec = 1050.94
ttcp-t: 0.0user 0.2sys 0:01real 13% 0i+0d 0maxrss 0+2pf 0+0csw


From the above 3 experiments:

We found the networking speed improves significantly, compared to last
week's experiment which is running kernel 2.2.16-22 on Dublin


The possible reason may be:

1. Dublin is using new Kernel 2.4.10 which may improves its routing algorithm 
   or something
   To test this assumption, reboot prague.cs.unh.edu to kernel 2.2.16 , which 
   means the end computer are the same as thoes in last week's experiment.

   Do the same experiment 2  for 3 times
   ============================================================
   [lin@prague ttcp]$ ttcp -r -s
   ttcp-r: buflen=8192, nbuf=2048, align=16384/0, port=5001  tcp
   ttcp-r: socket
   ttcp-r: accept from 192.168.1.2
   ttcp-r: 16777216 bytes in 2.10 real seconds = 7795.41 KB/sec +++
   ttcp-r: 11672 I/O calls, msec/call = 0.18, calls/sec = 5553.47
   ttcp-r: 0.0user 0.0sys 0:02real 1% 0i+0d 0maxrss 0+2pf 0+0csw

   [lin@madrid lin]$ ttcp/ttcp -t -s 192.168.2.2 < nttcp-1.47.tar.gz 
   ttcp-t: buflen=8192, nbuf=2048, align=16384/0, port=5001  tcp  -> 192.168.2.2
   ttcp-t: socket
   ttcp-t: connect
   ttcp-t: 16777216 bytes in 2.09 real seconds = 7822.83 KB/sec +++
   ttcp-t: 2048 I/O calls, msec/call = 1.05, calls/sec = 977.85
   ttcp-t: 0.0user 0.0sys 0:02real 2% 0i+0d 0maxrss 0+2pf 0+0csw
   ============================================================


   ============================================================
   [lin@prague ttcp]$ ttcp -r -s
   ttcp-r: buflen=8192, nbuf=2048, align=16384/0, port=5001  tcp
   ttcp-r: socket
   ttcp-r: accept from 192.168.1.2
   ttcp-r: 16777216 bytes in 2.15 real seconds = 7630.09 KB/sec +++
   ttcp-r: 11689 I/O calls, msec/call = 0.19, calls/sec = 5443.61
   ttcp-r: 0.0user 0.0sys 0:02real 1% 0i+0d 0maxrss 0+1pf 0+0csw

   [lin@madrid lin]$ ttcp/ttcp -t -s 192.168.2.2 < nttcp-1.47.tar.gz 
   ttcp-t: buflen=8192, nbuf=2048, align=16384/0, port=5001  tcp  -> 192.168.2.2
   ttcp-t: socket
   ttcp-t: connect
   ttcp-t: 16777216 bytes in 2.14 real seconds = 7644.09 KB/sec +++
   ttcp-t: 2048 I/O calls, msec/call = 1.07, calls/sec = 955.51
   ttcp-t: 0.0user 0.0sys 0:02real 2% 0i+0d 0maxrss 0+2pf 0+0csw
   ============================================================


   ============================================================
   [lin@prague ttcp]$ ttcp -r -s
   ttcp-r: buflen=8192, nbuf=2048, align=16384/0, port=5001  tcp
   ttcp-r: socket
   ttcp-r: accept from 192.168.1.2
   ttcp-r: 16777216 bytes in 2.10 real seconds = 7806.88 KB/sec +++
   ttcp-r: 11681 I/O calls, msec/call = 0.18, calls/sec = 5565.92
   ttcp-r: 0.0user 0.0sys 0:02real 0% 0i+0d 0maxrss 0+2pf 0+0csw


   [lin@madrid lin]$ ttcp/ttcp -t -s 192.168.2.2 < nttcp-1.47.tar.gz 
   ttcp-t: buflen=8192, nbuf=2048, align=16384/0, port=5001  tcp  -> 192.168.2.2
   ttcp-t: socket
   ttcp-t: connect
   ttcp-t: 16777216 bytes in 2.10 real seconds = 7818.16 KB/sec +++
   ttcp-t: 2048 I/O calls, msec/call = 1.05, calls/sec = 977.27
   ttcp-t: 0.0user 0.0sys 0:02real 2% 0i+0d 0maxrss 0+2pf 0+0csw
   ============================================================

   HERE we see that :
   if the 3COM card is in the same speed as tulip card, the new kernel 2.4.10
   DOES improve the routing speed significantly
   ( 7818.16 KB/sec +++    VS    230KB/sec ++++ ===> 34:1 )

   We still need to remove the possibility that this improvement is brought
   by the speed difference of 3COM card and tulip card. 


   In the following 3 experiments. We change NIC setting on dublin. USE tulip
   card as NIC involed in static routing between 192.168.1.0 and 192.168.2.0

                        ------------------
                        |   switch       |
                        |                |
                        | 10     9     11|
                        ------------------
                           |     |      |
                           |     |      |
                           |     |      |  132.177.8.0/25
         -------------------     |      -------------------
        |                        |                       |
        |                        |                       |
        |                        |                       |
        |                        |                       |
  -----------------     -------------------     ---------------------
  |    eth0       |     |       ETH2      |     |       eth0        |
  |               |     |                 |     |                   |
  |    Madrid     |     |      Dublin     |     |      Prague       |
  |               |     |                 |     |                   |
  | eth2   eth1   |     | ETH1    ETH0    |     | eth1    eth2      |
  -----------------     -------------------     ---------------------
  3.1|      |1.2          1.1|      |2.1       2.2 |        | 3.2
     |      -----------------        ----------------       |
     |       192.168.1.0/24               192.168.2.0/24    |
     |                                                      |
     --------------------------------------------------------
                     192.168.3.0/24

 ON DUBLIN tulip1:eth0	tulip2: eth1	3Com:eth2
 Dublin routing table:

 [lin@dublin lin]$ route
 Kernel IP routing table
 Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
 132.177.8.0     *               255.255.255.128 U     0      0        0 eth2
 192.168.3.0     192.168.2.2     255.255.255.0   UG    0      0        0 eth0
 192.168.2.0     *               255.255.255.0   U     0      0        0 eth0
 192.168.1.0     *               255.255.255.0   U     0      0        0 eth1
 127.0.0.0       *               255.0.0.0       U     0      0        0 lo
 default         phub0.cs.unh.ed 0.0.0.0         UG    0      0        0 eth2


   Do experiment 2 for 3 times
   ==========================================================
   [lin@prague ttcp]$ ttcp -r -s
   ttcp-r: buflen=8192, nbuf=2048, align=16384/0, port=5001  tcp
   ttcp-r: socket
   ttcp-r: accept from 192.168.1.2
   ttcp-r: 16777216 bytes in 2.09 real seconds = 7855.23 KB/sec +++
   ttcp-r: 11695 I/O calls, msec/call = 0.18, calls/sec = 5607.11
   ttcp-r: 0.0user 0.0sys 0:02real 3% 0i+0d 0maxrss 0+1pf 0+0csw


   [lin@madrid lin]$ ttcp/ttcp -t -s 192.168.2.2 < nttcp-1.47.tar.gz 
   ttcp-t: buflen=8192, nbuf=2048, align=16384/0, port=5001  tcp  -> 192.168.2.2
   ttcp-t: socket
   ttcp-t: connect
   ttcp-t: 16777216 bytes in 2.08 real seconds = 7866.14 KB/sec +++
   ttcp-t: 2048 I/O calls, msec/call = 1.04, calls/sec = 983.27
   ttcp-t: 0.0user 0.0sys 0:02real 3% 0i+0d 0maxrss 0+2pf 0+0csw
   ==========================================================



   ==========================================================
   [lin@prague ttcp]$ ttcp -r -s
   ttcp-r: buflen=8192, nbuf=2048, align=16384/0, port=5001  tcp
   ttcp-r: socket
   ttcp-r: accept from 192.168.1.2
   ttcp-r: 16777216 bytes in 2.10 real seconds = 7793.61 KB/sec +++
   ttcp-r: 11729 I/O calls, msec/call = 0.18, calls/sec = 5579.30
   ttcp-r: 0.0user 0.0sys 0:02real 2% 0i+0d 0maxrss 0+1pf 0+0csw

   [lin@madrid lin]$ ttcp/ttcp -t -s 192.168.2.2 < nttcp-1.47.tar.gz 
   ttcp-t: buflen=8192, nbuf=2048, align=16384/0, port=5001  tcp  -> 192.168.2.2
   ttcp-t: socket
   ttcp-t: connect
   ttcp-t: 16777216 bytes in 2.10 real seconds = 7804.45 KB/sec +++
   ttcp-t: 2048 I/O calls, msec/call = 1.05, calls/sec = 975.56
   ttcp-t: 0.0user 0.1sys 0:02real 4% 0i+0d 0maxrss 0+2pf 0+0csw
   ==========================================================


   ==========================================================
   [lin@prague ttcp]$ ttcp -r -s
   ttcp-r: buflen=8192, nbuf=2048, align=16384/0, port=5001  tcp
   ttcp-r: socket
   ttcp-r: accept from 192.168.1.2
   ttcp-r: 16777216 bytes in 2.11 real seconds = 7752.20 KB/sec +++
   ttcp-r: 11703 I/O calls, msec/call = 0.18, calls/sec = 5537.35
   ttcp-r: 0.0user 0.0sys 0:02real 4% 0i+0d 0maxrss 0+1pf 0+0csw

   [lin@madrid lin]$ ttcp/ttcp -t -s 192.168.2.2 < nttcp-1.47.tar.gz 
   ttcp-t: buflen=8192, nbuf=2048, align=16384/0, port=5001  tcp  -> 192.168.2.2
   ttcp-t: socket
   ttcp-t: connect
   ttcp-t: 16777216 bytes in 2.11 real seconds = 7760.29 KB/sec +++
   ttcp-t: 2048 I/O calls, msec/call = 1.06, calls/sec = 970.04
   ttcp-t: 0.0user 0.0sys 0:02real 4% 0i+0d 0maxrss 0+2pf 0+0csw
   ==========================================================

   In the above 3 experiment, on Dublin the NIC card involved in routing are
   2 tulip cards. Hence, we can draw a conclusion that:

   Kernel 2.4.10 is significantly faster than kernel 2.2.16-22 on static routing

   Reboot Dublin to 2.2.16-22 to prove that.

   Do experiments 2 for 3 times
   =================================================
   [lin@prague ttcp]$ ttcp -r -s
   ttcp-r: buflen=8192, nbuf=2048, align=16384/0, port=5001  tcp
   ttcp-r: socket
   ttcp-r: accept from 192.168.1.2
   ttcp-r: 16777216 bytes in 65.04 real seconds = 251.92 KB/sec +++
   ttcp-r: 5154 I/O calls, msec/call = 12.92, calls/sec = 79.25
   ttcp-r: 0.0user 0.0sys 1:05real 0% 0i+0d 0maxrss 0+2pf 0+0csw

   [lin@madrid lin]$ ttcp/ttcp -t -s 192.168.2.2 < nttcp-1.47.tar.gz 
   ttcp-t: buflen=8192, nbuf=2048, align=16384/0, port=5001  tcp  -> 192.168.2.2
   ttcp-t: socket
   ttcp-t: connect
   ttcp-t: 16777216 bytes in 64.82 real seconds = 252.78 KB/sec +++
   ttcp-t: 2048 I/O calls, msec/call = 32.41, calls/sec = 31.60
   ttcp-t: 0.0user 0.0sys 1:04real 0% 0i+0d 0maxrss 0+2pf 0+0csw
   =================================================



   =================================================
   [lin@prague ttcp]$ ttcp -r -s
   ttcp-r: buflen=8192, nbuf=2048, align=16384/0, port=5001  tcp
   ttcp-r: socket
   ttcp-r: accept from 192.168.1.2
   ttcp-r: 16777216 bytes in 65.26 real seconds = 251.07 KB/sec +++
   ttcp-r: 5164 I/O calls, msec/call = 12.94, calls/sec = 79.13
   ttcp-r: 0.0user 0.0sys 1:05real 0% 0i+0d 0maxrss 0+2pf 0+0csw

   [lin@madrid lin]$ ttcp/ttcp -t -s 192.168.2.2 < nttcp-1.47.tar.gz 
   ttcp-t: buflen=8192, nbuf=2048, align=16384/0, port=5001  tcp  -> 192.168.2.2
   ttcp-t: socket
   ttcp-t: connect
   ttcp-t: 16777216 bytes in 65.06 real seconds = 251.84 KB/sec +++
   ttcp-t: 2048 I/O calls, msec/call = 32.53, calls/sec = 31.48
   ttcp-t: 0.0user 0.0sys 1:05real 0% 0i+0d 0maxrss 0+2pf 0+0csw
   =================================================



   =================================================
   [lin@prague ttcp]$ ttcp -r -s
   ttcp-r: buflen=8192, nbuf=2048, align=16384/0, port=5001  tcp
   ttcp-r: socket
   ttcp-r: accept from 192.168.1.2
   ttcp-r: 16777216 bytes in 65.53 real seconds = 250.02 KB/sec +++
   ttcp-r: 5157 I/O calls, msec/call = 13.01, calls/sec = 78.70
   ttcp-r: 0.0user 0.0sys 1:05real 0% 0i+0d 0maxrss 0+2pf 0+0csw

   [lin@madrid lin]$ ttcp/ttcp -t -s 192.168.2.2 < nttcp-1.47.tar.gz 
   ttcp-t: buflen=8192, nbuf=2048, align=16384/0, port=5001  tcp  -> 192.168.2.2
   ttcp-t: socket
   ttcp-t: connect
   ttcp-t: 16777216 bytes in 65.31 real seconds = 250.85 KB/sec +++
   ttcp-t: 2048 I/O calls, msec/call = 32.66, calls/sec = 31.36
   ttcp-t: 0.0user 0.0sys 1:05real 0% 0i+0d 0maxrss 0+2pf 0+0csw
   =================================================

   So kernel 2.4.10 is  much faster than kernel 2.2.16
	

2. The speed difference of the 3 NIC cards. 3 COM card may be faster than 
   tulip card. Notice that in 2.4.10 kernel, eth2 is 3COM card.

3. From experiment 3 

    madrid:eth0( 3 COM ) <===> prague:eth0( tulip ), 
   we see that tulip seems slower then COM.( compared to last week experiment
   which is  3COM <====> 3COM.  8407.53 KB/sec VS 11461.40 KB/sec +++)
   To test this assumption, reboot prague to kernel 2.2.16 which will use 3COM
   as eth0. Re-do experiment 3 to see the speed.

	Do the same experiment 3 for 3 times
   ===================================================
   [lin@prague ttcp]$ ttcp -r -s
   ttcp-r: buflen=8192, nbuf=2048, align=16384/0, port=5001  tcp
   ttcp-r: socket
   ttcp-r: accept from 132.177.8.27
   ttcp-r: 16777216 bytes in 1.44 real seconds = 11409.07 KB/sec +++
   ttcp-r: 12275 I/O calls, msec/call = 0.12, calls/sec = 8547.75
   ttcp-r: 0.0user 0.2sys 0:01real 18% 0i+0d 0maxrss 0+1pf 0+0csw


   [lin@madrid lin]$ ttcp/ttcp -t -s 132.177.8.29 < nttcp-1.47.tar.gz 
   ttcp-t: buflen=8192,nbuf=2048, align=16384/0, port=5001  tcp  -> 132.177.8.29
   ttcp-t: socket
   ttcp-t: connect
   ttcp-t: 16777216 bytes in 1.43 real seconds = 11448.47 KB/sec +++
   ttcp-t: 2048 I/O calls, msec/call = 0.72, calls/sec = 1431.06
   ttcp-t: 0.0user 0.3sys 0:01real 26% 0i+0d 0maxrss 0+2pf 0+0csw
   ===================================================


   ===================================================
   [lin@prague ttcp]$ ttcp -r -s
   ttcp-r: buflen=8192, nbuf=2048, align=16384/0, port=5001  tcp
   ttcp-r: socket
   ttcp-r: accept from 132.177.8.27
   ttcp-r: 16777216 bytes in 1.44 real seconds = 11395.89 KB/sec +++
   ttcp-r: 12245 I/O calls, msec/call = 0.12, calls/sec = 8517.01
   ttcp-r: 0.0user 0.1sys 0:01real 11% 0i+0d 0maxrss 0+2pf 0+0csw


   [lin@madrid lin]$ ttcp/ttcp -t -s 132.177.8.29 < nttcp-1.47.tar.gz 
   ttcp-t: buflen=8192,nbuf=2048, align=16384/0, port=5001  tcp  -> 132.177.8.29
   ttcp-t: socket
   ttcp-t: connect
   ttcp-t: 16777216 bytes in 1.43 real seconds = 11454.46 KB/sec +++
   ttcp-t: 2048 I/O calls, msec/call = 0.72, calls/sec = 1431.81
   ttcp-t: 0.0user 0.3sys 0:01real 21% 0i+0d 0maxrss 0+2pf 0+0csw
   ===================================================

   ===================================================
   [lin@prague ttcp]$ ttcp -r -s
   ttcp-r: buflen=8192, nbuf=2048, align=16384/0, port=5001  tcp
   ttcp-r: socket
   ttcp-r: accept from 132.177.8.27
   ttcp-r: 16777216 bytes in 1.44 real seconds = 11413.35 KB/sec +++
   ttcp-r: 12284 I/O calls, msec/call = 0.12, calls/sec = 8557.23
   ttcp-r: 0.0user 0.1sys 0:01real 13% 0i+0d 0maxrss 0+1pf 0+0csw


   [lin@madrid lin]$ ttcp/ttcp -t -s 132.177.8.29 < nttcp-1.47.tar.gz 
   ttcp-t: buflen=8192,nbuf=2048, align=16384/0, port=5001  tcp  -> 132.177.8.29
   ttcp-t: socket
   ttcp-t: connect
   ttcp-t: 16777216 bytes in 1.43 real seconds = 11453.28 KB/sec +++
   ttcp-t: 2048 I/O calls, msec/call = 0.72, calls/sec = 1431.66
   ttcp-t: 0.0user 0.5sys 0:01real 39% 0i+0d 0maxrss 0+2pf 0+0csw
   ===================================================

   HERE we do see that 3COM <===> 3COM is a little faster than  
   3COM <==> tulip