RAM Disks over Gigabit Ethernet

Purpose

To evaluate the performance of network attached storage (NAS) RAM disks connected via Gigabit Ethernet using workstation-class PCs.

Materials

Network Components

• Two SMC 9552 TX Gigabit Ethernet Adapters (Broadcomm chip), a.k.a. "SMC Tigercard 1000" $15 each
• SMC 8505T 5-port Gigabit switch $110
• Two Adaptec CAT-6 10ft UTP cables. $15 each

Sources: Page Computers, Unity Electronics - http://www.unityelectronics.com, Retrieved 2004

PC Components

Server

• Red Hat 9
• P3-500 (actually a P3-1Ghz downclocked for stability)
• Abit BX6 motherboard
• 500-1 GB SDRAM
• Tigon3 driver

Client

• Windows XP Pro SP1a
• Athlon XP 2500+
• 1 GB DDR SDRAM
• SMC 95x2 TX driver version 1.1.415.2002

Procedure

• Install Gigabit cards in 32-bit/33Mhz PCI slots.
• Connect the computers together with CAT-6 cables via the Gigabit switch.
• Install and configure Samba in Red Hat.
• Set up a Samba shared folder by editing /etc/samba/smb.conf

  
  	[ramdisk]
  	comment = RAM Disk Fast Temporary File Space
  	path = /mnt/ramdisk
  	read only = no
  	public = yes
  	
  

• Set up a RAM disk to a Samba shared folder.
  • mkdir /mnt/ramdisk
  • mount -t tmpfs none /mnt/ramdisk -o size=256000
  • chmod 1777 /mnt/ramdisk
• Connect the RAM disk as a network drive from Windows XP.
• Run Sisoftware Sandra 2003 on the network drive.

Results

--------------------------
Fastest Sisoft Sandra Sample Benchmark
Firewire(1394) TCP/IP [NTFS/WinXP] (Win32 x86)

Drive Index: 10,740 kB/s

--------------------------
TEST: No RAM-Disk -- Sharing Maxtor 13GB HDD
* ramfs; tg3 linux driver; 500Mhz P3

Drive Index: 10,977 kB/s

Test File Size : 1023MB
Benchmark Breakdown
Buffered Read : 29 MB/s
Sequential Read : 15 MB/s
Random Read : 6 MB/s
Buffered Write : 15 MB/s
Sequential Write : 10 MB/s
Random Write : 5 MB/s
Average Access Time : 6 ms (estimated)
Drive Type : Network
Total Size : 11GB
Free Space : 8GB, 100%

-----------------------------
TEST: RAM Disk via Gigabit Switch
* ramfs; tg3 linux driver; 500Mhz P3

Drive Index: 20,912 kB/s
Test File Size : 425MB
Buffered Read : 29 MB/s
Sequential Read : 21 MB/s
Random Read : 23 MB/s
Buffered Write : 18 MB/s
Sequential Write : 13 MB/s
Random Write : 15 MB/s
Drive Type : Network
Total Size : 473MB
Free Space : 473MB, 100%

-----------------------------------
TEST: RAM Disk via Direct Cable (Crossover)
* ramfs; tg3 linux driver; 500Mhz P3

Drive Index: 22,092 kB/s
Test File Size : 425MB
Buffered Read : 27 MB/s
Sequential Read : 23 MB/s
Random Read : 24 MB/s
Buffered Write : 18 MB/s
Sequential Write : 14 MB/s
Random Write : 15 MB/s
Drive Type : Network
Total Size : 473MB
Free Space : 473MB, 100%

-----------------------------------
TEST: RAM Disk via Gigabit Switch
* tmpfs; smc95x2 linux driver; 850Mhz P3; 512MB RAM

Drive Index: 44,219 kB/s
Test File Size : 226MB
Buffered Read : 47 MB/s
Sequential Read : 45 MB/s
Random Read : 46 MB/s
Buffered Write : 34 MB/s
Sequential Write : 33 MB/s
Random Write : 34 MB/s
Drive Type : Network
Total Size : 251MB
Free Space : 251MB, 100%

-----------------------------------
TEST: RAM Disk via Gigabit Switch
* SiSoftware Sandra Standard 2004.10.9.89
* RAM Disk Server: Red Hat 9; P-III 1 Ghz; 512MB SDRAM
* Client: WinXP Pro; Athlon XP 2500+; 1GB DDR SDRAM
* Network: SMC 8505T Switch, SMC 9552TX Adapters

Drive Index: 45,535 kB/s
Test File Size : 405MB
Buffered Read : 48 MB/s  
Sequential Read : 47 MB/s
Random Read : 47 MB/s   
Buffered Write : 37 MB/s  
Sequential Write : 35 MB/s
Random Write : 36 MB/s  
Drive Type : Network
Total Size : 450MB
Free Space : 450MB, 100%

-----------------------------------
TEST: Swap/RAM Disk via Gigabit Switch
* 1GB tmpfs; smc95x2 linux driver; 850Mhz P3; 512MB RAM

Drive Index: 1.403 MB/s
Test File Size : 1023MB
Buffered Read : 43 MB/s
Sequential Read : 1196 kB/s
Random Read : 1641 kB/s
Buffered Write : 35 MB/s
Sequential Write : 1395 kB/s
Random Write : 1710 kB/s
Average Access Time : 15 ms (estimated)
Total Size : 1024MB
Free Space : 1024MB, 100%

-----------------------------------
TEST: RAM Disk via Gigabit Switch
* SiSoftware Sandra Standard 2004.10.9.89
* RAM Disk Server: Red Hat 9; P-III 1 Ghz; 512MB SDRAM
* Client: WinXP Pro; Athlon XP 2500+; 1GB DDR SDRAM
* Network: SMC 8505T Switch, SMC 9552TX Adapters
** Client connected via 100Mbit 3com Network card

Drive Index : 11,513 kB/s
Test File Size : 405MB
Buffered Read : 11 MB/s
Sequential Read : 11 MB/s
Random Read : 11 MB/s
Buffered Write : 11 MB/s
Sequential Write : 11 MB/s
Random Write : 11 MB/s
Total Size : 450MB
Free Space : 450MB, 100%

-----------------------------------
TEST: Repeated disk-to-disk 150 MB file transfer via FTP.
Connected via Gigabit Switch.

P3 @ 500 Mhz, tg3 driver = 27 MB/sec

P3 @ 850 Mhz, tg3 driver = 33 MB/sec

P3 @ 850 Mhz, smc95x2 driver = 36 MB/sec

P3 @ 1000 Mhz, smc95x2 driver = 36 MB/sec


-----------------------------------
TEST: PING TEST

Connected via SMC 8505T Gigabit Switch @ 1000Mbit
0.115 ms average

Connected via Linksys 10/100 Router/Switch @ 100Mbit
0.145 ms average

-----------------------------------

Discussion

Sustained transfer speeds were no greater than 30 MB/sec, which compares to a typical 100 GB single disk drive. With disk caching enabled, the FTP test was essentially memory to memory, reconfirming this maximum transfer speed.

Random read was 46 MB/sec and random write was 34 MB/sec. Compare this to the 15-25 MB/sec of a four-disk RAID-5 array using 10K RPM drives.

The benchmark through the gigabit Ethernet switch was about ten percent slower than the benchmark with a direct cable connection. This is an insignificant difference. The switch was otherwise under no load.

Latency, as measured by ping, improves only about twenty percent between 100 and 1000 megabit Ethernet, even though the raw bandwidth increases ten-fold.

These PCs are low-end workstations with 32-bit/33Mhz PCI slots. One PC is a P3-500Mhz and the other is a Athlon XP 2500+, both with up to one Gigabyte of RAM. Besides the PCI bus speed and processors, drivers make a big difference in the performance.

The SMC cards and switch support a jumbo-packet feature, but this was not tested. This would likely improve sustained transfer speeds.

Unlike ten or one hundred megabit link status indicators, the one gigabit link status does not light up unless a driver is installed that supports the feature, even if the cabling can support a link at that rate. Category 5 and 5e cables tested also linked at one gigabit, though their performance was not tested.

The SMC card driver has BSOD problems in Windows XP, probably because it's new. The driver for XP is not even available on their website, but only on the floppy that came with the card. Also, the network would hang randomly during large transfers. Couple the driver problems with flaky memory in the server box and the testing was arduous.

A nice feature of the SMC cards and switch is Auto-MDI Crossover. This means one can attach any type of cable (crossover or straight-through) and the ports will automatically detect the intended purpose.

Conclusion

A RAM disk connected via 1000Mbit Gigabit Ethernet provides very good random-read and write performance, and decent sustained transfer speeds. The only benefit of using a RAM disk was the higher random read/write performance, which compares to a workstation 4-disk RAID-5. The sustained transfer speeds, limited to 30MB/sec by the Gigabit Ethernet on 32-bit/33Mhz PCI, can easily be provided by single disk drives. Because the speeds were not much better than single or RAID disk drives, the RAM-disk based NAS would not provide much benefit over a disk-based NAS to typical applications. Eliminating the bottlenecks of the busses by using server motherboards with 64-bit/66Mhz or greater PCI slots might change the equation though. A faster, more stable server machine and better drivers might also improve the benchmarks considerably.

Originally written February 6, 2004.

---