2010-05-30

GuruPlug Server Plus : don't waste your money on it!

Introduction

Guruplug Server and Guruplug Server Plus are small computers that fit into a power plug and which are made by GlobalScale. I was first informed about them from someone who asked on the haproxy mailing list if haproxy had ever been tested on them. These devices run on a Marvell 88F6281 processor, which is a system-on-chip (SoC) powered by an ARM-derived Sheeva processor core at 1.2 GHz, coming with 512 MB of RAM and as much of flash. The Server Plus also has 2 Gigabit Ethernet ports, and the whole is supposed to consume just a few watts, so I found it very appealing for building high performance, low consumption haproxy servers.
I pre-ordered one at NewIT about two months ago and it finally arrived on Friday. The packaging was clean and the device looked compact and solid. A free JTAG+console adapter board was also provided, which is absolutely required to be able to connect to the serial port. The box also contains a european plug, as well as power and Ethernet cables. The devices come pre-installed with a debian system so that it's not needed to install anything to test it.

First connection

I plugged my Guruplug on my network and observed my DHCP server's logs in order to find the address I should connect to (no doc is shipped with the device, everything has to be downloaded from the net or guessed). Well, the dhcp client looked a bit buggy, it was sending DHCP_DISCOVER packets forever, ignoring the DHCP_OFFER it got in response to all of its packets. So I couldn't connect to the device over the network. I had to use the serial console.
Trying to connect using the serial console was another adventure. The JTAG adapter was detected by the usbserial driver, but all I got were endless "1". I finally found that I had to build and load the ftdi_siodriver too to use the JTAG adapter. Nice, I can now connect via minicom, using login root and password nosoup4u. I manually killed dhclient3 and restarted it by hand without all the strange arguments on its command line and this time it worked. I did not spend more time investigating this, probably that some of these args passed via the init script are not very functional.

First network test

Being interested in running haproxy on it, I had to run a benchmark. I installed the haproxy-1.4.4 package which was already packaged for this distro, because it was faster than building a cross-compiler on my machine. I started it with a simple test configuration which only forwards traffic to a single server without logging nor playing with headers or cookies. This is the configuration with which I push my ALIX to 2700 hits/s, and a Celeron 1.3 GHz to about 11000 hits/s. Being running at 1.2 GHz with some DDR2 memory, I expected to see something between 8000 and 10000 hits/s. I carefully unloaded iptables and fired the test. What a disappointment : 2200 hits/s at 100% CPU ! Even 20% less than my ALIX running at 500 MHz on DDR-400 and 100 Mbps ports ! I tried changing some TCP optimisations in the configuration but I could only hardly reach 2400 hits/s with a config in which my ALIX reaches 3000. Still 20% less.
Then I decided to run a bit rate test which should favor this CPU which integrates the network controllers. Instead of fetching empty objects, I fetch large ones (10 MB). Result : only 310 Mbps at 100% CPU where the ALIX gives 200 Mbps with some CPU margin (two 100 Mbps ports), and the Celeron slightly more than 800. Quite obviously this CPU is a donkey !

Test on other kernels

I finally decided to give this box a second chance by booting another distro's kernel on it, just in case we'd have a problem with debian's 2.6.32. Fedora and Slackware have pre-built kernels and images that should be able to run there. Following the Slackware's howto hangs at boot. The howto also says that some plugs were shipped with a buggy bootloader which hangs during USB detection. Mine does and is 6 months old. A new bootloader is available on plugcomputer.org so I carefully updated it, hoping not to brick the box. That was successful, I could boot the Slackware and run my test with the same haproxy binary.It was slightly faster, 2250 hits/s and 320 Mbps, but still far from what I'd expect.
After a reboot, I discovered that the debian kernel shipped from factory does not boot anymore after the bootloader has been upgraded. There is a platform identifier which is different between what is documented as being a Guruplug (0x0A63) and what was installed in this beast (0x0A29). I was starting to get a bit fed up with the low level of platform preparation at GlobalScale, so I unplugged the thing.

Massive heat dissipation

While unplugging wires, I burnt my fingers on the RJ45 plug! The metal was at something like 70 or 80 degrees celsius, I don't know. It was simply untouchable, even after a long idle period. The Geode in my ALIX runs under less than 1W at full load and around 0.1W when in idle. It does not even have a heatsink. Many people have reported major heating issues on these Guruplugs in the past, with power supplies dying after less than 3 months, but GlobalScale said they had replaced the PSUs in recent units. Check here for photos taken with a thermal camera and reporting more than 72 degrees C on external parts.

So I decided to open the unit to see how it was inside. There were two signs that the power supply got replaced. The first one is that the "Warranty void" seal was a new one stuck over the previous one, whose remains were pinched in the plastic. So this box was reopened, closed and had a new stick on it.






The second sign is that two plastic legs in it were cut to accomodate the new, larger power supply. Click on the images on the right to get a full sized view.






I ran the tests again with the box open. This power supply does not heat at all. It's only the CPU which heats like mad. The small aluminum plate it completely untouchable, it really hurts to touch it even half a second. And the power supply is located just on top of that plate. So now I realized that it was not the power supply that was killing the Guruplugs, but the CPU which is cooking the power supplies. I can't leave that running when I'm not at home, I would fear that it puts my flat into fire !

Poor design

Considering the amount of heat emitted by this Marvell CPU, the box would have needed a large heatsink. I don't think anybody would have complained if the box had been 1.5cm thicker to accept a normal heatsink, with aeration holes on the sides so that the heat dissipates. Instead we have dangerous plugs which are too hot to touch, and which die in a few months of idling because capacitors can't stand sustained heat.
Also, the internal wiring is of poor quality. Two of the power output wires broke while I was taking photos of them, and a third one is about to break too. The mains input wires are similarly bad and risk to break too. And it looks like the PSU was modded by adding resistors to it, maybe to slightly increase output power. This PSU features a 105 degree capacitor though, so it may last slightly longer than previous ones.



Close-up of the 5V connector

Inscriptions at the bottom

The PSU in the bottom block

Bottom with battery, SD connector,
Blutooth controller and WiFi

Connectors side

Left side with UART and JTAG
connectors

Right side with reset button and
Micro-SD

Motherboard at the top


PSU cover removed

PSU removed

The 105°C capacitor

The PSU was partched

The PSU

First 5V wire broke instantly

Second 5V wire followed

Next one will be the ground wire

Mains wire about to break

Conclusion

The Guruplug was announced by some sites as the NSLU2 killer, but for me it's just right now a Slow Heater, and by no way will it replace even one NSLU2 as long as it heats like that. It's slower than my old ALIX, and heats a lot more. I think that one of the reasons for it to be so slow despite the high frequency is the RAM bus width : at first I thought it would be a fast system because it runs on DDR2-800, but when you see that it's only a 16-bit bus, it is equivalent to only 200 MHz for a 64-bit bus, but with higher latencies due to DDR2. My ALIX's Geode LX processor runs on a 64-bit, 400 MHz bus, so it has twice the memory bandwidth the Guruplug has. Marvell also has another CPU in the family (MV78200) with a 64-bit RAM bus, which should be better, but I have not seen it anywhere yet. The fact that the Marvell CPU consumes 5 times more power than the Geode LX and is slower is also an indication of something wrong with the design. Apparently this CPU makes it way into entry-level NAS appliances. It's probably one of the applications where the heat issues and limited memory bandwidth will not be too much of a concern. I'm really disappointed, I would have expected it either to replace my old NSLU2 (but it heats too much) or to become sort of an ALIX upgrade with Gigabit ports, but now I think that if I need Gigabit, I'll turn to these dual-gigabit mini-PCI cards. They will give me about the same level of performance without having an overheating CPU.

I don't even think I will waste my time trying to repair and adapt that Guruplug, it will probably spend the next 10 years lying in a box with other dead boards, waiting for my solering iron to pick some components out of it. However, I think that if/when the manufacturer finally decides to do a complete redesign of the product (ie include a BIG heatsink and leave some holes for airflow), then it may be an acceptable replacement for the NSLU2.

External links