I, like many others, am interested in getting a better grip on Docker, Docker Swarm, and Kubernetes. I want to be able to understand how things work across physical computers, so, ideally, I would use physically separate computers in my training lab setup. This would be cost prohibitive using standard server hardware, so alternative means are required. Enter the Raspberry Pi Cluster.
My initial thought was that I wanted to do this as cheaply as possible. I figured the best way was to do what others had done and buy inexpensive parts and build the cluster.
I looked at several builds and then went part hunting on Amazon.
Optional (but make things neater and easier)
6-node Raspberry Pi Cluster Parts List
|Item||Quantity||Price Each||Price Total|
|USB Power Supply (w/cables)||1||$36||$36|
|1ft Cat 6 Ethernet Cable (6 pack)||1||$9||$9|
|Heat Sinks (2 piece)||6||$5||$30|
|Hook & Loop Straps||bunch||$7||$7|
|Small Cable Ties||bunch||$4||$4|
|32 GB SD Card||6||$15||$90|
|Raspberry Pi 3||6||$35.50||$213|
1/2-day hardware build
1/2-day software build
I made the mistake of equating simple with quick. I imagined that the hardware assembly would not be difficult, but I wrongly associated this with the idea that it wouldn't take much time. I was wrong about the latter.
Although the process of assembling all of the hardware was a simple process it took me a good half of the day on a weekend to build everything and connect everything. Strapping everything together was easy.
My fingertips were quickly becoming sore from working with the small nuts and screws. After they began to hurt I luckily remembered that I had a set of nut drivers in the garage. The use of a 3/16 inch nut driver made things much less painful and increased the speed with which I built the stack.
After I used small zip-ties to pretty up the cables, I used a wire snipper to cut off the excess of the zip-ties. Otherwise the thing looked like a spined beast.
I took a break for some lunch and then tackled the software side of things.
Once again I made the incorrect assumption that simple meant speedy. Although the actual amount of typing at the command prompt I needed to do with minimal, the time for the downloading, extracting, installing, and configing took exceptionally long. And it wasn't due to a slow internet pipe. (I am able to routinely download at 200+ Mb/s) All together, another half day spent on software. And this was just to get the basics installed and barely tested.
For the software build I used the notes in Alex Ellis' GitHub Gist on the subject of Kubernetes on Raspbian Lite.
So what do I have for a whole day's worth of work? A functional 6-node Raspberry Pi cluster that is physically capable to standing on its own and has a LAN port to join a larger LAN which would provide internet access.
However the things is kind of an ugly kludge and isn't very stable. It is fairly easy to bump the top end of the contraption and knock the whole thing over. I have a bunch of spaghetti cables which can get caught on just about anything.
What I wanted at this point was to put it all in some kind of case that would contain it all and make it more stable. I was even thinking about building the stack as a 2x3 instead of a 1x6.
IOW I was thinking about putting more money and time into it to make it more resemble something that I imagined I would feel more comfortable with in the long run.
If I'd Have Only Known
I don't know why I was searching or what I was searching for, but somehow, one day soon after I finished building the cluster, I stumbled across Picocluster's website. My jaw dropped. Here was something more along the lines of what I had envisioned. And the price! Not much more than what I paid already. (Don't forget, I was still willing to pay a bit more and spend a bit more time to get something better than what I built.) I could kick myself.
Additionally, besides the physical hardware, Picocluster offers premade SD Card sets with your favorite cluster software preloaded and configured. That had to be quicker and easier than doing everything manually from scratch.
I really wanted to let the next person who was going to build a homemade cluster to know about these. However, I was just speculating. Without having built one of their kits, I could not personally recommend them.
I decided What The Heck. I'll email Picocluster and ask to do a review if they would send me a kit. They agreed!!
What I asked to receive and review was a Pico 5E with a Kubernetes SD card set. They agreed and shipped it out to my address. (n.b. They did not send me any RasPis. They are available as an "Advanced Kit" upgrade from Picocluster.)
The shipment arrived well packed and without any damaged or missing items. When I unpacked the carton, I was pleased to see things well labeled and protected.
It took me a little less than half a day Saturday to build the kit. And literally only minutes to insert the SD cards and fire it up into a working Kubernetes cluster.
One of the things that I liked about the kit was the completeness and clarity of the instructions (on their website) and the well labeled pieces. It was reasonably easy to assemble the mechanical aspects of the hardware.
However, there is one challenge that the Picocluster kit brings to the table. That is the requirement to do some some basic electrical work. None of it was particularly challenging, but it was one area that could use some improvement.
This was also where the first problem I ran into came about. With the moving of wires around as I assembled the kit, one of the connectors separated from the wire. I pulled the others off to check to see what condition they were in. They were ok outside of the one.
Luckily I am a maker and electronic hobbyist and already had a spare connector and crimping tools on hand. If I had not had those items, I would probably would have needed to make a trip to the local hardware store (additional time and money) or waited a few days for Picocluster to send me a replacement part.
There was also the need to cut the connector off of the fan's wires as well as strip the plastic off the ends of several wires.
I already had a pair of cutters and a wire stripping tool in my workroom. Had I not, I would probably screwed up a pair of scissors and/or my teeth stripping the ends of the wires.
Although it will add to production cost and time, I believe Picocluster should do this to avoid the need for the end user to do this work and possibly need to purchase a tool.
Which setup is best? I think that depends. As the title of this article suggests, the homemade cluster is a bit of a beast, while the Picocluster looks a lot nicer and it a lot more balanced on its feet. If you are going for looks, then a Picocluster is the way to go and your probably want to even go with the optional accessories -- the Blinkt! LEDs for PicoCluster.
I would also spend the extra money and get a Pico 5E if I planned on moving the cluster around often - for example traveling with it. It would be much more convenient to to have a nice cube that could slide into a foam packing core and be pulled out and plugged in easily. If traveling, I assume it will be to "show off" your cluster, so again, the beauty over the beast may be to your liking.
Let me discuss the idea of only five RasPis in the Picocluster versus the six in the homemade cluster. If you are like me, you are just interested in learning the basics of cluster computing and not looking to locally model the universe. In such a case, five RasPis is more than enough. And if you want to cut costs, go with the less expensive Pico 3S.
You'll be able to have one as master and two as slaves.
Then there is the other side of the coin, going with more than 5-6 nodes. If you are going for computing power, then money is less of a concern, per se, than getting a functional mid-sized cluster together. In that case I would definitely go with a Pico 10, Pico 20, or Pico 48. I would probably also like the ability to get the Raspberry Pi computers from the same company as the Pico (IOW all the parts required, minus the Pis). As often a single source, and single invoice, is prefered by business and educational institutions.
If cost is a big concern, then I suggest first rethinking your plan to build a 5-6 node cluster on the cheap and move down to a 3-node cluster, smaller capacity SD cards, and maybe do away with the separate switch/hub (and the isolation it brings). And you are going to have to allocate a whole day for the build because of the software side of things taking so long.
The kicker is the support you get. If you go with a Picocluster you get the assistance of their support staff to put the hardware together and get the software tweaked to meet your needs. With a DIY project you are pretty much on your own.
In summary, I think the Pico 5E Raspberry Pi cluster Advanced Kit (the one with the RasPis included) along with the premade Application Cluster Set finds the sweet spot between cost, appearance, time required, and convenience. The cost of $579 versus $479 is definitely worth that additional money for what you get.