Kurt Keller
2003.March
For some time now, Linux has been cropping up here, there and everywhere, starting from embedded systems, like your coffee machine, to PDA's, PC's, workstations, servers and up to multi-million dollar high end machines. Some of these platforms are common enough so lots of people do have a chance to lay hands on it and check out how Linux performs. Others are harder to get at. Some platforms most people never will be able to test by themselves. Over the last few months I've been running some tests with Linux on one of the platforms most Linux enthusiasts will never have a chance to use: IBM's zSeries mainframe computers, formerly called S/390.I wouldn't exactly call myself a Linux expert, but surely I am a Unix guy, working mostly with Solaris on the sparc architecture and with FreeBSD on intel machines. Now I've also begun to have a bash at Linux on intel and IBM mainframes. For me Linux is not a separate OS family, as it is for some people. I consider it to be just another Unix dialect, no more and no less.
Here are my experiences with and thoughts about zLinux.
Such a physical machine can be split into several LPAR's, logical partitions. Each of these LPAR's does have physical resources such as CPU, memory, disk, assigned to it out of the pool available for the whole machine. For people knowing the SUN architecture, the E10'000 family would make a good comparison. People familiar with intel PC's should imagine the zSeries as a big box with several PC's built into one single housing, or something not that far from a rack full of blade servers.
Each LPAR does run it's own OS and is totally separate from all other LPAR's. It is possible, to run zLinux natively in an LPAR. However, as there are only 16 of these possible, doing so would not make much sense; such a Linux instance would bind quite a lot of monetary resources. So the usual way is to run zVM in an LPAR and create zLinux instances under zVM. zVM is a virtual machine manager, just as VMware and can manage hundreds of zLinux instances in one LPAR.
There is even a mainframe hardware emulator, called Hercules, which runs under Linux and Windows. So it is possible to have a whole zSeries machine emulated in software on your intel machine and I have seen some IBM representatives which had this setup on their laptop.
When talking to IBM technicians it often sounds as if SuSE was the only game in town for the zSeries and if you try to run something else you'll often be met with disbelief. In the beginning I had the impression that SuSE in fact was the only distro available for the IBM mainframe. So the first installation I did was SuSE. Later I learnt that other distributions were available as well and finally switched to RedHat, because that was really what I want to run should we add Linux to our portfolio. If possible in any way I want to avoid running a mix of different distributions.
For the one native installation (Linux directly in an LPAR of it's own, without underlying zVM) we needed to create a boot tape and boot the LPAR up from that tape. Under zVM the booting process is similar, but you don't need to physically create a tape. Instead you can fetch the required files to your zVM environment and emulate sending it to the tape reader.
For actually fetching the bootstrap code and later the software to be installed, an FTP server must be available. Depending on you hardware you might also require some special device drivers. We were required to install via an OSA-Express gigabit network adapter. The device driver for this was only included with the latest version of SuSE, but the documentation on how to activate and use it was outright wrong. For copyright reasons the OSA-Express driver is not included with RedHat, so it is necessary to download an OCO (Object Code Only) Module from IBM's website and compile a new bootstrap image on a separate Linux machine. Getting past the obstacle of activating the OSA-Express network card cost us a couple of days. It's not such a big deal if you know what is required and the instructions are correct, but if not...
Well, this definitely was not what I had expected, even though I knew well that mainframes are not the right tool for CPU intensive work; if good parallel I/O performance is required, mainframes are said to have the lead. IBM claims you can have literally hundreds of zLinux instances running under zVM and this way replace lots of smaller machines. The zLinux instance I had running for this test was the only one, nothing else running. It had 256 MB of physically available RAM (the whole LPAR `only' had 512 MB, but ask some mainframe guy to give you 2 GB of RAM and they will look at you in disbelief, just like you were telling them you want a monthly salary of US$ 100'000.-) My zLinux installation had the whole IFL processor, which should be able to run many instances of zLinux in parallel, all to itself.
As a comparison, the intel machine had 512 MB of RAM, an 1.4 GHz processor and a single IDE disk drive. On this machine there were even other things running during the test, and still it finished in half the time.
Comparing only MB and MHz or GHz might not make much sense, we need to know what all this hardware costs. I don't know exactly what the intel machine had set us back, but I would guess around 1'500.- Swiss francs. I do know a zVM license costs 100'000 Swiss francs and the list price for an IFL processor is about 300'000 Swiss francs. Wow! For 1'500.- Swiss Francs I get a machine which runs twice as fast as the machine for which I pay 200 times more for the CPU only! Sure, over the lifetime of a server, hardware cost is only a small fraction of the total cost, but these numbers are quite a league apart.
While trying to find out why performance was so poor on the zLinux machine, I stumbled upon another interesting fact. Mainframe computers are said to be the I/O champs. Creating a 512 MB file full of the character ``0'', took about 75 seconds. I had done something similar just a few hours ago on the intel machine and had the impression that it was way faster there. Finally I ended up doing a series of tests creating sequential files on various platforms. Indeed, when having DMA enabled on the intel platform, I had a performance on an IDE disk which was roughly 5 times better than on the mainframe. What a surprise! Only when I used a bunch of striped disks and striped the stripes themselves once more, got I similar performance on the mainframe. So much about the much cited I/O performance on the mainframe.
The next thing we tried was porting our proxy infrastructure to zLinux. This was no problem and actually, after getting astronomical amounts of RAM (astronomical for mainframe people, moderate for other people), the zLinux instance created for this test could take over the load of two moderately equipped Sun Ultra 420R servers, serving about 5'000 users or 160 requests per second. After all, this looked fine and if needed, we might replace one of our three proxy servers with a zLinux instance. When doing the tests with the proxy, suddenly my colleague turned all white gasping. I wanted to find out whether one instance of zLinux could affect the performance of another one. So I logged in to my test statistics server and did a simple unzip of some files. While I was uncompressing these files, the performance of the proxy literally collapsed and it was was almost unusable!
We did some more testing and also had IBM people come round to check whether we had some misconfiguration somewhere but our configuration was quite ok, especially the configuration of Linux itself was better than most sites they had seen, they said.
The largest zLinux installation, for example, is in the USA. A webserver provider hosts over 6'000 webservers on a zSeries machine. Each customer does get his or her own zLinux instance, his or her own virtual Linux server. Typically a large number of webservers are idle most of the time. By running all these servers under zVM on a zSeries machine you can save big by sharing all available resources among all the servers. With separate machines you'd not only need lots of floor space and an enormous cooling system, but each machine would also need to be fitted with enough memory and CPU cycles for expected peak load. Most of this RAM and CPU would be wasted, because it sits in your server room doing nothing for 99% of the time. With zLinux you can assign enough virtual resources to each instance but only need to have as much physical resources, as all the virtual machines actually do use at any given time. Another plus for such an installation is the possibility to simply copy an OS image in order to setup a new virtual zLinux machine. So creating a new server for a new customer can be a matter of minutes.
Air New Zealand, for example, consolidated 150 file, print and email servers onto a zSeries machine under zLinux. If these servers are mainly small and for some reason can't be moved to one big fileserver, maybe because each section must have their own fileserver, this may well make sense.
Something heard of then and when is the superb stability of mainframe computers. I do have my doubts whether moving to zLinux solely for reasons of stability is a valid argument. After all, if you have a problem with the underlying zVM, which happened twice in our installation, all your zLinux instances will go down as well. Or if the mainframe hardware needs a power-on-reset, which does happen as well, everything needs to be shutdown. And last, but not least, if you have stability problems with your non-zSeries servers, then you're doing something wrong anyway. If it's hardware related you should buy quality hardware, if it's software related I doubt you'll do any better on zLinux; you still need to know what you're doing whether running on zLinux or something else.
A thing which IBM mentions often are the networking capabilities of the mainframe. There are so called CTC connections, IUCV connections and hipersockets. These can be used to network different LPAR's or guest systems under zVM within the mainframe computer itself. With the latter two communication is done by way of memory-to-memory copy and thus these connections are blindingly fast. However, whether you can make use of these features very much depends on the structure of your network and your security requirements. I'd advise you to research this subject well, before using it as an argument to buy into zLinux. And don't listen to IBM too much; they always say what you want is no problem at all, without listening to what you want. When they wanted to install zVM at our site, we had to be quite persistent to get things installed the way we needed it and not the way they wanted to force upon us.
I see many companies playing around with Linux on mainframe computers, but only few of them are really saving money this way. My guess is that often they might be trying to reuse mainframe hardware which would otherwise be idling, because some mainframe applications have been replaced with applications running on other platforms.
| Linux for S/390 | http://linuxvm.org/ |
| IBM | http://www.ibm.com/servers/eserver/zseries/os/linux/ |
| PINBOARD | http://www.pinboard.com/ |
| HighTechSamurai | http://kurt.www.pinboard.com/ |