17 July 2017 marked the IBM announcement of the latest IBM mainframe: the IBM z14. Congratulations to the IBM Z team on a successful launch of what looks to be yet another in a long and distinguished history of superb enterprise computing platforms. Similar to the z13 launch in 2015, the z14 features several new, impressive I/O features and functionality that will provide tremendous business and technical value to IBM Z clients. In the remainder of this blog post, I’ll discuss the highlights of what I view to be the three most important: RoCE Express2, FICON Express 16S+, and zHyperlink Express.
RoCE Express2
RDMA over Converged Ethernet (RoCE) is a constantly evolving technology, and with RoCE Express2 IBM Z keeps the IBM Z RoCE technology on par within the industry. The 10GbE RoCE Express2 feature is installed in the z14 PCIe I/O drawer as a native PCIe feature. It does not use a CHPID, instead it is defined by using the IOCP FUNCTION statement. RoCE Express2 features may be direct attached (point to point), or use switched connections via enterprise-class 10GbE switches. However, the switches must support 3 critical features themselves: priority flow control (PFC) disabled, global pause functionality enabled, and no use of firewalls, IEDN, or routing. The maximum unrepeated distance point to point is 300m. In my opinion, the most important advance over the prior generation RoCE Express feature is the increased virtualization/sharing capabilities. RoCE Express on z13 supported 31 Virtual Functions (VFs) per PCHID, while RoCE Express 2 supports a total of 126 VFs per PCHID (63 VFs per physical port).
FICON Express16S+
The FICON Express16S+ channel shares many of the same characteristics as the prior generation FICON Express16S channel. However, there are some enhancements that have been made that have resulted in truly impressive performance gains. In IBM performance testing, IBM reported results for zHPF small data transfer I/O operations of just over 300,000 IOs/second. That is over 3x the IOPS capability of the FICON Express16S channel with z13! In testing a workload using zHPF and a mix of large sequential read and write data transfer I/O operations, IBM reported a maximum full duplex throughput of 3200 MB/sec (reads + writes) compared to 2560 MB/sec for the same test on z13 and FICON Express16S channels. In other words, the z14 with the new FICON Express16S+ channels can run full line rate, full duplex. The performance testing numbers using the FCP protocol which is typically used in Linux on Z environments was even more impressive when compared to z13 and FICON Express16S. What does this mean? If you have currently implemented SSD/flash storage in your mainframe environment (hybrid or all flash arrays), or are planning on doing so during this mainframe product cycle, you will likely see significant performance improvements in terms of increasing IOPS and reducing latency with the FICON Express16S+ channels. Just remember, upgrade the FICON storage network connecting it all together.
zHyperlink Express
This is the most revolutionary mainframe I/O technology to appear since FICON made its debut nearly 20 years ago. IBM Fellow Harry Yudenfriend and his team really have invented something amazing with this new technology. IBM zHyperLink Express is a direct connect short distance (150m), IBM Z I/O adapter offering extremely low latency connectivity to IBM FICON attached DS8880 storage arrays. Working in conjunction with your existing FICON SAN infrastructure, zHyperLink fosters a new I/O paradigm for IBM mainframes. zHyperLink is a new technology that works with your FICON SAN infrastructure to provide up to a 5x reduction in I/O latency times for DB2 read requests with the qualities of service IBM Z clients expect from I/O infrastructure, for DB2 v11 and v12 with z/OS 2.1 with patches. it dramatically reduces latency by interconnecting the z14 CPC directly to I/O Bay of the DS8880, using PCIe Gen3 x 8 physical link (up to 150m distance).
Part of the zHyperlink technology contributing to the impressive latency reduction is using synchronous I/O as the I/O model type. Synchronous I/O is sometimes referred to as polling driven I/O, in contrast to asynchronous I/O which is interrupt-driven. Synch I/O allows the operating system to read data records synchronously, and avoid the scheduling and interrupt overhead associated with Async I/O operations. Where these types of improvements really come into play are when using Sync I/O techniques to attach to SSD/flash storage. Scientists from Intel have demonstrated nearly 40% reductions in latency using Sync I/O techniques with 4K block size traffic with Linux applications.
There are caveats described by IBM regarding the performance testing for zHyperlink Express and I encourage you to read the details of this new technology. One important point to emphasize: zHyperlink connections complements FICON connectivity, it does not replace them. Also, they only support z/OS and ECKD.
Steve Guendert, aka “Dr. Steve”, is in an IBM Z senior engineering role based in Poughkeepsie where he is on the IBM Z Hardware architecture team. He works on all aspects of IBM Z I/O and timing architecture, including the Server Time Protocol (STP), FICON and zHPF architectures. He is a member of the Mainframe Hall of Fame and the IBM Academy of Technology.