Robotic Process Automation (RPA) is among the hottest enterprise technologies today, as it promises simple automation of a wide variety of tasks that humans have heretofore performed in front of computers.
RPA is especially useful when the interactions are with older, legacy applications, especially applications running on mainframes. To this end, RPA tools typically take a ‘screen scraping’ approach that works at the user interface (UI) level, mimicking how the user clicks through screens.
Automation at the application programming interface (API) level is more resilient than UI automation, but the older apps that are prime targets for RPA often lack APIs. There’s little choice but to interact with the UI – at least, according to the leading RPA vendors.
The appeal of such screen scraping is clear: achieve all the business benefits of bot-driven automation without having to touch the code on the mainframe (or legacy midrange system).
However, while such an approach is expedient, it adds to the organization’s technical debt. Instead of resolving any issues with legacy application assets, RPA simply layers on a veneer that kicks the technical debt can down the road.
Somebody will have to clean up the legacy mess someday, but it won’t be you and it won’t be today.
When RPA screen scrapes mainframe apps, the technical debt problem is even worse. In reality, implementing bots that interact with mainframe screens not only adds to technical debt, it can adversely impact cost and performance as well.
Given that mainframe apps are typically mission critical, poorly planned RPA initiatives can drag the entire business down. It’s finally time to pay off some of that technical debt in order to succeed with RPA.
The Wrong Way to use RPA to Automate Mainframe Tasks
I spoke to the director of global IT platforms at a Fortune 100 firm. The company didn’t want me to use its name, so let’s call it Horizon.
Horizon has long depended on its mainframes, and for years, personnel in offices around the globe have been writing macros to access mainframe data. Macros are simple scripts they would write in Excel or directly in their terminal emulation software of choice.
Over the years, the sheer volume of macros at Horizon became a management nightmare. It was impossible for IT to keep track of them all, leading to quality issues and overlapping functionality – a classic shadow IT situation.
To address this mainframe shadow IT problem, Horizon decided to implement one of the leading RPA solutions. Its goal was to replace its plethora of macros with a consolidated set of ‘bots’ it could centrally manage and control.
However, Horizon immediately ran into problems. “By the time that initiative showed up on the doorstep of the mainframe, it was doing things in the most inefficient way possible,” explained Russ Teubner, CEO of HostBridge, a mainframe expert who helped Horizon diagnose the problems they were having with RPA.
One RPA bot, for example, fetched pricing and item information from the mainframe almost 7,000 times to complete a single automated task. “There must be some better way, more efficient way than running almost 7,000 terminal-oriented transactions, each requiring a full travel across the corporate network,” Teubner added.
Not only did RPA exacerbate the inefficiency of its macros, but it also ran sequences of mainframe transactions more quickly than either human interactions or macros did, thus bogging down the mainframe further.
The result: mainframe transaction volume increased exponentially relative to underlying business growth. Mainframe performance became so poor the system essentially ground to a halt, taking Horizon’s business with it.
Where RPA Went Wrong
Horizon’s RPA vendor scraped TN3270 (‘green screen’) mainframe screens. As a result, each bot interacted with a sequence of one or more screens, even when a particular process required hundreds or thousands of such interactions.
More sophisticated approaches to screen scraping exist, however. Would one of them have solved Horizon’s problems?
Some mainframe integration tools leverage Extended High-Level Language Application Programming Interface (EHLLAPI), a mainframe API standard that dates from 1986. However, EHLLAPI is difficult to use and incomplete, and wouldn’t solve Horizon’s problems.
Other tools access the TN3270 bitstream directly, thus avoiding the green screen UI entirely. ConnectiQ from ActiveOps (formerly OpenConnect) works this way. RPA leaders UiPath and Blue Prism both leverage ConnectiQ for their mainframe bot integration.
However, all of these approaches are simply screen scraping under the covers, and thus none of them addresses the performance challenges that threatened Horizon’s business.
Solving the Problem
For the example above, the solution was surprisingly simple. “Instead of running seven thousand transactions, run a handful of DB2 requests,” Teubner explained. In other words, create a few SQL queries on the mainframe database that will return a single result set with all the required data.
Why, then, didn’t Horizon take this dramatically straightforward approach from the beginning? The answer is surprisingly complex.
First, running a new query on the mainframe ‘touches’ the mainframe, which is a step that many IT leaders are loathe to take for fear it will break something.
The second reason is perhaps the most broadly applicable. In this particular situation a few SQL queries would have solved the problem, but in the more general case, the problem is that a sequence of screen interactions forms a workflow that may have complex branching and error conditions.
In such a situation, interacting with each screen individually and implementing the workflow off the mainframe often appears to be the best approach to automation.
However, it’s not. Instead, running the workflow directly on the mainframe is often the best solution to this more general problem.
Fortunately, there are tools available on the market now that enable organizations to create such automations. HostBridge, for example, supports a variety of scripting languages and related protocols directly on the mainframe, including JavaScript and JSON as well as XML and Web Services protocols.
With its HB.js product, developers can incorporate complex CICS mainframe transactions that include interactions with dozens or hundreds of screens into a single service. RPA bots can then invoke such services.
The Intellyx Take
The moral of this story is certainly not that there’s something wrong with RPA per se. Rather, companies like Horizon are simply implementing it wrong.
As this article illustrates, there are some situations where process logic belongs on the host. But there are also plenty of use cases for process logic in the RPA tool as well. You could certainly come up with situations were the best course of action is a mix of both.
Like borrowing money from a loan shark, the short-term appeal of Horizon’s approach to RPA is manifest: simply install RPA and all your macro shadow IT problems would be magically whisked away.
However, there is always technical debt to pay, and in this example, the interest is usurious. Instead, make the more courageous choice to implement process logic on the mainframe in order to reduce your technical debt. When your debt comes due, you’ll be glad you did.
HostBridge is an Intellyx customer. None of the other organizations mentioned in this article are Intellyx customers.
Jason Bloomberg is a leading IT industry analyst, author, keynote speaker, and globally recognized expert on multiple disruptive trends in enterprise technology and digital transformation.
He is founder and president of Digital Transformation analyst firm Intellyx. He is ranked among the top nine low-code analysts on the Influencer50 Low-Code50 Study for 2019, #5 on Onalytica’s list of top Digital Transformation influencers for 2018, and #15 on Jax’s list of top DevOps influencers for 2017.
Mr. Bloomberg is the author or coauthor of five books, including Low-Code for Dummies, published in October 2019.