Abacus System Administrator discusses the “day-to-day” aspects of the popular service offering
It’s no secret that Operations Management can help companies refocus their teams from day-to-day maintenance tasks to more productive and profitable projects. Companies that are still running the AS/400 in their IT infrastructure can benefit from relying on a managed service provider to keep their IBM i environments running smoothly. Today we’re speaking with Tom McLendon, Systems Administrator for the Abacus Solutions Managed Services team, about what the Operations Management service offering entails for the IBM i specialized team.
McLendon’s career in IT became more IBM iSeries focused in 1987, and Abacus Solutions has been fortunate to have him as a System Administrator since 2013. Like many at Abacus, McLendon wears a variety of hats for the Operation Management team, caring for customer environments, managing backups, and just “trying to keep our system monitoring software happy.”
When asked about the scope of his responsibilities, McLendon credits his team’s success to their strong sense of ownership. With the hardware aspects of operations management handled by a separate, dedicated team of engineers, McLendon and his fellow system administrators are free to focus on the software side of things.
________
Let’s start with the basics. What is Operations Management and how does it differ for IBM i environments?
In its simplest form, Operations Management entails watching systems passively and responding to any problems that the monitoring software reports or that the customer asks about. From an “Abacus perspective” we do pretty much what any other managed services provider does – monitor the hardware and software of customer systems, collaborate with customers to set up said monitoring, and customize that monitoring to some extent.
Operations Management is basically the same concept when it comes to IBM i versus Windows (PC) or UNIX systems, but since the IBM i is a unique platform there is a slightly different language in use. The operations might be the same, but they’re executed using different commands, so there is a degree of specialization necessary to do this for the IBM i platform because its market share is “declining.”
You’re referring to the fact that a lot of sys admins with IBM i experience are exiting the industry?
Yeah. It’s not as popular of a platform as it was 20-30 years ago. Most of the remaining talent is considered “grey-beard” like me. Because of this, it’s becoming more and more rare to find administrators that are well-versed in the IBM i. That’s another way that Abacus differs from the competition. Our engineering staff is a mixture of young and old, and we make a conscious decision to invest a great deal in their training so that they’re comfortable with the IBM i platform.
Of the Operations Management we perform, how much occurs in the Abacus Cloud versus on Customer Premise?
About 80% of our customers are running in our proprietary Abacus Cloud, and 20% are on customer premise. We can offer similar services to customers that want to keep Operations Management on premise, but the majority of them shift to the cloud as it becomes more and more expensive to host and own their IBM machines. Eventually it becomes a better deal for Abacus to run it for them.
What does the day-to-day operations for IBM i Operations Management look like for your team?
Most of our day-to-day work involves monitoring and responding to passive alerts from the Halcyon monitoring software. We get a lot of alerts in a day. The team determines which alerts to respond to and how to address them. We either notify the customer or resolve the issue ourselves. If the customer ends up needing assistance, we open a ticket for them and start to dig deeper into the issue.
One distinction I’ll make is that Abacus does not take over application work. We only provide OS work. It’s our belief that there’s no replacement for the institutional knowledge that a customer will have of their own applications, so we make it a point to not offer those services.
How many alerts do you typically get in a day?
Last time I checked I was getting some 1200 emails a week.
That’s a lot for one person!
If you really want to have fun, watch [our CTO] Josh Osborne’s email inbox. It’s like watching a waterfall.
How do you keep from becoming overwhelmed by all the alerts?
You learn quickly how to determine which alerts are most important. But we still pay attention to the less urgent alerts as well. For example, if we see that a customer is using 90+% CPU for several days straight, we’ll notify them to ease their workloads. If that’s not possible we’ll bring in the sales team to suggest the customer expand how many CPU slices they need.
You mentioned that being on the software side of the Operations Management team involves having to wear multiple hats. What else are you and your teammates responsible for?
In between alerts and customer tickets, the team works on utilities for internal use to improve documentation and monitoring. We also run and monitor hundreds of automated backups every night.
________
We’ve never lost a system, but we’ve saved a lot of ‘em. – Tom McLendon, System Administrator
________
Other than the monitoring software, do we use any dedicated tools to manage our customers’ IBM i environments? How do we handle things like job scheduling and backup management?
The monitoring software we use for most aspects of Operations Management is Halcyon. Job scheduling is application centric, which we don’t support as part of our service offering, so we don’t need a tool for this. As far as backup management goes, we run our automated backups using the data domains from the iSeries.
Are there other processes in addition to backups that are automated?
We automate everything we can. The main benefit of combining automation and managed services is that you need fewer staff on hand. By leveraging monitoring software like Halcyon, you can theoretically watch more with less people. And if it’s all set up correctly, which is true in our case, you can catch a lot more than a human can. We have hundreds of systems monitored by our specialized administrator team plus our NOC (Network Operations Center).
Sounds like your team works pretty efficiently.
We’ve never lost a system, but we’ve saved a lot of ‘em.
Can customers interface directly with and perform their own Operations Management when they want? Or is it all done through the Abacus engineering team?
Some of our larger customers have their own administrative staff and we collaborate with their team with regards to what we monitor. In those instances, who monitors what is determined beforehand. But most of our customers don’t want to have to worry about any of that, so we usually take over all their Operations Management for them.
It sounds like our Operations Management offering has some flexibility to it. What can Abacus customize about the service and how does it change what the customer gets?
We offer three different levels of services: Tier 1, which is our general, and basic, operations management offering. Tier 2 involves a little customization and additional support in the form of notifications of critical issues, but we won’t fix applications. Tier 3 takes it a bit further where our support team will respond directly to fix specific errors or messages, such as disk file fill ups.
Anything last thoughts you’d like to share about how Abacus approaches Operations Management?
One of the things I appreciate the most about how we offer Operations Management is that we’re very careful about setting expectations with customers during the sales/on-boarding process. We make it very clear that we don’t deal with customer applications. I’ve previously worked on teams that “promise everything” and then are unable to deliver. There’s something to be said about knowing your limitations and setting realistic expectations for the customer.
________
With much of the existing IBM i platform talent pool retiring out of the industry, companies are needing to find ways to keep their IT infrastructure running. Abacus Solutions combines advanced software tools with engineering teams dedicated to specific parts of a customer’s AS/400 environment to provide comprehensive Operations Management. With our Managed Services, customer IT teams are supported with unmatched AS/400 expertise and know exactly what they’re getting out of the service offerings provided by Abacus. If your company is looking for IBM i-focused Operations Management support, contact Abacus Solutions today.
Note: Parts of this interview have been edited for length and clarity.