By Destiny Washington
The IT Operations Command Center is a centralized location where the Office of Management and Enterprise Services technical staff provide incident and event management to monitor and manage an agency’s service. “An incident is a disruption in service and incident management is the practice of restoring service as quickly as possible,” said Keith Hillemeyer, Technology Services manager.
ITOCC oversees systems that keep the state running, such as infrastructure equipment, wireless systems, databases, firewalls, servers and applications.
With updated processes, the ITOCC team has seen case numbers go down because incidents are being caught earlier. “Our goal is to respond to calls before they even come in,” said Hillemeyer.
These are the people working in the background protecting, connecting and getting the service team ahead before an incident happens. It’s a job most folks don’t know about.
ITOCC uses Paessler Router Traffic Grapher network monitor software to ensure computer systems are running smoothly and that no outages occur. The team relies on PRTG to work toward delivering data to other applications as it happens, meaning you get data immediately.
The ITOCC team uses Microsoft Teams group functionality to directly contact agency customers for support of their major systems. Agencies such as Oklahoma Department of Human Services, Oklahoma Department of Public Safety, Oklahoma Department of Transportation, Oklahoma State Department of Health, and Oklahoma Department of Veterans Affairs use this channel to directly report issues.
This process has helped decrease call times as the team is able to communicate the issue and work on getting it fixed before many employees and Oklahomans are impacted. ITOCC monitors over 2,000 network routers and switches, 36,900 devices and has responded to over 27,400 service degradations or outages.
“Our updated channels allow agencies to directly report issues ranging from application outages to network connectivity to state agency website status,” said Aleta Seaman, senior director of IT Operations.
In addition to becoming more accessible to customers, ITOCC is focused on the Incident Command System. This is a standardized model for command, control and coordination of a technology response when there is a system outage. It provides a means to coordinate the efforts of individual towers as they work toward the common goal of stabilizing the incident.
To expand awareness of the Incident Command System, ITOCC created an internal initiative where a different tower leader takes the role of incident commander for a week. This experience allows leadership to better prepare their teams in case of an incident. In this role, various tower leaders can personally experience the communication and information flowing in and out of ITOCC.
“This brings all of our towers together. Incident Command makes our team work together and run toward the fire to put it out,” said Hillemeyer. “Because of this process, customer experience has improved and new processes were created.”
Future transformation steps for ITOCC include a move to an updated critical event management system and IT service management system.
The team uses ServiceNow to update its change management and outage reporting processes as well as provide an up-to-date system status page for our state’s most important IT services and applications.
AtHoc is a critical event management system to help notify state employees of a technology outage or other critical event such as a cybersecurity alert. This tool allows notification to anyone, anywhere on any state-owned device with one click.
Readiness and response for state systems are taken very seriously. When an event has the potential to escalate into an incident, disruption, disaster or a full-blown crisis, the ability to communicate specific warnings and instructions becomes even more crucial.
These products advance Oklahoma’s technology monitoring and enhance the way we serve Oklahomans.
