Disaster Recovery Planning: Sadder But Wiser

Disaster recovery planning always seemed to get pushed to the bottom of the to-do list, despite constant reminders from the executive who oversees contingency planning activities.

In the post September 11 world, however, disaster recovery (DR) planning is experiencing a renaissance, of sorts, as CEOs want to know: "What will happen to the business in an emergency?"

"Disaster recovery was always the project left on the shelf," says Heidi Rybak, senior manager and head of Business Continuity, Arc Partners (New York), a financial services consulting provider. "9/11 was a wake-up call for executives."

But how long will the heightened awareness last? Insurers are facing a slumping economy and tighter budgets. Still, CIOs say the post-9/11 realities facing all firms make ignoring disaster preparedness a non-option.

At New York-based MetLife ($302.5 billion in assets), disaster and business continuity planning has definitely moved up on the priority list, says Mike Ehrenzweig, vice president and one of the executives who helps oversee MetLife's disaster planing. "Business continuity has always been a part of what we do," he says. "We gave a report to the board on disaster planning after 9/11. The report was actually scheduled to be given before 9/11, but it kept getting pushed back.

"Clearly, after September 11, business continuity is getting more attention," continues Ehrenzweig. "September 11 has made our job easier; you no longer have to convince people how important this is. Now when there is a fire drill, everyone pays attention. People no longer shut their door and continue working."

At MassMutual (Springfield, MA, $220 billion in assets), parent of former World Trade Center (WTC) tenant OppenheimerFunds, Inc., there is obviously an increased focus on reviewing DR plans, according to Christine Modie, executive vice president and CIO. "We have been planning a soup-to-nuts review of our business continuity and disaster recovery plans," Modie says. "It will obviously get more attention now."

Technology suppliers are hoping that the increased attention will mean increased spending by insurers, but insurance technology executives don't see an immediate jump in DR spending. "I am not seeing an increase because we are already spending on it," says Steve Sheinheit, CTO at MetLife. "It's not just a case of spending. We have a dedicated staff that works on disaster recovery. The priority is to make sure the plans remain up to date."

Redundant Tech Is Key

David Snow, COO at Empire Blue Cross Blue Shield ($2.2 billion in assets, New York), one of the WTC's former tenants, says that since he and his superiors "were very pleased" with the DR plan after if was enacted on 9/11, there probably will not be large increases in disaster recovery spending. "We have made some adjustments in the plan," Snow says. "The plan worked so well because of the investments we made before 9/11. We built redundancy into our technology three years ago.

"So when we lost the offices in the World Trade Center," which included a call center, "the calls were routed seamlessly to other service centers," Snow says. "Customers didn't know we were in the World Trade Center." Snow considers Empire's redundant technology as a key to helping the company survive 9/11 attacks. "The core to disaster recovery is having redundancy in place to handle the loss of a system. Obviously, we never built the redundancy with the intent that our office would be destroyed, but with backup call switches, routers and service centers the incoming calls easily rerouted," he says.

Because all of Empire's operations were routed to other locations—including all incoming documents that are scanned and imaged immediately upon receipt—almost no data was lost. "Everything that comes in the door is digitized," says Snow. "In fact, for the few weeks after 9/11 we saw a decrease in incoming calls because people were not going to New York City hospitals. There was an overall decrease in medical services. There was such a reduction of incoming claims that we worked down the inventory of claims until there were no claims left to process."

Other companies, fresh with the memory of 9/11, have begun to build redundancy into their systems, says Monica Doshi, product manager, Network Attached Storage (NAS) Division, Quantum Networks (San Jose, CA). "We have seen a huge up-tick in companies inquiring about redundant back-up storage," Doshi says. "Companies are looking to back up data in real time to remote locations."

OppenheimerFunds utilizes real-time back-up for all data. "They did not lose a single transaction that day," Modie says. "They recently changed from nightly tape back-up to real-time back-up using SunGard Availability Systems."

As more companies move toward real-time back-up—something that will make recovery much easier—they are also realizing that spreading data center operations can reduce risk (See I&T's January Cover Story, "Swinging Back to Centralization"). MassMutual has three data centers, one each in Hartford, Denver and Springfield, MA. In fact, the three separate locations helped MassMutual and OppenheimerFunds in their recovery effort. "We took advantage of the facilities that we had," Modie relates. "We had spare capacity in Springfield and Denver and some limited capacity in Hartford. We distributed the load through the three locations."

Some companies learned a harsh lesson about the geographic separation of data centers on 9/11. "A few companies had a hot site a couple of blocks away in downtown Manhattan," points out Arc Partner's Rybak. "Most companies thought that if an office was lost, say to a fire, the company would switch to a site close by. No one thought that blocks of buildings would be closed."

Greg MacSweeney is editorial director of InformationWeek Financial Services, whose brands include Wall Street & Technology, Bank Systems & Technology, Advanced Trading, and Insurance & Technology. View Full Bio

1 of 2