Getting Your Disaster Recovery Plan Going

by Steve Lewis
Editor-in-Chief
Edwards Disaster Recovery Directory

Many organizations make the recovery process harder for themselves - or even impossible - by not planning ahead for disaster recovery. While they may take steps to try to prevent disasters, they ignore the reality that prevention won't always work.

Creating a disaster recovery plan can seem overwhelming, given the complexity and demands of even the smallest organizations. We have found it helpful to keep the following points in mind as you proceed.

• Remember, a disaster plan is never a fixed, finished document. A good plan evolves and improves over time. Therefore, it doesn't have to be perfect the first time you do it. The important thing is to get started!

• Be systematic in your plan. Don't try to outguess Nature and plan for a flood, a hurricane, or a fire. You’ll quickly become overwhelmed by the possibilities. Instead, look at the common results from any disaster:

• Loss of information
• Loss of access to information, people and/or facilities
• Loss of personnel

Make a matrix, with these three items as the columns, and each of your activities as a row. (Beyond the obvious, your activities include things like "accounts receivable," "payroll," "real estate management," etc., depending on your situation.) Then determine how you would respond to loss of information, access, and/or personnel for each function.

RECOVERY TIME PERIODS

Following any disaster, there will be two time periods that must be planned for. First will be the immediate, disorganized, "limited-operations" time span, which will then be followed by a period of "makeshift-operations," which can be quite lengthy until normal operations can be resumed.

Typically, following a physical disaster, the limited-operations time span can extend for up to a week or more, while the makeshift-operations time span can last for several months until normal operations are restored.

This need to recover in phases is typically very difficult for top management to accept. Often, when asked to prioritize among the organization's services or products, management's first reaction is to consider them all equal. Following that, people are often unrealistic in their estimation of how quickly departments can accomplish their tasks. In one example, the organization had planned to relocate a key department to a hotsite four hours away - without realizing that most of the affected people were single parents, who couldn't possibly go there!

THE DISASTER RECOVERY PLANNING PROCESS

Once management has a proper mindset to build upon, the objective of the planning process is to systematically sort out the various issues and priorities so that a cost-effective plan can be developed which is in perspective to the level of loss exposure that the organization is risking.

The process itself can be summarized in the following steps:

• Provide top-management guidelines
• Identify serious risks
• Prioritize the operations to be maintained and decide how to maintain them
• Assign the disaster team
• Take a complete inventory
• Know where to get help
• Document the plan
• Review the plan with key employees, test the plan, and train all employees

Provide top management guidelines

Top management has to indicate the length of time the organization is willing to accept disruption of each of its key functions, and the amount of money the organization is willing to invest in procuring standby equipment, paper forms, testing, etc., as part of being prepared for an emergency.

Identify serious risks

This is a "brainstorming" process, which is best accomplished by working with the employees themselves during department or group meetings. It serves the dual role of building the awareness of the employees to the issue of disaster planning as well as surfacing potential risk areas about which management may not have been aware.

Prioritize the operations

As an example of prioritizing, most managers never think about it, but for the typical organization, the highest priority is payroll. Even if this is performed by an outside service, there is usually a terminal for remote input of the payroll data. So, in the event of a disruption, either at the source of the data or at the payroll processor, there must be a delegation of authority to someone (remember, the president, owner, etc. may well not be available) to be able to issue substitute manual advance checks.

In general, top management will have to decide how long they are willing to operate without being able to perform each of their daily operations, such as accepting customer credit applications, receiving deliveries, etc., in addition to their more obvious operations such as buying and selling. Banks need to create policies on accessing safe deposit boxes, sending out mortgage bills, commercial night depository, etc., in addition to just worrying about deposits and withdrawals. Based on its priorities, the organization can plan out how long to suspend each operation, and designate either a manual backup mode or a longer lead-time approach for each function.

Assign the disaster team

Disasters always seem to happen at the worst possible times, when the fewest personnel are available. Therefore, it is crucial that as part of the disaster plan, management appoint one person in charge of recovery and one person as second-in-command. Following this, as many specific tasks as possible within the plan should be pre-assigned. In the wake of hurricane Hugo, with most telephone service knocked out, one company in South Carolina that had not pre-assigned tasks reported that it took four days just to assemble their key personnel.

Take a complete inventory

While most organizations have records covering the make and model numbers of their equipment, they are usually not updated and almost never kept off-site. Inventory information should include emergency vendor contacts for all equipment (including microfilmers, specialty mailing and manufacturing equipment - not just computer hardware and software), descriptions and formats of all data files, and copies of all business forms used, along with the vendor contact for each.
Know where to get help

Actively collect any additional names of service or equipment providers as you come across them.

Document the plan

The plan should be written down - remembering that if the core document is longer than 15-20 pages it will never be read or used - along with the various assignments, updated inventory, and all key phone numbers. Key personnel should have a copy of this documentation at home.

Review, train, and test

The key types of tests applicable to contingency planning include:

• Blink test
• "Independent" expert assessments/structured walk-through
• Component tests
• "Pull-the-plug" evaluation
By assigning specific people to each of the key task areas, it is possible to generate the most useful and least expensive of these tests - the "blink" test. This is where personnel speak up and simply say "I can't do that," or "I don't retain that information," etc. This can include a yearly review with experienced employees and can be part of the introductory training for new employees.

Following this, non-assigned employees will be asked to review the plan as it pertains to them. As part of this process, they will be encouraged to provide their independent comments on the plan, based on their detailed expertise and familiarity with the daily ebb and flow of their specific operations.

The next step is to test those plan components which can be tested independently of one another. Specifically, this includes items such as the recovery of backup files and the procurement and testing
of dial-up backup links to hotsites, etc.

With respect to "pull-the-plug" exercises, it may not be feasible to bring the entire organization "down"; however, the typical day-to-day mini-disasters which knock out installations and affect the entire organization should be treated as this type of a test. Following the recovery from these events, the results as well as lessons learned should be documented as if the event were a planned test, and any corrective actions warranted should then be taken.

Conclusion

At minimum, you should do the following:

• Designate a second-in-command,
• List individual responsibilities ahead of time, and assign specific people to each task,
• Protect critical paper records,
• Keep copies of all of your forms off site - especially checks and purchase orders,
• Set clear priorities among your activities and specify beforehand the longest amount of time you are willing to be "dead in the water" for each of your activities,
• Have backup communications, including dial-up to replace “leased-lines” and radios to replace telephones and cell-phones,
• Keep a copy of your disaster plan at home

Finally, don’t try to go it alone. People throughout your organization may have dealt with disasters before and may have thei r own ideas about what they’d do in an emergency. Beyond that, your competitors and colleagues at other companies in the region can be resources in the planning process. If you include their input and advice you’ll not only benefit from their experience, but also find it easier to gain their cooperation for mutual assistance when the inevitable disasters actually happen!