13 Aug

Nesting Selectors with Retrospect Backup

Nesting Selectors with Retrospect Backup

Retrospect selectors offer a surprisingly  powerful method to precisely control what does, or does not get backed up. However creating complex selectors can be a bit mind bending.

This article discusses nesting selectors within each other to create a flexible backup management toolset.

Ground Rules

  • Retrospect Selectors have sections to INCLUDE and EXCLUDE specified items
  • EXCLUDES always take priority over INCLUDES
  • You can use another selector in a selector item definition.

However, doing this presents some subtleties.

How it works

Some experimentation suggest that it works like this:-

Selectors intended for inclusion in other selectors should only specify INCLUDES OR EXCLUDES but not both {it gets too mind bending otherwise}

THUS:-

Selectors specified in the EXCLUDE section of the parent MUST specify the files to be INCLUDED {in the exclude}.
Similarly
Selectors specified in the INCLUDE section of the parent MUST specify the files to be INCLUDED {in the include}

This behaviour CAN be reversed by using the IS NOT operator when specifying the Child Selector but this starts to get too mind bending

Put another way if you include a selector which says EXCLUDE something, in an EXCLUDE section then those items will be EXCLUDED from the EXCLUDE, and will be INCLUDED in the backup.

THUS it appears one can apply traditional mathematical sign rules to nesting Selectors
INCLUDE + INCLUDE = INCLUDE
EXCLUDE + INCLUDE = EXCLUDE
INCLUDE + EXCLUDE = EXCLUDE
EXCLUDE + EXCLUDE = INCLUDE !!!!

It pays to use the TEST facility. While EDITING a selector hit the BLUE TICK button and select a suitable volume or subvolume to try out your rule.

An approach to using Nested Selectors

So in sites with more complex Selector Requirements I use the following approach.

Create Selectors Named for the types of files, and ONLY specify items in the INCLUDE section
EG:-  5-Xco-Temp;    include tmp Temp etc etc
EG:-  5-Xco-Music;    include *.mp3, m:\Artists etc etc.

Then create a selector named for the Script or Function it refers to EG: 1- Xco-Servers. This Selector will only ever reference other selectors.
EG add EG 5-Xco-Server Temp to the EXCLUDE Section,  and add EG 5-Xco-Music to the INCLUDE Section

You can then create many selectors for various files, types etc, and simply add them to your Script Selectors.
Using a good naming convention results in a very readable list of Selectors which indicates quite intuitively what they are doing.
Preceding ALL your custom selectors with a standard character or two ensures they are grouped together in the list and not confused with the standard Retro Selectors. (I use “Digit-Company or Site abbreviation-Description” which allows me to order & group logically.
I have found it best to NOT change the default selectors, rather Duplicate and Rename them, thus you always have the originals to refer to as examples and use as templates.

21 Jul

Never under-estimate the bandwidth of a car load of tape

The original saying was ” Never under-estimate the bandwidth of a station-wagon full of tapes hurtling down the highway” and is attributed to legendary professor of computer science Andrew Tanenbaum. ( Computer Networks, 4th ed., p. 91)

https://en.wikipedia.org/wiki/Andrew_Tanenbaum

Back then tape capacities were measured in MegaBytes.  With the latest LTO technologies, never was this more true than in today’s world of big data.

You need to shift a load of data offsite to your Disaster Recovery site.

A box of 50 LTO 6 Tapes in your car for a 30 mile trip which takes 1 Hour

(includes loading time and travel in that city congestion!)

50 x 3TB = 150 TeraBytes per hour

That’s 150,000 GigaBytes in 3600 seconds = 40GBytes/S = ~330Gbits/S

You’ll have to go a long way, and with deep pockets to find that kind of WAN bandwidth!

 

20 Jul

Backup & Archiving – the difference

So what is the difference between Backup & Archiving? Don’t they both store your data for safe-keeping?

Well, yes but there are key differences  in how the data is collected, and these days, more importantly a clear distinction in the purpose for which the data is collected.

The How

Two words – Copy & Move

Backup takes a COPY of your data for safe-keeping.
Archiving MOVES your data to safe-keeping

From this simple difference flows the purposes for which each is used.

The Why

Backup for Disaster Recovery

Back in the day you could backup your entire data environment within your regular backup plan, and backup tended to serve both as disaster recovery and long term retention.

Today you run into all sorts of problems with this approach :-

  • You don’t have enough secondary or backup storage
  • It takes too long
  • It is difficult to search and retrieve
  • It is too expensive
  • It is too difficult to manage

The only real approach to addressing these issues is to reduce the amount of data you backup. Since you still have to protect all your data to some extent or another you must, therefore, classify the data in some way.

A small percentage of the data is the hot, dynamic, growing data which is actively being created, modified, and worked with.

This is the data that needs to be backed up.

Archiving for Retention

A large percentage of data is historical,  more static, and less frequently accessed. Reference material, completed projects, knowledge repositories, old data and the like.

This is data that can (should) be archived.