Data Domain uses an application called "Enterprise Manager" to administer, monitor and configure the DD appliance. Several appliances can be added to a single Enterprise Manager and administered as one system. The Data Domain Management Framework contains the CLI that can be accessed via SSH or at the console.
DD Management Framework contains the CLI, alerts and monitoring either through SNMP or email, RBAC and autosupport, which is EMC-provided 24/7. The DD also uses a Lights Out management, which gives the user the ability to power cycle and connect to the DD appliance whether or not the unit is running. DDOS 5.0 or later has this feature, which includes power management as well as serial over Ethernet (or serial over LAN).
Working my way toward being a Full Stack Engineer. I work for a state University leading the Systems Team, looking to chart a future of the data center and how it looks to build into the cloud in a responsible and innovative manner. This blog is largely a place to stash things I pick up in daily work life and pursuit of knowledge.
Friday, November 1, 2013
Data Domain System Management
Data Domain Extended Retention
Data Domain Extended Retention is the new name for Data Domain Archiver. It is a licensed software feature which supports VTL and MTree, as well as CIFS and NFS data. It also utilizes "Fast Seeding" which improves data migration from the active to retention tiers of storage, and "packing" which reclaims duplicate segments in the retention tier. Deduplication takes place only within a specific tier, not across active and retention tiers.
Extended Retention is supported only on the DD860 and DD990 and with the ES20 or ES30 expansion cabinets.
The benefits of ER are that data can be written to the active tier very quickly and then moved off to slower retention tiers as time permits. The active tier will typically hold data for 90 days or less, with the retention tiers holding data as long as policy requires. There is also a high data ingress rate (up to 31TB/hour), and the cost per GB goes down drastically as the system scales with more expansion shelves.
The retention tier is made up of several retention units. Each unit can span multiple shelves if configured to do so, or can be portions of shelves. Data moves out of the active tier into the first retention unit until that unit is completely full. Once full, the unit is "sealed" and the second unit is started until it is full, etc. This way, oldest data is in one retention unit, followed by the next and the next. Deduplication takes place only in each unit to reduce the likelihood of losing multiple units should there be a failure. This is the Fault Isolation of Data Domain, as well.
All data on a DD Extended Retention system appears active. Tiering is transparent. To accommodate scalability, not all retention units are readily available in memory. Oldest data will be de-prioritized and put on standby but is still more accessible than tape. Data can also be replicated from one DD ER unit to another using collection replication, which replicates the entire box with each tier in place.
Extended Retention is supported only on the DD860 and DD990 and with the ES20 or ES30 expansion cabinets.
The benefits of ER are that data can be written to the active tier very quickly and then moved off to slower retention tiers as time permits. The active tier will typically hold data for 90 days or less, with the retention tiers holding data as long as policy requires. There is also a high data ingress rate (up to 31TB/hour), and the cost per GB goes down drastically as the system scales with more expansion shelves.
The retention tier is made up of several retention units. Each unit can span multiple shelves if configured to do so, or can be portions of shelves. Data moves out of the active tier into the first retention unit until that unit is completely full. Once full, the unit is "sealed" and the second unit is started until it is full, etc. This way, oldest data is in one retention unit, followed by the next and the next. Deduplication takes place only in each unit to reduce the likelihood of losing multiple units should there be a failure. This is the Fault Isolation of Data Domain, as well.
All data on a DD Extended Retention system appears active. Tiering is transparent. To accommodate scalability, not all retention units are readily available in memory. Oldest data will be de-prioritized and put on standby but is still more accessible than tape. Data can also be replicated from one DD ER unit to another using collection replication, which replicates the entire box with each tier in place.
Data Domain Systems Overview
As model of DD increases, so do ingest rate and capacity.
Logical capacity is the amount of data, based on standard set of calculations that can be backed up on a DD appliance, while usable capacity is the actual disk space available.
DD hardware follows N+1 philosophy, where hot-swappable components such as fans, power supplies, drives have an available spare in the event of a failure. Currently drive sizes are 1 or 2 TB, 7200 RPM SATA II.
The ES20 expansion shelf has 4 SAS connectors, or 2 for each module. 1 connector goes to the host and the other to another expansion shelf. The ES30 is the same as the ES20, but consumes 45% less power. For either expansion shelf, a "shelf capacity" license and DD OS 5.1 or newer is required.
The DD16 and DD640 do not support expansion shelves, and are internal storage capacity only. The 640 and 670 have internal storage while supporting optional external storage, and the 860, 890 and 99 are all external-only storage.
Logical capacity is the amount of data, based on standard set of calculations that can be backed up on a DD appliance, while usable capacity is the actual disk space available.
DD hardware follows N+1 philosophy, where hot-swappable components such as fans, power supplies, drives have an available spare in the event of a failure. Currently drive sizes are 1 or 2 TB, 7200 RPM SATA II.
The ES20 expansion shelf has 4 SAS connectors, or 2 for each module. 1 connector goes to the host and the other to another expansion shelf. The ES30 is the same as the ES20, but consumes 45% less power. For either expansion shelf, a "shelf capacity" license and DD OS 5.1 or newer is required.
The DD16 and DD640 do not support expansion shelves, and are internal storage capacity only. The 640 and 670 have internal storage while supporting optional external storage, and the 860, 890 and 99 are all external-only storage.
Data Domain Replication
Data Domain replication involves copying data from one DD to another. A replication pair is called a "context" in the DD GUI. The primary purpose of DD replication is disaster recovery, although archiving data for longer periods than is desired to have onsite is also a function.
There are 4 types of replication in Data Domain:
Collection replication is a full-system mirror of the col1 directory. The destination is read-only, and can back up one source only. Collection replication backups up snapshots as well as users and permissions, and is the fastest and lightest replication method of replicating one DD to another.
Directory replication is more flexible in its ability to configure replicated data from one DD to point to multiple destinations and selecting just the data to be replicated. CIFS and NFS shares can be sent to the same DD destination, but not within the same directory. When directory replication is configured, the destination folder will be automatically created if it does not exist, and after created ownership and permissions on files and folders is maintained. The directory replication process is triggered when a file closes, but to conserve high-demand bandwidth can be forced to a specific time. Snapshots are not replicated automatically and must be configured separately.
MTree replication is a new feature with the introduction of MTrees. In this method, the destination MTree is set read only, and the main difference is in how data is determined for transfer. MTree replication still sends changed data only, but instead of sending vector summaries back and forth, MTree replication creates a snapshot and compares one to another. MTree replication can use encryption and Retention Lock as well.
Pool replication is exactly like directory replication, but the source is VTL data. A separate VTL license is not needed onthe destination DD if VTL restores will not be done directly from it.
DD systems can replicate in the following topologies:
There are 4 types of replication in Data Domain:
Collection replication is a full-system mirror of the col1 directory. The destination is read-only, and can back up one source only. Collection replication backups up snapshots as well as users and permissions, and is the fastest and lightest replication method of replicating one DD to another.
Directory replication is more flexible in its ability to configure replicated data from one DD to point to multiple destinations and selecting just the data to be replicated. CIFS and NFS shares can be sent to the same DD destination, but not within the same directory. When directory replication is configured, the destination folder will be automatically created if it does not exist, and after created ownership and permissions on files and folders is maintained. The directory replication process is triggered when a file closes, but to conserve high-demand bandwidth can be forced to a specific time. Snapshots are not replicated automatically and must be configured separately.
MTree replication is a new feature with the introduction of MTrees. In this method, the destination MTree is set read only, and the main difference is in how data is determined for transfer. MTree replication still sends changed data only, but instead of sending vector summaries back and forth, MTree replication creates a snapshot and compares one to another. MTree replication can use encryption and Retention Lock as well.
Pool replication is exactly like directory replication, but the source is VTL data. A separate VTL license is not needed onthe destination DD if VTL restores will not be done directly from it.
DD systems can replicate in the following topologies:
- One to one
- Many to one
- Cascading
- Bi-directional
- ONe to many
- Cascading one to many
Data Domain File System and Data Management
The Data Domain file system starts with the ddvar directory. In NFS it can be shared as /ddvar and in CIFS it can be shared as \ddvar. You can't rename or delete it because it holds the operating environment binaries and configuration.
The data directory is mounted at /data/col1/. "Col1" is short for "collection #1" and there is only one collection at this point. Inside /col1 you will find folders, or "MTrees." The default is "backup," and up until recently all backup data was stored there. With later releases, however, you can create and administer more MTrees under /col1.
Each MTree can be individually managed with its own policies and permissions. The max number of MTrees is 100, however after 14 there is a performance degradation in environments where there is a lot of read/write streams to each MTree. Best practice dictates that you keep MTrees to 14 and to aggregate operations to one MTree where possible. You can mount MTrees like any other CIFS/NFS share, however it is not recommended to mix protocol type. VTL and DDBoost clients create their own MTree, as well.
MTrees are configurable with quotas. Hard quotas stop any data from being written, where soft quotas send warnings and alerts. These quotas can be used in charge-back scenarios, and administrators can be alerted when thresholds are reached to ease administration.
Data Domain Snapshots create a full copy of the data at a point in time (PIT). A DD snapshot copies the pointers from the production data, and then marks new data with a new pointer leaving the old in place for the snapshot. Snapshots are kept in /backup/.snapshot. The max number of snapshots is 750, with warnings at 90% (from 675-749) and alerts at 750.
If the DD is a collection source, snapshots are replicated, but if the DD is a directory replication source, snapshots are not replicated. One other benefit of using snapshots is that there is no additional license required.
Fastcopy is a feature often used to revert from a snapshot. It copies the pointers in full, but where a snap is read-only, a Fastcopy copy is read/write.
Data Sanitization is essentially electronic shredding of data. It is a filesys clean operation, including overwriting free space, metadata and reference data. The sanitization deletes unique pieces of the file only, but if blocks are being used by other files in the deduplicated data, those pieces are left in tact to maintain the integrity of the other files. This feature is often required by DoD/NIST compliance organizations. Sanitization can only be accessed through the command-line, and it is licensed along with the Retention Lock license.
Filesystem Cleaning is a process that runs by default every week on Tuesday at 6:00 AM. When data is deleted, it remains on the disk until the retention lock period expires ur until the cleaning process runs. Space on the DD is not recovered until the filesystem is cleaned. Cleaning self-throttles to 50% CPU and is configurable by an administrator.
To clean the filesystem, data is copied forward to a free container, after which unclaimed data is deleted and finally the original container is deleted.
Monitoring Filesystem Usage depends largely on the rate of change within your data set. If your data changes often and your retention period is long your data on the DD will grow rapidly as well. The size and number of data sets will affect the growth of data on the DD, as well does "compressibility." Essentially, data that is already compressed or deduplicated will not compress well, so your DD data set will actually grow. Modifying the retention period is one way to alleviate this problem. Data can be graphed from 7-120 days to monitor and maintain utilization.
The data directory is mounted at /data/col1/. "Col1" is short for "collection #1" and there is only one collection at this point. Inside /col1 you will find folders, or "MTrees." The default is "backup," and up until recently all backup data was stored there. With later releases, however, you can create and administer more MTrees under /col1.
Each MTree can be individually managed with its own policies and permissions. The max number of MTrees is 100, however after 14 there is a performance degradation in environments where there is a lot of read/write streams to each MTree. Best practice dictates that you keep MTrees to 14 and to aggregate operations to one MTree where possible. You can mount MTrees like any other CIFS/NFS share, however it is not recommended to mix protocol type. VTL and DDBoost clients create their own MTree, as well.
MTrees are configurable with quotas. Hard quotas stop any data from being written, where soft quotas send warnings and alerts. These quotas can be used in charge-back scenarios, and administrators can be alerted when thresholds are reached to ease administration.
Data Domain Snapshots create a full copy of the data at a point in time (PIT). A DD snapshot copies the pointers from the production data, and then marks new data with a new pointer leaving the old in place for the snapshot. Snapshots are kept in /backup/.snapshot. The max number of snapshots is 750, with warnings at 90% (from 675-749) and alerts at 750.
If the DD is a collection source, snapshots are replicated, but if the DD is a directory replication source, snapshots are not replicated. One other benefit of using snapshots is that there is no additional license required.
Fastcopy is a feature often used to revert from a snapshot. It copies the pointers in full, but where a snap is read-only, a Fastcopy copy is read/write.
Data Sanitization is essentially electronic shredding of data. It is a filesys clean operation, including overwriting free space, metadata and reference data. The sanitization deletes unique pieces of the file only, but if blocks are being used by other files in the deduplicated data, those pieces are left in tact to maintain the integrity of the other files. This feature is often required by DoD/NIST compliance organizations. Sanitization can only be accessed through the command-line, and it is licensed along with the Retention Lock license.
Filesystem Cleaning is a process that runs by default every week on Tuesday at 6:00 AM. When data is deleted, it remains on the disk until the retention lock period expires ur until the cleaning process runs. Space on the DD is not recovered until the filesystem is cleaned. Cleaning self-throttles to 50% CPU and is configurable by an administrator.
To clean the filesystem, data is copied forward to a free container, after which unclaimed data is deleted and finally the original container is deleted.
Monitoring Filesystem Usage depends largely on the rate of change within your data set. If your data changes often and your retention period is long your data on the DD will grow rapidly as well. The size and number of data sets will affect the growth of data on the DD, as well does "compressibility." Essentially, data that is already compressed or deduplicated will not compress well, so your DD data set will actually grow. Modifying the retention period is one way to alleviate this problem. Data can be graphed from 7-120 days to monitor and maintain utilization.
Thursday, October 31, 2013
Data Domain Protocols and Data Paths
Data Domain uses standard protocols to simplify administration and integration into existing networks. It can connect to systems either by fiber channel or Ethernet.
When using Fiber Channel, the DD uses Virtual Tape Library (VTL) to communicate with the backup server. When using Ethernet, the backup server will mount the DD like any other disk share using CIFS or NFS. DD can also use DDBoost, which requires software agent be installed on the client, and can also use NDMP which essentially is VTL over Ethernet instead of FC.
DD Replicator is a licensed feature, and when used on two DD appliances, replication and backup can take place simultaneously. VTL is also a licensed feature.
When using Fiber Channel, the DD uses Virtual Tape Library (VTL) to communicate with the backup server. When using Ethernet, the backup server will mount the DD like any other disk share using CIFS or NFS. DD can also use DDBoost, which requires software agent be installed on the client, and can also use NDMP which essentially is VTL over Ethernet instead of FC.
DD Replicator is a licensed feature, and when used on two DD appliances, replication and backup can take place simultaneously. VTL is also a licensed feature.
Data Domain Core Technologies
Data Domain uses two core technologies. SISL and DIA.
SISL is Stream Informed Segment Layout. This is the target-based inline deduplication process that focuses on speed instead of 100% accuracy, although SISL does provide 99% accuracy in limiting duplicated data segments. There is minimal disk access because the process takes place in RAM, and a faster CPU provides increases in SISL efficiency. You dn't spend any time waiting on spindle speed.
The process is such:
SISL is Stream Informed Segment Layout. This is the target-based inline deduplication process that focuses on speed instead of 100% accuracy, although SISL does provide 99% accuracy in limiting duplicated data segments. There is minimal disk access because the process takes place in RAM, and a faster CPU provides increases in SISL efficiency. You dn't spend any time waiting on spindle speed.
The process is such:
- Streams data into RAM
- Stream is segmented - into 4-12 KB chunks
- Fingerprints are created from the segments
- Verify if segments are unique using two functions
- Summary vector - a list of segments from disk that are predictivly selected
- Segment locality - because data follows predictable patterns, DD selects segments for the summary vector that are most likely to be used based on the fingerprint list in RAM
- Store unique segments
- End-to-end Verification - after writing new segments to dis, DD OS performs a read of the data to ensure it can reassemble the file, computes and verifies the checksum
- Fault Avoidance and Containment
- New data never overwrites existing data. Regenerating data is fast because data is never missing due to overwrite. Deleted data is removed during disk cleanup.
- DD uses NVRAM to buffer all data not written to disk
- Fault Detection and Healing - DD OS uses the logging file system and RAID6 disks to verify the integrity of the data on every read. It computes and verifies the checksum each time data is accessed
- File System Recovery - because data is never overwritten, there are no block maps or reference counts required. DD OS merely needs to find the head of the log file and it can rebuild the filesystem. Once it knows where the log file is, it scans the log and rebuilds data using RAID6 where necessary
Subscribe to:
Posts (Atom)