syncreplica

Exports or imports update packets

Applicability

Product Command type
MultiSite multitool subcommand
Platform
UNIX
Linux
Windows

Synopsis

  • Export an update packet from a replica that may not reside on the local host:
    sync/replica
    –export [ –maxsize max-packet-size [ –limit num-packets ] ] [ –comment comment | –cfile comment-file-pname | –cquery | –cqeach | –ncomment ] [ –compress ] [ –endrange ] [ –oprange replica-selector=[oplog-start][:oplog-end][,...]] { {–ship | –fship} [–sclass storage-class] [–pexpire date-time] [ –notify email-addr] | –out packet-file-pname } replica-selector ...
  • Import an update packet to a replica that may not reside on the local host (requires the syncmanager server to be running on the importing replica's host and for the importing replica to be in "managed" mode):
    sync/replica
    –import replica-selector [ –c comment | –cfile pname | –cq | –cqe | –nc ] { –receive [ –sclass storage-class ] | packet-file-pname ...}
  • sync/replica
    –import [ –c comment | –cfile pname | –cq | –cqe | –nc ] { –receive [ –sclass storage-class ] | { packet-file-pname | staging-area-pname } ...}
  • Identify missing packet dependencies:
    sync/replica
    –import –diagnose { –receive [ –sclass storage-class ] { packet-file-pname | staging-area-pname } ...}

Description

Synchronization of an existing replica with one or more sibling replicas is a three-phase process:

  1. At one site, a syncreplica –export command creates a logical update packet that contains changes that have occurred in the replica at that site (and perhaps other replicas, also).
  2. The logical packet is sent to one or more other sites.
  3. At another site, a syncreplica –import command applies the changes in the logical update packet to its replica of the same VOB. (The changes are applied at all sites that receive the logical packet.)

Contents of a logical update packet:

  • All changes that have occurred in the current replica since the last update generated for the destination replicas. (Changes already sent to the destination replicas are excluded from the packet).
  • Changes that have occurred in other replicas, which the current replica has received in previous update packets from those replicas, but have not already passed on to the destination replicas.

In all cases, syncreplica –export creates a single logical update packet for use at all the specified destinations. The packet can be used to update those particular replicas only.

Notes on the export phase

MultiSite is designed for efficient updating of replicas. syncreplica –export attempts to exclude operations that have been sent previously. (However, there is no harm in sending an operation multiple times to the same replica; the first operation is imported and subsequent identical operations are ignored.)

The replica is not locked during the export phase; in fact, the syncreplica –export command fails if the VOB is locked. Therefore, you must not schedule synchronizations during VOB backups (when the VOB must be locked). See also Retrying synchronization when the VOB is locked.

Specifying a directory for temporary files

syncreplica –export stores temporary files in the directory specified by the TMPDIR environment variable on Linux and the UNIX system and the TMP environment variable on Windows.

Notes on the import phase

A logical update packet is applied to the appropriate replica on the host on which you import it, unless you restrict processing with the –invob argument. syncreplica consults the VOB registry in the current region to determine the locations of these replicas' storage directories. Thus, you do not have to specify particular replicas or storage locations.

The import process applies logical update packets in the correct order. Therefore, you can specify packets in any order on the command line.

The VOB replica is not locked during the import phase. Synchronization fails if the VOB is locked. See also Retrying synchronization when the VOB is locked.

Specifying a directory for temporary files

syncreplica –import stores temporary files in the directory specified by the TMPDIR environment variable on Linux and the UNIX system and the TMP environment variable on Windows.

Skipping packets

syncreplica –import refuses to process an update packet in the following situations:

  • The update packet contains changes that depend on other changes that have not yet been imported to this replica. This situation typically means that an update packet destined for this replica has not been sent or was lost during transport.
  • Problems were encountered processing an earlier physical packet in a multiple-part logical packet.

In these cases, syncreplica –import displays an explanatory message.

Update failures / replaying packets

In some cases, syncreplica –import begins to apply operations to a replica, but fails with an error message. For example, another process may have locked the VOB, causing the import to fail. After the VOB is unlocked, you can run syncreplica –import to process the entire update packet again.

Not every failure to create packets is a problem. A true failure of syncreplica returns a code of 1. However, a packet may not be created when replicas appear to be synchronized. This is correct behavior and therefore returns a code of 0.

There is no harm in importing update packets that have already been processed successfully; the same change will not be made twice.

For more information about update failures, see "Troubleshooting MultiSite operations" in ClearCase MultiSite Administrator's Guidethis guide.

Deletion of update packets

If a single invocation of syncreplica –import applies a packet successfully to all target replicas on the host, the update packet is deleted when the command completes its work. If the packet is processed with multiple syncreplica –import –invob commands, it is not deleted.

Preservation of identities and permissions

If a VOB replica preserves identities and permissions, syncreplica –import maintains the consistency of identities and permissions information for elements mastered by the VOB family's identities- and permissions-preserving replicas. For each such element, an error occurs if the element's group is not on the group list of the importing replica (on Linux and the UNIX system) or is not the same as the group of the importing replica (on Windows).

If a VOB replica preserves permissions only, syncreplica –import maintains the consistency of permissions information for elements mastered by the VOB family's identities- and permissions-preserving replicas. Changes to identities for existing elements are ignored during import. New elements are assigned to the owner of the VOB at the current site, and the group of all new elements is the primary group of the VOB. (This is true even if the root user or a member of the ClearCase® administrators group imports the packet.)

If a VOB replica is nonpreserving, changes to identities and permissions of existing elements are ignored during import. New elements are assigned to the owner of the VOB at the current site, and the group of all new elements is the primary group of the VOB. (This is true even if the root user or a member of the ClearCase administrators group imports the packet.) Permissions set when the element is created are preserved, but subsequent permissions changes are ignored. Identities and permissions changes made at nonpreserving replicas are not propagated to other replicas.

Storage pools

Data containers from the update packets are placed in storage pools according to the standard element assignments. If the pool assignment for a new element cannot be determined, the element is assigned to the VOB's default source pool.

Trigger firing

ClearCase triggers do not fire in response to changes made during packet import.

Handling naming conflicts

syncreplica resolves naming conflicts among objects created at different replicas. For more information, read about conflict resolution in the "MultiSite operation" chapter of ClearCase MultiSite Administrator's Guidethis guide.

Delayed view updates

syncreplica does not inform any views (not even the view from which you enter the command) of the updates to replicas. All active views see updates within a few seconds, through their normal VOB-polling routines. You can force a view to recognize VOB updates by entering a cleartool setcs –current command.

Retrying synchronization when the VOB is locked

By default, synchronization exports and imports fail if the VOB is locked. To allow syncreplica to retry a synchronization when it encounters a lock, set the CLEARCASE_VOBLOCKWAIT environment variable to the amount of time (in minutes) for syncreplica to keep trying to write to the VOB.

During that time, syncreplica retries the write operation every minute. If the time elapses or CLEARCASE_VOBLOCKWAIT is set to zero, and the VOB is still locked, syncreplica exits with an error.

Note: syncreplica waits only if it detects the lock before it starts processing operations. If an administrator locks the VOB during processing, syncreplica exits with an error.

Restrictions

Identities: You must have one of the following identities:
  • VOB owner
  • root (Linux and the UNIX system)
  • Member of the ClearCase administrators group (Windows)

Locks: An error occurs if one or more of these objects are locked: VOB.

Mastership: No mastership restrictions.

Other: You must run syncreplica on the host where the VOB storage directory resides.

Options and arguments -- Export phase

Specifying the update packet size

Default

Each oplog together with its version data is treated as one physical packet.

If you do not specify –maxsize, the default packet size depends on the shipping option:

  • Packets created with –ship or –fship are no larger than the maximum packet size specified in the shipping.conf file (Linux and the UNIX system) or the MultiSite Control Panel (Windows).
  • Packets created with –out are no larger than 2097151 KB (2GB-1 KB).
–max/size max-packet-size [ –lim/it num-packets ]
The maximum size for a physical packet, expressed as a number followed by a single letter. For example:
500k
500 kilobytes
20m
20 megabytes
1.5g
1.5 gigabytes

The –limit option limits the number of files syncreplica generates; each file is no larger than max-packet-size. Use this option when the disk space for your storage bay or staging area is limited.

Event records and comments

Default
Creates one or more event records, with commenting controlled by the standard ClearCase user profile (default: –nc). See Event records and comments in the multitool reference page. To edit a comment, use cleartool chevent.
–c/omment comment | –cfi/le comment-file-pname | –cq/uery | –cqe/ach | –nc/omment
Overrides the default with the specified comment option.

Packet compression

Default
Packets are not compressed.
–compress
Compresses all export packets.

Compression does not affect packet naming. Use lspacket –long to display packet compression information.

Disposition of the update packet

Default
None. You must specify how the update packets created by syncreplica –export are to be stored or transmitted to other sites.

If you use –ship or –fship and omit the –sclass option, syncreplica places the packet in the storage bay location specified for the –default class in the shipping.conf file or the MultiSite Control Panel. By default, this location is /var/adm/rational/clearcase/shipping/ms_ship on Linux and the UNIX system and ccase-home-dir\var\shipping\ms_ship on Windows.

–shi/p –fsh/ip
Stores the update packet in one or more files in a store-and-forward storage bay; syncreplica creates a separate shipping order for each physical packet, indicating how and where it is to be delivered. The destinations are the host names associated in the VOB database with the replica-name arguments. (Replica-name/host-name associations are created with mkreplica –export and can be changed with chreplica.)

Using –f/ship (force ship) invokes the shipping server to send the update packet immediately. Using –shi/p does not invoke this server. To run shipping_server to send packets in storage bays, schedule sync_export_list –poll with the schedule command. (See the schedule reference page in the DevOps Code ClearCase Command Reference.)

–scl/ass class-name
Specifies the storage class of the packet and shipping order. syncreplica looks up the storage class in the shipping.conf (Linux and the UNIX system) or the MultiSite Control Panel (Windows) to determine the location of the storage bay to use.
Attention: Be sure to deliver a packet created with –out to its specified destinations promptly. If a replica has not yet received and applied this packet, it may not accept any subsequently generated packets from your replica until the first packet is received and processed.
–out packet-file-pname
The name of the first update packet. Additional physical packets, if any, are placed in files named packet-file-pname_2, packet-file-pname_3, and so on.

The update packets are not delivered automatically; use an appropriate method to deliver them.

You can create a packet using –out, and deliver it using the store-and-forward facility. See the mkorder reference page.

Identifying missing packet dependencies

Default
None.
–diagnose
Identify missing packet dependencies. If no missing oplogs are detected, no output is displayed. If missing oplogs are detected, then this option indicates the oplogs that need to be be re-exported. The output is in the format of a syncreplica –export –oprange command that is the suggested operation to be performed to re-export the oplogs:
Suggested Export Replica "<exporting-replica-selector>"
multitool syncreplica -export -endrange -oprange
<replica-name>=<oplog-id>:<oplog-id> <target-replica-name>

The suggested command specifies replica names unless a packet references a replica sibling that is unknown to the local importer, in which case the replica OID of the unknown sibling is displayed. Because replica OIDs are not valid input to the export operation, you need to use the cleartool describe command to obtain the replica names that correspond to the OIDs and substitute the replica OIDs with their names. You can then execute the suggested command, edited as appropriate, at the exporting host.

Treatment of redundant packets

Default
Attempts to replay each oplog in every packet, regardless of whether the packet contains no oplogs that are new to the importer (that is, whether or not the packets are redundant).
–endrange
Records in the packet header the last oplog ID enclosed for each sibling that is known to the exporter. When you specify this option (and if the version of multitool running at the importer supports this feature), multitool compares the oplog IDs in the packet header with the importer's own epoch row, and discards redundant packets without replaying them.
–oprange replica-selector=[oplog-start][:oplog-end][,...]]
Specifies the range of oplog packets to be replayed.

Handling packet-delivery failures

Default
If a packet cannot be delivered, it is sent through the store-and-forward facility back to the administrator at the site of the originating replica. A mail message is sent to the store-and-forward administrator. This occurs after repeated attempts to deliver the packet have failed, and the allotted time has expired; it can also occur when the destination host is unknown or a data file does not exist. The store-and-forward configuration settings specify the expiration period, the e-mail address of the administrator, and the notification program.
–pex/pire date-time
Specifies the time at which the store-and-forward facility stops attempting to deliver the packet and generates a failure mail message instead. This option overrides the expiration period specified for the storage class in the shipping.conf file (Linux and the UNIX system) or MultiSite Control Panel (Windows).

The date-time argument can have any of the following formats:

date.time | date | time | now
where:
date:
= day-of-week | long-date
time:
= h[h]:m[m][:s[s]] [UTC [ [ + | - ]h[h][:m[m] ] ] ]
day-of-week:
= today |yesterday |Sunday | ... |Saturday |Sun | ... |Sat
long-date:
= d[d]–month[–[yy]yy]
month:
= January |... |December |Jan |... |Dec

Specify the time in 24-hour format, relative to the local time zone. If you omit the time, the default value is 00:00:00. If you omit the date, the default value is today. If you omit the century, year, or a specific date, the most recent one is used. Specify UTC if you want the time to be resolved to the same moment in time regardless of time zone. Use the plus (+) or minus (-) operator to specify a positive or negative offset to the UTC time. If you specify UTC without hour or minute offsets, the default setting is Greenwich Mean Time (GMT). (Dates before January 1, 1970 Universal Coordinated Time (UTC) are not valid.)

Examples:
  • 22-November-2002
  • sunday
  • yesterday.16:00
  • 0
  • 8-jun
  • 13:00
  • today
  • 9-Aug.10:00UTC
–not/ify e-mail-address
The delivery-failure message is sent to the specified e-mail address.

If a failure occurs on a Windows host that does not have e-mail notification enabled, a message appears in the Windows Event Viewer. The message includes the e-mail-address value specified with this option and a note requesting that this user be informed of the status of the operation. For information about enabling e-mail notification, see the MultiSite Control Panel reference page.

Specifying the destination replicas

Default
None.
replica-selector ...
Specifies the replicas to which you want to send update packets. These replicas must be in the same VOB family. Specify replica-selector in the form [replica:]target-replica-name[@source-vob-selector]
target-replica-name
Name of the replica to which you want to send the packet (you can display replica names with lsreplica)
source-vob-selector
VOB family of the replica; can be omitted if the current working directory is within the VOB.

Specify vob-selector in the form [vob:]pname-in-vob

pname-in-vob
Pathname of the VOB tag (whether or not the VOB is mounted) or of any file system object within the VOB (if the VOB is mounted)

Examples

Exports

  • Generate an update packet to be sent to replica boston_hub. Store the packet in a file in directory c:\tmp.


    multitool syncreplica –export –out c:\tmp\boston_hub_packet1
    boston_hub@\dev




    Generating synchronization packet c:\tmp\boston_hub_packet1

  • Similar to preceding example, but place the packet file in a storage bay, for shipping at some later time by the store-and-forward facility.


    multitool syncreplica –export –ship boston_hub@\dev




    Generating synchronization packet c:\Program Files\DevOps\Code\ClearCase\var
    \shipping\ms_ship\outgoing\sync_bangalore_19-May-02.09.33.02_3447_1
    - shipping order file is c:\Program Files\DevOps\Code\ClearCase\var\
    shipping\ms_ship\outgoing\sh_o_sync_bangalore_19-May-02.09.33.02_3447_1

  • Similar to preceding example, but ship the packet immediately.


    multitool syncreplica –export –fship boston_hub@\dev




    Generating synchronization packet c:\Program Files\DevOps\Code\ClearCase
    \var\shipping\ms_ship\outgoing\sync_bangalore_19-May-02.09.33.
    02_3447_1 - shipping order file is c:\Program Files\DevOps\Code\ClearCase
    \var\shipping\ms_ship\outgoing\sh_o_sync_bangalore_19-May-
    02.09.33.02_3447_1
    Attempting to forward/deliver generated packets...
    -- Forwarded/delivered packet c:\Program Files\DevOps\Code\ClearCase\var
    \shipping\ms_ship\sync_bangalore_19-May-02.09.33.02_3447_1

  • Send a sync update to replica boston_hub and compress the packet


    multitool syncreplica –export –fship –compress boston_hub@\dev



    Generating synchronization packet c:\Program Files\DevOps\Code\ClearCase\var\shipping\ms_ship\outgoing\
    sync_bangalore_19-May-02.09.33.02_3447_1 ...
    Shipping order c:\Program Files\DevOps\Code\ClearCase\var\shipping\ms_ship\outgoing\sh_o_
    sync_bangalore_19-May-02.09.33.02_3447_1
    Attempting to forward/deliver generated packets ...
    -- Forwarded/delivered packet c:\Program Files\DevOps\Code\ClearCase\var\shipping\ms_ship\outgoing\
    sync_bangalore_19-May-02.09.33.02_3447_1


Imports

  • Process an incoming update packet in directory /usr/tmp.


    multitool syncreplica –import /usr/tmp/boston_hub_packet1




    Applied sync. packet /usr/tmp/boston_hub_packet1 to VOB /net
    /minuteman/vobstg/dev.vbs

Diagnosing and correcting missing packet dependencies

  • Diagnose all packets in the shipping bay to determine if there is a gap in the oplogs that causes the host to fail to import packets. In this example, oplog 300 of replica tokyo_hub is missing at the importing host.


    multitool syncreplica -import -diagnose -receive

    Suggested Export Replica "tokyo_hub@/vobs/dev" multitool syncreplica  -export -endrange -oprange tokyo_hub=300:300 boston_hub

  • Create an update packet to be sent to replica boston_hub, which includes replica sanfran_hub's oplogs from 151 to 155.

    multitool syncreplica -export -endrange -ship -oprange sanfran_hub=151:155 boston_hub@/vobs/dev

  • Create an update packet to be sent to replica boston_hub, which includes replica sanfran_hub's oplogs from 151 to 155, and tokyo_hub's oplogs from 301 to 310.

    multitool syncreplica -export -endrange -ship -oprange sanfran_hub=151:155,tokyo_hub=301:310 boston_hub@/vobs/dev