cdr view

The cdr view command shows information about every Enterprise Replication server in the domain.

Syntax

ATS and RIS Directory Options

Notes:

¹ See Connect Option.


Element	Purpose	Restrictions
`time`	The number of seconds before the cdr view command is repeated.	Must be a positive integer.

The following table describes the cdr view subcommands.


Long Form	Meaning
apply	Show a summary of how data is being applied on each of the target servers, including the latency of each target server.
ats	Show a portion of each ATS file that is in text format.
atsdir	Show the names of the files in the ATS directory that are in text format and optionally run repair operations that are based on those files. If you are running this command as a DBSA, you must have read permission on the ATS files. Permissions on ATS files can be set with the chown operating system command.
ddr	Show the state, key log positions, and the proximity to transaction blocking for each server in the replication domain.
nif	Show information about the network connections between Enterprise Replication servers, including the number of transactions that are waiting to be transmitted to target servers.
profile	Show a summary of the state, data capture, data apply, errors, connectivity, queues, and the size of spooling files for every Enterprise Replication server.
rcv	Show information about the receive statistics for each target server, including the number of transaction failures and the rate at which transactions are applied.
ris	Show a portion of each RIS file that is in text format.
risdir	Show the names of the files in the RIS directory that are in text format and optionally run repair operations that are based on those files. If you are running this command as a DBSA, you must have read permission on the RIS files. Permissions on RIS files can be set with the chown operating system command.
sendq	Show information about the send queues for each Enterprise Replication server.
servers	Show information about the state, connection status to each peer server, and queue size for each Enterprise Replication server.
state	Show the Enterprise Replication state and the state of data capture, network connections, and data apply for each Enterprise Replication server.

The following table describes the cdr view options.


Long Form	Short Form	Meaning
--check	-C	Check the consistency between the database server and the ATS or RIS file. Send repair operations to stderr, but do not perform the repair operations.
--delete	-d	Delete ATS or RIS files after processing them with the repair operation.
--help	-h	Show the cdr view command usage.
logstage	-l	Show log staging statistics.
--quiet	-q	Quiet mode. Repair operations are not sent to stderr.
--repair	-R	Synchronize data based on ATS or RIS files in text format.
--repeat=	-r	Repeat the cdr view command after the number of seconds specified by the `time` element.
--verbose	-v	Verbose mode (default). All repair operations are sent to stderr.

Usage

Use the cdr view command to monitor the Enterprise Replication domain. Each subcommand results in different output information.

You can choose to show the output of multiple subcommands sequentially by including them in the same cdr view command. You can choose to automatically repeat the command by using the --repeat option to specify the seconds in between commands.

You can repair inconsistencies that are listed in ATS or RIS files on every server by using the --repair option. Use the --delete option to delete the ATS or RIS files after the repair is complete.

Tip: Using the --repair option is equivalent to running the cdr repair command. The --check option is equivalent to the cdr repair --check command.

The cdr view state Command Output

The following example of the output of the cdr view state command shows the state of Enterprise Replication and each of its main components for every server in the Enterprise Replication domain.

STATE
Source      ER               Capture          Network          Apply
            State            State            State            State
-----------------------------------------------------------------------------
cdr1        Active           Running          Running          Running
cdr2        Active           Running          Running          Running
cdr3        Active           Running          Running          Running
cdr4        Active           Running          Running          Running

In this example, Enterprise Replication is active and running normally on all servers.

Possible values in the ER State column include:

Abort: Enterprise Replication is aborting on this server.
Active: Enterprise Replication is running normally.
Down: Enterprise Replication is stopped.
Dropped: The attempt to drop the syscdr database failed.
Init Failed: The initial start of Enterprise Replication on this server failed, most likely because of a problem on the specified global catalog synchronization server.
Initializing: Enterprise Replication is being defined.
Initial Startup: Enterprise Replication is starting for the first time on this server.
Shutting Down: Enterprise Replication is stopping on this server.
Startup Blocked: Enterprise Replication cannot start because the server was started with the oninit -D command.
Synchronizing Catalogs: The server is receiving a copy of the syscdr database.
Uninitialized: The server does not have Enterprise Replication defined on it.

Possible values in the Capture State, Network State, and Apply State columns include:

Running: The Enterprise Replication component is running normally.
Down: The Enterprise Replication component is not running.
Uninitialized: The server is not a source server for replication.

The cdr view profile Command Output

The following example of the output of the cdr view profile command shows a summary of the other cdr view commands and information about the sbspaces that are designated for spooled transaction data.

ER PROFILE for Node cdr2                ER State Active
                                                                                
DDR - Running                           SPOOL DISK USAGE
  Current        4:16879616               Total                   100000
  Snoopy         4:16877344               Metadata Free             5025
  Replay         4:24                     Userdata Free            93193
  Pages from Log Lag State   43879
                                        RECVQ
SENDQ                                     Txn In Queue                 0
  Txn In Queue               0            Txn In Pending List          0
  Txn Spooled                0
  Acks Pending               0          APPLY - Running
                                          Txn Processed             1838
NETWORK - Running                         Commit Rate              76.58
  Currently connected to 3 out of 3       Avg. Active Apply         1.16
  Msg Sent                1841            Fail Rate                 0.00
  Msg Received            5710            Total Failures               0
  Throughput           1436.94            Avg Latency               0.00
  Pending Messages           0            Max Latency                  0
                                          ATS File Count               0
                                          RIS File Count               0

In this example, only the output for a single server, cdr2, is shown. The actual output of the cdr view profile command includes a similar profile for every server.

The DDR section is a summary of the cdr view ddr command.

The SPOOL DISK USAGE section shows the total amount of memory, in bytes, in the sbspaces that Enterprise Replication uses to store spooled transaction row data, and the amount of available metadata and user data space.

The SENDQ section is a summary of the cdr view sendq command.

The RECVQ section is a summary of the cdr view rcv command.

The NETWORK section is a summary of the cdr view nif command.

The APPLY section is a summary of the cdr view apply command.

The cdr view ddr Command Output

The following example of the output of the cdr view ddr command shows the status of log capture.

Server   Snoopy       Replay       Current      total        log pages to   LogLag   Cur LogLag
         log page     log page     log page     log pages    LogLag State   State    Action  
-------------------------------------------------------------------------------------------
g_bombay 16:133       16:0         16:134       30000        17866          Off       dlog
g_delhi  30:490       30:0         30:491       5000         3508           Off       logstage

The following example of the output of the cdr view ddr -l command shows the status of log capture.

Server   Disk Space   Max allowed  Max disk     Cur Staged  
         Usage(%)     Space(KB)    ever used(KB) log file cnt
-------------------------------------------------------------------------------------------
g_bombay 0.00         0            0.00         0 
g_delhi  0.00         1048576      0.00         0

The columns in the output of the cdr view ddr command provide the following information:

Server: The name of the Enterprise Replication server.
Snoopy log page: The current log ID and position at which transactions are being captured for replication.
Replay log page: The current log ID and position at which transactions have been applied. This is the position from which the log would must be replayed to recover Enterprise Replication if Enterprise Replication or the database server shut down.
Current® log page: The log page on which replicated transactions are being captured.
total log pages: The total number of log pages on the server.
log pages to LogLag State: The number of log pages that must be used before transaction blocking occurs.
LogLag State: The state of DDR log lag: on or off.
Cur LogLag Action: The action being taken to catch up logs.

For more information on interpreting this output, see onstat -g ddr: Print status of ER log reader.

The cdr view servers Command Output

The following example of the output of the cdr view servers command shows the state of the Enterprise Replication servers and their connections to each other.

SERVERS
Server Peer   ID   State    Status     Queue  Connection Changed
---------------------------------------------------------------------------
cdr1   cdr1   1    Active   Local      0
       cdr2   2    Active   Connected  0      Apr 15 10:46:16
       cdr3   3    Active   Connected  0      Apr 15 10:46:16
       cdr4   4    Active   Connected  0      Apr 15 10:46:15
cdr2   cdr1   1    Active   Connected  0      Apr 15 10:46:16
       cdr2   2    Active   Local      0
       cdr3   3    Active   Connected  0      Apr 15 10:46:16
       cdr4   4    Active   Connected  0      Apr 15 10:46:16
cdr3   cdr1   1    Active   Connected  0      Apr 15 10:46:16
       cdr2   2    Active   Connected  0      Apr 15 10:46:16
       cdr3   3    Active   Local      0
       cdr4   4    Active   Connected  0      Apr 15 10:46:16
cdr4   cdr1   1    Active   Connected  0      Apr 15 10:46:16
       cdr2   2    Active   Connected  0      Apr 15 10:46:16
       cdr3   3    Active   Connected  0      Apr 15 10:46:16
       cdr4   4    Active   Local      0

In this example, each of the four servers is connected to each other.

The output of this command is similar to the output of the cdr list server command, except that the cdr view server command shows all servers in the Enterprise Replication domain, not just the servers connected to the one from which the command is run. For information about the columns in this output, see cdr list server.

The cdr view sendq Command Output

The following example of the output of the cdr view sendq command shows information about the send queue for each server.

RQM SENDQ
Server    Trans.    Trans.    Trans.      Data    Memory      ACKS
          in que    in mem   spooled  in queue    in use   pending
---------------------------------------------------------------------------
cdr1         594       594         0     49896     49896         0
cdr2           0         0         0         0         0         0
cdr3           0         0         0         0         0         0
cdr4           0         0         0         0         0         0

In this example, only the server cdr1 has transactions in the send queue, all of which are in memory.

The columns of the cdr view sendq command provide the following information in addition to the server name:

Trans. in que: The number of transactions in the send queue.
Trans. in mem: The number of transactions in the send queue that are currently in memory.
Trans. spooled: The number of transactions in the send queue that have been spooled to disk.
Data in queue: The number of bytes of data in the send queue, including both in-memory and spooled transactions.
Memory in use: The number of bytes of data in the send queue that resides in memory.
ACKS pending: The number of acknowledgments that have been received but have not yet been processed.

The cdr view rcv Command Output

The following example of the output of the cdr view rcv command shows information about the receive queue for each server.

RCV
Server Received Spooled    Memory Pending Waiting
           Txn.    Txn.    In Use    Txn.    Txn.
---------------------------------------------------------------------------
cdr1          0       0         0       0       0
cdr2        372       0    871164     372       0
cdr3        220       0     18480     220       0
cdr4          0       0         0       0       0

In this example, the servers cdr2 and cdr3 have transactions in the receive queue, all of which have been preprocessed and are in the pending state waiting to be applied.

The columns of the cdr view rcv command provide the following information in addition to the server name:

Received Txn.: The number of transactions in the receive queue.
Spooled Txn.: The number of transactions in the receive queue that have been spooled to disk.
Memory In Use: The size, in bytes, of the receive queue.
Pending Txn.: The number of transactions that have been preprocessed but not yet applied.
Waiting Txn.: The number of acknowledgments waiting to be sent back to the source server.

The cdr view apply Command Output

The following example of the output of the cdr view apply command shows how replicated data is being applied.

APPLY
Server   Pl  Failure       Num     Num   Apply  --Latency--  ATS  RIS
       Rate    Ratio       Run  Failed    Rate  Max    Avg.    #    #
---------------------------------------------------------------------------
cdr1      0    0.000         0       0   0.000    0   0.000    0    0
cdr2      0    0.000     10001       0   0.112    0   0.000    0    0
cdr3      0    0.000     10001       0   0.112    0   0.000    0    0
cdr4      0    0.000     10001       0   0.112    0   0.000    0    0

In this example, the servers cdr2, cdr3, and cdr4 each applied 10 001 transactions.

The columns of the cdr view apply command provide the following information in addition to the server name:

Pl Rate: Indicates the degree of parallelism used when data is being applied. Zero indicates the highest possible rate of parallelism.
Failure Ratio: The ratio of the number of times data could not be applied in parallel because of deadlocks or lock time outs.
Num Run: The number of transactions processed.
Num Failed: The number of failed transactions because of deadlocks or lock time outs.
Apply Rate: The number of transactions that have been applied divided by the amount of time that replication has been active. The Apply Rate is equal to the Commit Rate in the cdr view profile command.
Max. Latency: The maximum number of seconds for processing any transaction.
Avg. Latency: The average number of seconds of the lifecycle of a replicated transaction.
ATS #: The number of ATS files.
RIS #: The number of RIS files.

The cdr view nif Command Output

The following example of the output of the cdr view nif command shows the status and statistics of connections between servers.

NIF
Source Peer   State         Messages  Messages Messages  Transmit
                                Sent  Received  Pending      Rate
---------------------------------------------------------------------------
cdr1   cdr2   Connected        24014       372        6 21371.648
       cdr3   Connected        24020        17        0 20527.105
       cdr4   Connected        24014        23        6 21925.727
cdr2   cdr1   Connected          392     24015        0 21380.879
       cdr3   Connected           14        14        0    10.857
       cdr4   Connected           14        14        0    11.227
cdr3   cdr1   Connected           17     24021        0 20310.611
       cdr2   Connected           14        14        0    10.739
       cdr4   Connected           14        14        0    11.227
cdr4   cdr1   Connected          236     24015        0 21784.225
       cdr2   Connected           14        14        0    11.101
       cdr3   Connected           14        14        0    11.101

In this example, all servers are connected to each other. The server cdr1 has six messages that have not yet been sent to server cdr2 and server cdr4.

The columns of the cdr view nif command provide the following information in addition to the source server name:

Peer

The name of the server to which the source server is connected.

State

The connection state. Values include:

Connected: The connection is active.
Disconnected: The connection was explicitly disconnected.
Timeout: The connection attempt has timed out, but will be reattempted.
Logic error: The connection disconnected due to an error during message transmission.
Start error: The connection disconnected due to an error while starting a thread to receive remote messages.
Admin close: Enterprise Replication was stopped by a user issuing the cdr stop command.
Connecting: The connection is being established.
Never Connected: The servers have never had an active connection.

Messages Sent

The number of messages sent from the source server to the target server.

Messages Received

The number of messages received by the source server from the target server.

Messages Pending

The number of messages that the source server must send to the target server.

Transmit Rate

The total bytes of messages sent and received by the server divided by the amount of time that Enterprise Replication has been running. Same as the Throughput field in the cdr view profile command.

The cdr view ats and cdr view ris Command Output

The following example of the output of the cdr view ats command shows that there are no ATS files in text format.

ATS for cdr1 - no files
---------------------------------------------------------------------------
 
ATS for cdr2 - no files
---------------------------------------------------------------------------
 
ATS for cdr3 - no files
---------------------------------------------------------------------------
 
ATS for cdr4 - no files

The following example of the cdr view ris command shows two RIS files in text format.

RIS for cdr1 - no files
---------------------------------------------------------------------------
 
RIS for cdr2 - 1 files
Source Txn. Commit         Receive
       Time                Time
---------------------------------------------------------------------------
cdr1   08-04-15 11:56:13 | 08-04-15 11:56:14
File:ris.cdr2.cdr1.D_4.080415_11:56:14.1
 
Row:2 / Replicate Id: 262146 / Table: stores_demo@user.customer / DbOp:Update
CDR:6 (Error: Update aborted, row does not exist in target table) / SQL:0 / ISAM:0
---------------------------------------------------------------------------
 
RIS for cdr3 - no files
---------------------------------------------------------------------------
 
RIS for cdr4 - 1 files
Source Txn. Commit         Receive
       Time                Time
---------------------------------------------------------------------------
cdr1   08-04-15 11:56:13 | 08-04-15 11:56:14
File:ris.cdr4.cdr1.D_1.080415_11:56:14.1
 
Row:3 / Replicate Id: 262146 / Table: stores_demo@user.customer / DbOp:Update
CDR:6 (Error: Update aborted, row does not exist in target table) / SQL:0 / ISAM:0

In this example, the servers cdr2 and cdr4 each have one RIS file.

The cdr view atsdir and cdr view risdir Command Output

The cdr view atsdir command and cdr view risdir command outputs have the same format. The following example of the output of the cdr view risdir command shows the names of two RIS files.

RISDIR
Server File                                  Size               Create
       Name                                                       Time
---------------------------------------------------------------------------
cdr2   ris.cdr2.cdr1.D_4.080415_11:56:14.1    465  2008-04-15 11:56:15
cdr4   ris.cdr4.cdr1.D_1.080415_11:56:14.1    475  2008-04-15 11:56:15

In this example, both server cdr2 and server cdr4 have a single RIS file. The Size column shows the size of the file, in bytes.

Examples

The following command would show information about the send queue and the network every 10 seconds:

cdr view sendq nif --repeat=10

The following command can be used in a daemon or script that runs every five minutes to check all servers for ATS and RIS files, repair inconsistencies, and delete the processed ATS and RIS files:

cdr view atsdir risdir --repair --delete --repeat=300