Thursday, August 22, 2019

Error Retry on Queue

The Batch framework allows retry of errors, is there a similar setting for Queue's for FAIL transactions?

In order to see the retry in the logs we need to enable Step #1

1.       Log onto WAS Console: Goto Troubleshooting-> Logs and Trace-> Server1-> Change Log Level Detail and add the bolded text.
*=info: com.dwl.*=warning: com.ibm.mdm.*=warning: com.ibm.mdm.server.config.*=info: com.ibm.mdm.common.brokers.*=info: com.ibm.mdm.mds.*=info: com.initiatesystems.*=info: com.dwl.base.report.mbean.TransactionDataListener=fine: com.dwl.jmsadapter.mdb.*=info
  
2.       SQL
UPDATE CONFIGELEMENT SET VALUE = ' SQLCODE=-911,SQLCODE=-913,ORA-00370,ORA-02049,ORA-00060,SQLCODE=-968,SQLCODE=-803, SQLCODE=-723'
, LAST_UPDATE_DT = CURRENT_TIMESTAMP WHERE NAME = '/IBM/DWLCommonServices/ErrorHandling/retryErrorStringList';

UPDATE CONFIGELEMENT SET VALUE = '12,2037,4930, 109,10506,49359',
LAST_UPDATE_DT = CURRENT_TIMESTAMP WHERE NAME = '/IBM/DWLCommonServices/ErrorHandling/retryErrorCodeList';


Once we set the error codes the retry should work automatically, do we have a setting to enable retries? This is a CM configuration, use sql to update it. 
If I add the log statement in WAS console then this retry information will be logged in trace.log? This should appear in SystemOut.log, not sure it will appear in trace.log. 

Reference:  

/IBM/MessagingAdapter/Exception/maxRetryLimit

·         Description: Determines how many times the Messaging Adapter attempts to reprocess a failed inbound request message by invoking DWLServiceController.
·         Default value3
·         Dynamic: true
If you want to see it in action, you will have to turn on INFO level logging for
com.dwl.jmsadapter.mdb.MDBRequestReceiverBean class

as the logging code is below

                               
                        

if(currentRetryCount <= getMaxRetryLimit(manager.getControl()) && manager.shouldRetry(e)) {
                                
                        

if (logger.isInfoEnabled()) {
                               
                        

logger.info(ResourceBundleHelper.resolve(
                               
                        

ResourceBundleNames.MESSAGING_ADAPTER_STRINGS,
                               
                        

INFO_DWLJMSMessageProcessor_MessageRetry,
                               
                        

new Object[] {manager.getMessageId(), currentRetryCount}));
                                
                        

}
                               
                        

processMessage(manager, currentRetryCount);
                               
                        

}
                       

Reprocessing inbound messages in case of an error
When there is an error processing an inbound request message, Messaging Adapter can re-invoke the DWLServiceController to reprocess the request. You can define the error conditions that trigger the Messaging Adapter to re-invoke the DWLServiceController using the following properties in the Configuration and Management component:
/IBM/DWLCommonServices/ErrorHandling/retryErrorStringList
/IBM/DWLCommonServices/ErrorHandling/retryErrorCodeList

Error Retry on Queue


Here is the SQL to run for retry:

UPDATE CONFIGELEMENT SET VALUE = ' SQLCODE=-911,SQLCODE=-913,ORA-00370,ORA-02049,ORA-00060,SQLCODE=-968,SQLCODE=-803, SQLCODE=-723'
, LAST_UPDATE_DT = CURRENT_TIMESTAMP WHERE NAME = '/IBM/DWLCommonServices/ErrorHandling/retryErrorStringList';

UPDATE CONFIGELEMENT SET VALUE = '12,2037,4930, 109,10506,49359',
LAST_UPDATE_DT = CURRENT_TIMESTAMP WHERE NAME = '/IBM/DWLCommonServices/ErrorHandling/retryErrorCodeList';

Summary of error response for defects which have been configured for retry:
1:
      <DWLError>
        <ComponentType>1012</ComponentType>
        <ErrorMessage>The following is required: PersonName</ErrorMessage>
        <ErrorType>FVERR</ErrorType>
        <LanguageCode>100</LanguageCode>
        <ReasonCode>2037</ReasonCode>
        <Severity>0</Severity>
        <Throwable>com.dwl.tcrm.exception.TCRMDataInvalidException: com.dwl.base.exception.DWLDataInvalidException: [1012 FVERR 2037 The following is required: PersonName]</Throwable>
      </DWLError>
2:
                        <DWLError>
                                <ComponentType>111</ComponentType>
                                <ErrorMessage>Duplicate primary key already exists.</ErrorMessage>
                                <ErrorType>DKERR</ErrorType>
                                <LanguageCode>100</LanguageCode>
                                <ReasonCode>12</ReasonCode>
                                <Severity>0</Severity>
                                <Throwable>com.dwl.tcrm.exception.TCRMDuplicateKeyException: [Exception_Common_DuplicateKey:] CDKCS0001E:The object cannot be created due to duplicate primary key. Primary key = null; object = com.dwl.tcrm.coreParty.component.TCRMSuspectBObj; exception = DB2 SQL Error: SQLCODE=-803, SQLSTATE=23505, SQLERRMC=3;RYY4.SUSPECT, DRIVER=3.72.52</Throwable>
                        </DWLError>
3:
<DWLError>
        <ComponentType>108</ComponentType>
        <ErrorMessage>BTM ITxRxException occurred in Request and Response Framework.</ErrorMessage>
        <ErrorType>READERR</ErrorType>
        <LanguageCode>100</LanguageCode>
        <ReasonCode>4930</ReasonCode>
        <Severity>0</Severity>
        <Throwable>com.dwl.base.exception.DWLBaseException</Throwable>
      </DWLError>
4:
<DWLError>
        <ComponentType>904</ComponentType>
        <ErrorMessage>The following transaction failed: updateLobRelationship</ErrorMessage>
        <ErrorType>UPDERR</ErrorType>
        <LanguageCode>100</LanguageCode>
        <ReasonCode>10506</ReasonCode>
        <Severity>0</Severity>
        <Throwable>com.ibm.pdq.runtime.exception.DataRuntimeException: [pdq][10103][3.200.75] An error prevented the update operation from completing successfully.;  Caused by: com.ibm.db2.jcc.am.SqlException: DB2 SQL Error: SQLCODE=-723, SQLSTATE=09000, SQLERRMC=RYY4.U_LEBREL;-803;23505;1|RYY4.H_LOBREL, DRIVER=3.72.52</Throwable>
      </DWLError>

5:
<DWLError>
                                                                <ComponentType>99</ComponentType>
                                                                <ErrorMessage>[Exception_Shared_LastUpdateDtNotMatched:] CDKCH2001E:Another transaction has updated the database record you are trying to update. Cannot update record at this time, please try again.</ErrorMessage>
                                                                <ErrorType>UPDERR</ErrorType>
                                                                <LanguageCode>100</LanguageCode>
                                                                <ReasonCode>109</ReasonCode>
                                                                <Severity>0</Severity>
                                                                <Throwable>java.lang.RuntimeException: com.dwl.base.exception.LastUpdateDateException: class com.dwl.tcrm.businessServices.entityObject.EObjLobRelationship | 879956570600676001</Throwable>
                                                </DWLError>
6:
<DWLError>
                                                                <ComponentType>4405</ComponentType>
                                                                <ErrorMessage>Suspect search adapter execution failed.</ErrorMessage>
                                                                <ErrorType>DIERR</ErrorType>
                                                                <LanguageCode>100</LanguageCode>
                                                                <ReasonCode>49359</ReasonCode>
                                                                <Severity>0</Severity>
                                                                <Throwable>java.lang.NullPointerException</Throwable>
                                                </DWLError>


Trace log Setup instructions


Please follow the steps below:
      1.  Log in to WAS console
      2.  Goto Troubleshooting -> Logs and Trace -> server1
3.  Set the file size to 20 MB and number of historical files to 20