HOW TO SUPPRESS BEA-000337 ALERTING

I receive lots of emails with these errors reported as Critical either from Oracle Enterprise Manager Cloud Control 12c Release 4 (12.1.0.4) and from Oracle Enterprise Manager Cloud Control 13c Release 1 (13.1.0.0.0). I want to stop them to be generated.

HOW TO SUPPRESS BEA-101020 ALERTING

Host=dbpilot.net
Target type=Oracle WebLogic Server 
Target name=/EMGC_GCDomain/GCDomain/EMGC_OMS1 
Categories=Diagnostics, Fault 
Message=Incident (BEA-000337 [/em/websvcs/emws/ConsoleJobStepExecutorService]) detected in /orastage/oem13100/gc_inst/user_projects/domains/GCDomain/servers/EMGC_OMS1/adr/diag/ofm/GCDomain/EMGC_OMS1/alert/log.xml at time/line number: Tue Dec 20 10:10:23 2016/1622 
Severity=Critical 
Event reported time=Dec 20, 2016 10:13:14 AM MSK 
Customer Support Identifier=10000001
Operating System=Linux
Platform=x86_64
Associated Incident Id=13093 
Associated Incident Status=New 
Associated Incident Owner= 
Associated Incident Acknowledged By Owner=No 
Associated Incident Priority=None 
Associated Incident Escalation Level=0 
Event Type=Metric Alert 
Event name=alertLogAdrIncident:adr_problemKey 
Metric Group=Incident
Metric=Problem Key
Metric value=BEA-000337 [/em/websvcs/emws/ConsoleJobStepExecutorService]
Key Value=Tue Dec 20 10:10:23 2016/1622
Key Column 1=Timeline
Rule Name=Notifications All,rule 156 
Rule Owner=SYSMAN 
Update Details:
Incident (BEA-000337 [/em/websvcs/emws/ConsoleJobStepExecutorService]) detected in /orastage/oem13100/gc_inst/user_projects/domains/GCDomain/servers/EMGC_OMS1/adr/diag/ofm/GCDomain/EMGC_OMS1/alert/log.xml at time/line number: Tue Dec 20 10:10:23 2016/1622

The error details at log.xml on weblogic server

[oracle@dbpilot ~] tail /orastage/oem13100/gc_inst/user_projects/domains/GCDomain/servers/EMGC_OMS1/adr/diag/ofm/GCDomain/EMGC_OMS1/alert/log.xml
</msg>
<msg time='2016-12-20T10:10:23.377+03:00' org_id='oracle' comp_id='ofm'
 msg_id='1235543872' type='INCIDENT_ERROR' level='1'
 host_id='dbpilot.net' host_addr='10.10.10.10' prob_key='BEA-000337 [/em/websvcs/emws/ConsoleJobStepExecutorService]'
 upstream_comp='' downstream_comp='' ecid='3feb7623-3f11-4fdc-8654-90eda224f5c1-00000005'
 errid='153' detail_path='/orastage/oem13100/gc_inst/user_projects/domains/GCDomain/servers/EMGC_OMS1/adr/diag/ofm/GCDomain/EMGC_OMS1/incident/incdir_153'>
 <txt>Errors in directory: /orastage/oem13100/gc_inst/user_projects/domains/GCDomain/servers/EMGC_OMS1/adr/diag/ofm/GCDomain/EMGC_OMS1/incident/incdir_153  (incident=153):
stuck thread detected: [STUCK] ExecuteThread: &apos;17&apos; for queue: &apos;weblogic.kernel.Default (self-tuning)&apos;
 </txt>
</msg>

 
GATHER DETAILS

By querying the OMS database get details about the latest alert

SET LINES 300
SET PAGES 999
COL MESSAGE FOR A150

SELECT TO_DATE(COLLECTION_TIMESTAMP,'DD-MM-YYYY HH24:MI:SS') "DATE" ,MESSAGE "NUMBER" FROM MGMT_VIEW.MGMT$ALERT_NOTIF_LOG 
WHERE MESSAGE LIKE '%BEA-000337%'
GROUP BY TO_DATE(COLLECTION_TIMESTAMP,'DD-MM-YYYY HH24:MI:SS'),MESSAGE ORDER BY 1,2;

20-DEC-16
Incident (BEA-000337 [/em/websvcs/emws/ConsoleJobStepExecutorService]) detected in /orastage/oem13100/gc_inst/user_projects/domains/GCDomain/servers/EMGC_OMS1/adr/diag/ofm/GCDomain/EMGC_OMS1/alert/log.xml at time/line number: Tue Dec 20 09:10:23 2016/1613

20-DEC-16
Incident (BEA-000337 [/em/websvcs/emws/ConsoleJobStepExecutorService]) detected in /orastage/oem13100/gc_inst/user_projects/domains/GCDomain/servers/EMGC_OMS1/adr/diag/ofm/GCDomain/EMGC_OMS1/alert/log.xml at time/line number: Tue Dec 20 10:10:23 2016/1622

 
SOLUTION

Set StuckThread to false in Module-FMWDFW-2818.xml file and restart service (OMS, weblogic).

[oracle@dbpilot ~]  locate Module-FMWDFW-2818.xml
/orastage/oem13100/gc_inst/user_projects/domains/GCDomain/config/diagnostics/Module-FMWDFW-2818.xml

[oracle@dbpilot ~] cd /orastage/oem13100/gc_inst/user_projects/domains/GCDomain/config/diagnostics/
[oracle@dbpilot ~] cp Module-FMWDFW-2818.xml Module-FMWDFW-2818.xml_orig_21Dec2016

.. Before 
[oracle@dbpilot ~] egrep -i stuck Module-FMWDFW-2818.xml -A 4
      <name>StuckThread</name>
      <enabled>true</enabled>
      <rule-type>Log</rule-type>
      <rule-expression>(SEVERITY = 'Error') AND ((MSGID = 'WL-000337') OR (MSGID = 'BEA-000337'))</rule-expression>
      <alarm-type>AutomaticReset</alarm-type>

.. After 
[oracle@dbpilot ~] egrep -i stuck Module-FMWDFW-2818.xml -A 4
      <name>StuckThread</name>
      <enabled>false</enabled>
      <rule-type>Log</rule-type>
      <rule-expression>(SEVERITY = 'Error') AND ((MSGID = 'WL-000337') OR (MSGID = 'BEA-000337'))</rule-expression>
      <alarm-type>AutomaticReset</alarm-type>

After service restart no new BEA-000337 error will be spooled into log.xml anymore.