Print Friendly, PDF & Email

 
REQUEST

I receive lots of emails with these errors reported as Critical from Oracle Enterprise Manager Cloud Control 13c Release 1 (13.1.0.0.0). I want to stop them to be generated.

 
SYMPTOMS

Host=dbpilot.net 
Target type=Oracle Management Service
Target name=dbpilot.net:4889_Management_Service 
Categories=Diagnostics, Error 
Message=Incident (ADFC-00032 [ADFc]) detected in /orastage/oem13100/gc_inst/user_projects/domains/GCDomain/servers/EMGC_OMS1/adr/diag/ofm/EMGC_DOMAIN/EMOMS/alert/log.xml at time/line number: Tue Dec 26 13:07:22 2017/11012.
Severity=Critical 
Event reported time=Dec 26, 2017 1:07:43 PM MSK 
Customer Support Identifier=10000001
Operating System=Linux
Platform=x86_64
Associated Incident Id=45967 
Associated Incident Status=New 
Associated Incident Owner= 
Associated Incident Acknowledged By Owner=No 
Associated Incident Priority=None 
Associated Incident Escalation Level=0 
Event Type=Metric Alert 
Event name=adrAlertLogIncidentError:genericIncidentErrStack 
Metric Group=Incident
Metric=Diagnostic Incident
Metric value=Errors in directory: /orastage/oem13100/gc_inst/user_projects/domains/GCDomain/servers/EMGC_OMS1/adr/diag/ofm/EMGC_DOMAIN/EMOMS/incident/incdir_1191 (incident=1191):~ADF Controller encountered a serious error.
Key Value=Tue Dec 26 13:07:22 2017/11012
Key Column 1=Time/Line Number
Rule Name=Notifications All,rule 156 
Rule Owner=SYSMAN 
Update Details:
Incident (ADFC-00032 [ADFc]) detected in /orastage/oem13100/gc_inst/user_projects/domains/GCDomain/servers/EMGC_OMS1/adr/diag/ofm/EMGC_DOMAIN/EMOMS/alert/log.xml at time/line number: Tue Dec 26 13:07:22 2017/11012.

The error details at log.xml on OMS server

[oracle@dbpilot ~] cd /orastage/oem13100/gc_inst/user_projects/domains/GCDomain/servers/EMGC_OMS1/adr/diag/ofm/EMGC_DOMAIN/EMOMS/alert
[oracle@dbpilot ~] tail log.xml
</msg>
<msg time='2017-12-26T13:07:22.187+03:00' org_id='oracle' comp_id='ofm'
 msg_id='1809474518' type='INCIDENT_ERROR' level='1'
 host_id='dbpilot.net' host_addr='10.10.10.10' prob_key='ADFC-00032 [ADFc]'
 upstream_comp='' downstream_comp='' ecid='005OJhmavbmFg485zj1FiW0009Nm0006cv'
 errid='1191' detail_path='/orastage/oem13100/gc_inst/user_projects/domains/GCDomain/servers/EMGC_OMS1/adr/diag/ofm/EMGC_DOMAIN/EMOMS/incident/incdir_1191'>
 <txt>Errors in directory: /orastage/oem13100/gc_inst/user_projects/domains/GCDomain/servers/EMGC_OMS1/adr/diag/ofm/EMGC_DOMAIN/EMOMS/incident/incdir_1191  (incident=1191):
ADF Controller encountered a serious error.
 </txt>
</msg>

A content of readme file in incdir_1191 directory

[oracle@dbpilot ~] cat readme.txt
Incident Id: 1191
Incident Source: SYSTEM
Create Time: Tue Dec 26 13:07:22 MSK 2017
Problem Key: ADFC-00032 [ADFc]
ECID: 005OJhmavbmFg485zj1FiW0009Nm0006cv
Thread Id: 1775376
Application Name: emgc
User Name: SYSMAN
Error Message Id: ADFC-00032

Context Values
--------------
ApplicationName : emgc
SubjectUsername : SYSMAN
DSID : 0000M2IIQMPFg485zj1FiW1QAF^c0001Mr
ServletUsername : SYSMAN
DFW_SERVER_NAME : EMGC_OMS1
DFW_DOMAIN_NAME : GCDomain
ServletURI : /em/faces/core-jobs-results-executionDetails
DFW_USER_NAME : SYSMAN
DFW_APP_NAME : emgc


HTTP Request
------------
StartTime: 2017-12-26 13:07:17.327
User: SYSMANECID: 005OJhmavbmFg485zj1FiW0009Nm0006cv,0:1
Context Values:
   ApplicationName: emgc
   SubjectUsername: SYSMAN
   ServletUsername: SYSMAN
   DSID: 0000M2IIQMPFg485zj1FiW1QAF^c0001Mr
   ServletURI: /em/faces/core-jobs-results-executionDetails

URI: /em/core/jobs/results/executionDetails.jspx
RequestType: GET
QueryString: Adf-Rich-Message=true&unique=1514282804577&oracle.adf.view.rich.STREAM=emT:r1:exec_table:pc1:exectt,emT:r1:0:r1:0:opTbl&Adf-Page-Id=3&javax.faces.ViewState=!-197klq3ezk&Adf-Window-Id=w0
All Headers Names:
  Host, User-Agent, Accept, Accept-Language, Accept-Encoding, Referer, Cookie, ECID-Context, Connection, X-Forwarded-For, X-WebLogic-KeepAliveSecs, X-WebLogic-Force-JVMID
Selected Header Values:
  ECID-Context : 1.005OJhmavbmFg485zj1FiW0009Nm0006cv;kXjE
  Accept-Language : ru-RU,ru;q=0.8,en-US;q=0.5,en;q=0.3
  Host : dbpilot.net:7802
  Referer : https://dbpilot.net:7802/em/faces/core-jobs-results-executionDetails?execId=612C3DBC81E583C8E0531E01040A863E&_afrRedirect=3733355492352977
  Accept-Encoding : gzip, deflate
  User-Agent : Mozilla/5.0 (Windows NT 6.1; WOW64; rv:30.0) Gecko/20100101 Firefox/30.0
  Connection : Keep-Alive
  Accept : text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8

Stack Trace
-----------
java.lang.Throwable: dummy exception to generate stacktrace leading up to incident
        at oracle.dfw.impl.incident.DiagnosticsDataExtractorImpl.createIncident(DiagnosticsDataExtractorImpl.java:287)
        at oracle.adfinternal.controller.util.LogUtils.reportIncident(LogUtils.java:844)
        at oracle.adfinternal.controller.util.LogUtils.reportIncident(LogUtils.java:793)
        at oracle.adf.controller.internal.binding.TaskFlowRegionModelViewPort.doProcessBeginRegion(TaskFlowRegionModelViewPort.java:237)
        at oracle.adf.controller.internal.binding.TaskFlowRegionModelViewPort.processBeginRegion(TaskFlowRegionModelViewPort.java:163)
        at oracle.adf.controller.internal.binding.TaskFlowRegionModelLocal.processBeginRegion(TaskFlowRegionModelLocal.java:12)
        at oracle.adf.view.rich.component.fragment.UIXRegion$RegionContextChange.doChangeImpl(UIXRegion.java:1436)
        at oracle.adf.view.rich.context.DoableContextChange.doChange(DoableContextChange.java:91)
        at oracle.adf.view.rich.component.fragment.UIXRegion._beginInterruptibleRegion(UIXRegion.java:777)
        at oracle.adf.view.rich.component.fragment.UIXRegion.processRegion(UIXRegion.java:520)
        at oracle.adfinternal.view.faces.taglib.region.AbstractRegionTag.doStartTag(AbstractRegionTag.java:70)
        at oracle.adfinternal.view.faces.taglib.region.RegionTag.doStartTag(RegionTag.java:92)
        at oracle.jsp.runtime.tree.OracleJspBodyTagNode.executeHandler(OracleJspBodyTagNode.java:50)
        at oracle.jsp.runtime.tree.OracleJspCustomTagNode.execute(OracleJspCustomTagNode.java:263)
        at oracle.jsp.runtime.tree.OracleJspClassicTagNode.evalBody(OracleJspClassicTagNode.java:87)
        at oracle.jsp.runtime.tree.OracleJspBodyTagNode.executeHandler(OracleJspBodyTagNode.java:58)
        at oracle.jsp.runtime.tree.OracleJspCustomTagNode.execute(OracleJspCustomTagNode.java:263)
        at oracle.jsp.runtime.tree.OracleJspClassicTagNode.evalBody(OracleJspClassicTagNode.java:87)
        at oracle.jsp.runtime.tree.OracleJspBodyTagNode.executeHandler(OracleJspBodyTagNode.java:58)
        at oracle.jsp.runtime.tree.OracleJspCustomTagNode.execute(OracleJspCustomTagNode.java:263)
        at oracle.jsp.runtime.tree.OracleJspClassicTagNode.evalBody(OracleJspClassicTagNode.java:87)
        at oracle.jsp.runtime.tree.OracleJspBodyTagNode.executeHandler(OracleJspBodyTagNode.java:58)
        at oracle.jsp.runtime.tree.OracleJspCustomTagNode.execute(OracleJspCustomTagNode.java:263)
        at oracle.jsp.runtime.tree.OracleJspClassicTagNode.evalBody(OracleJspClassicTagNode.java:87)
        at oracle.jsp.runtime.tree.OracleJspBodyTagNode.executeHandler(OracleJspBodyTagNode.java:58)
        at oracle.jsp.runtime.tree.OracleJspCustomTagNode.execute(OracleJspCustomTagNode.java:263)
        at oracle.jsp.runtime.tree.OracleJspClassicTagNode.evalBody(OracleJspClassicTagNode.java:87)
        at oracle.jsp.runtime.tree.OracleJspIterationTagNode.executeHandler(OracleJspIterationTagNode.java:45)
        at oracle.jsp.runtime.tree.OracleJspCustomTagNode.execute(OracleJspCustomTagNode.java:263)
        at oracle.jsp.runtime.tree.OracleJspClassicTagNode.evalBody(OracleJspClassicTagNode.java:87)
        at oracle.jsp.runtime.tree.OracleJspBodyTagNode.executeHandler(OracleJspBodyTagNode.java:58)
        at oracle.jsp.runtime.tree.OracleJspCustomTagNode.execute(OracleJspCustomTagNode.java:263)
        at oracle.jsp.runtime.tree.OracleJspClassicTagNode.evalBody(OracleJspClassicTagNode.java:87)
        at oracle.jsp.runtime.tree.OracleJspBodyTagNode.executeHandler(OracleJspBodyTagNode.java:58)
        at oracle.jsp.runtime.tree.OracleJspCustomTagNode.execute(OracleJspCustomTagNode.java:263)
        at oracle.jsp.runtime.tree.OracleJspClassicTagNode.evalBody(OracleJspClassicTagNode.java:87)
        at oracle.jsp.runtime.tree.OracleJspIterationTagNode.executeHandler(OracleJspIterationTagNode.java:45)
        at oracle.jsp.runtime.tree.OracleJspCustomTagNode.execute(OracleJspCustomTagNode.java:263)
        at oracle.jsp.runtime.tree.OracleJspClassicTagNode.evalBody(OracleJspClassicTagNode.java:87)
        at oracle.jsp.runtime.tree.OracleJspBodyTagNode.executeHandler(OracleJspBodyTagNode.java:58)
        at oracle.jsp.runtime.tree.OracleJspCustomTagNode.execute(OracleJspCustomTagNode.java:263)
        at oracle.jsp.runtime.tree.OracleJspNode.execute(OracleJspNode.java:89)
        at oracle.jsp.runtimev2.ShortCutServlet._jspService(ShortCutServlet.java:89)
        at oracle.jsp.runtime.OracleJspBase.service(OracleJspBase.java:29)
        at oracle.jsp.runtimev2.JspPageTable.service(JspPageTable.java:424)
        at oracle.jsp.runtimev2.JspServlet.internalService(JspServlet.java:845)
        at oracle.jsp.runtimev2.JspServlet.service(JspServlet.java:760)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:844)
        at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:280)
        at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:254)
        at weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:136)
        at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:346)
        at weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:25)
        at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:79)
        at oracle.sysman.eml.app.JSPFilter.doFilter(JSPFilter.java:86)
        at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:79)
        at oracle.security.jps.ee.http.JpsAbsFilter$1.run(JpsAbsFilter.java:137)
        at java.security.AccessController.doPrivileged(Native Method)
        at oracle.security.jps.util.JpsSubject.doAsPrivileged(JpsSubject.java:315)
        at oracle.security.jps.ee.util.JpsPlatformUtil.runJaasMode(JpsPlatformUtil.java:460)
        at oracle.security.jps.ee.http.JpsAbsFilter.runJaasMode(JpsAbsFilter.java:120)
        at oracle.security.jps.ee.http.JpsAbsFilter.doFilter(JpsAbsFilter.java:217)
        at oracle.security.jps.ee.http.JpsFilter.doFilter(JpsFilter.java:81)
        at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:79)
        at weblogic.servlet.internal.RequestDispatcherImpl.invokeServlet(RequestDispatcherImpl.java:588)
        at weblogic.servlet.internal.RequestDispatcherImpl.include(RequestDispatcherImpl.java:481)
        at weblogic.servlet.jsp.PageContextImpl.include(PageContextImpl.java:162)
        at weblogic.servlet.jsp.PageContextImpl.include(PageContextImpl.java:183)
        at oracle.adfinternal.view.faces.taglib.region.IncludeTag.__include(IncludeTag.java:473)
        at oracle.adfinternal.view.faces.taglib.region.AbstractRegionTag$1.call(AbstractRegionTag.java:101)
        at oracle.adfinternal.view.faces.taglib.region.AbstractRegionTag$1.call(AbstractRegionTag.java:71)
        at oracle.adf.view.rich.component.fragment.UIXRegion.processRegion(UIXRegion.java:526)
        at oracle.adfinternal.view.faces.taglib.region.AbstractRegionTag.doStartTag(AbstractRegionTag.java:70)
        at oracle.adfinternal.view.faces.taglib.region.RegionTag.doStartTag(RegionTag.java:92)
        at oracle.jsp.runtime.tree.OracleJspBodyTagNode.executeHandler(OracleJspBodyTagNode.java:50)
        at oracle.jsp.runtime.tree.OracleJspCustomTagNode.execute(OracleJspCustomTagNode.java:263)
        at oracle.jsp.runtime.tree.OracleJspClassicTagNode.evalBody(OracleJspClassicTagNode.java:87)
        at oracle.jsp.runtime.tree.OracleJspIterationTagNode.executeHandler(OracleJspIterationTagNode.java:45)
        at oracle.jsp.runtime.tree.OracleJspCustomTagNode.execute(OracleJspCustomTagNode.java:263)
        at oracle.jsp.runtime.tree.OracleJspClassicTagNode.evalBody(OracleJspClassicTagNode.java:87)
        at oracle.jsp.runtime.tree.OracleJspBodyTagNode.executeHandler(OracleJspBodyTagNode.java:58)
        at oracle.jsp.runtime.tree.OracleJspCustomTagNode.execute(OracleJspCustomTagNode.java:263)
        at oracle.jsp.runtime.tree.OracleJspClassicTagNode.evalBody(OracleJspClassicTagNode.java:87)
        at oracle.jsp.runtime.tree.OracleJspIterationTagNode.executeHandler(OracleJspIterationTagNode.java:45)
        at oracle.jsp.runtime.tree.OracleJspCustomTagNode.execute(OracleJspCustomTagNode.java:263)
        at oracle.jsp.runtime.tree.OracleJspClassicTagNode.evalBody(OracleJspClassicTagNode.java:87)
        at oracle.jsp.runtime.tree.OracleJspBodyTagNode.executeHandler(OracleJspBodyTagNode.java:58)
        at oracle.jsp.runtime.tree.OracleJspCustomTagNode.execute(OracleJspCustomTagNode.java:263)
        at oracle.jsp.runtime.tree.OracleJspClassicTagNode.evalBody(OracleJspClassicTagNode.java:87)
        at oracle.jsp.runtime.tree.OracleJspIterationTagNode.executeHandler(OracleJspIterationTagNode.java:45)
        at oracle.jsp.runtime.tree.OracleJspCustomTagNode.execute(OracleJspCustomTagNode.java:263)
        at oracle.jsp.runtime.tree.OracleJspClassicTagNode.evalBody(OracleJspClassicTagNode.java:87)
        at oracle.jsp.runtime.tree.OracleJspBodyTagNode.executeHandler(OracleJspBodyTagNode.java:58)
        at oracle.jsp.runtime.tree.OracleJspCustomTagNode.execute(OracleJspCustomTagNode.java:263)
        at oracle.jsp.runtime.tree.OracleJspClassicTagNode.evalBody(OracleJspClassicTagNode.java:87)
        at oracle.jsp.runtime.tree.OracleJspBodyTagNode.executeHandler(OracleJspBodyTagNode.java:58)
        at oracle.jsp.runtime.tree.OracleJspCustomTagNode.execute(OracleJspCustomTagNode.java:263)
        at oracle.jsp.runtime.tree.OracleJspClassicTagNode.evalBody(OracleJspClassicTagNode.java:87)
        at oracle.jsp.runtime.tree.OracleJspBodyTagNode.executeHandler(OracleJspBodyTagNode.java:58)
        at oracle.jsp.runtime.tree.OracleJspCustomTagNode.execute(OracleJspCustomTagNode.java:263)
        at oracle.jsp.runtime.tree.OracleJspClassicTagNode.evalBody(OracleJspClassicTagNode.java:87)
        at oracle.jsp.runtime.tree.OracleJspBodyTagNode.executeHandler(OracleJspBodyTagNode.java:58)
        at oracle.jsp.runtime.tree.OracleJspCustomTagNode.execute(OracleJspCustomTagNode.java:263)
        at oracle.jsp.runtime.tree.OracleJspNode.execute(OracleJspNode.java:89)
        at oracle.jsp.runtimev2.ShortCutServlet._jspService(ShortCutServlet.java:89)
        at oracle.jsp.runtime.OracleJspBase.service(OracleJspBase.java:29)
        at oracle.jsp.runtimev2.JspPageTable.service(JspPageTable.java:424)
        at oracle.jsp.runtimev2.JspServlet.internalService(JspServlet.java:845)
        at oracle.jsp.runtimev2.JspServlet.service(JspServlet.java:760)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:844)
        at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:280)
        at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:254)
        at weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:136)
        at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:346)
        at weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:25)
        at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:79)
        at oracle.sysman.eml.app.JSPFilter.doFilter(JSPFilter.java:86)
        at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:79)
        at oracle.adf.library.webapp.LibraryFilter.doFilter(LibraryFilter.java:202)
        at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:79)
        at oracle.security.jps.ee.http.JpsAbsFilter$1.run(JpsAbsFilter.java:137)
        at java.security.AccessController.doPrivileged(Native Method)
        at oracle.security.jps.util.JpsSubject.doAsPrivileged(JpsSubject.java:315)
        at oracle.security.jps.ee.util.JpsPlatformUtil.runJaasMode(JpsPlatformUtil.java:460)
        at oracle.security.jps.ee.http.JpsAbsFilter.runJaasMode(JpsAbsFilter.java:120)
        at oracle.security.jps.ee.http.JpsAbsFilter.doFilter(JpsAbsFilter.java:217)
        at oracle.security.jps.ee.http.JpsFilter.doFilter(JpsFilter.java:81)
        at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:79)
        at weblogic.servlet.internal.RequestDispatcherImpl.invokeServlet(RequestDispatcherImpl.java:588)
        at weblogic.servlet.internal.RequestDispatcherImpl.forward(RequestDispatcherImpl.java:273)
        at com.sun.faces.context.ExternalContextImpl.dispatch(Unknown Source)
        at javax.faces.context.ExternalContextWrapper.dispatch(Unknown Source)
        at javax.faces.context.ExternalContextWrapper.dispatch(Unknown Source)
        at javax.faces.context.ExternalContextWrapper.dispatch(Unknown Source)
        at javax.faces.context.ExternalContextWrapper.dispatch(Unknown Source)
        at org.apache.myfaces.trinidadinternal.context.FacesContextFactoryImpl$OverrideDispatch.dispatch(FacesContextFactoryImpl.java:167)
        at com.sun.faces.application.view.JspViewHandlingStrategy.executePageToBuildView(Unknown Source)
        at com.sun.faces.application.view.JspViewHandlingStrategy.buildView(Unknown Source)
        at org.apache.myfaces.trinidad.view.ViewDeclarationLanguageWrapper.buildView(ViewDeclarationLanguageWrapper.java:94)
        at org.apache.myfaces.trinidad.view.ViewDeclarationLanguageWrapper.buildView(ViewDeclarationLanguageWrapper.java:94)
        at org.apache.myfaces.trinidadinternal.application.ViewDeclarationLanguageFactoryImpl$ChangeApplyingVDLWrapper.buildView(ViewDeclarationLanguageFactoryImpl.java:322)
        at oracle.adfinternal.view.faces.lifecycle.ResponseRenderManager._processViewDefinitionLanguage(ResponseRenderManager.java:105)
        at oracle.adfinternal.view.faces.lifecycle.ResponseRenderManager.runRenderView(ResponseRenderManager.java:41)
        at oracle.adfinternal.view.faces.lifecycle.LifecycleImpl._renderResponse(LifecycleImpl.java:1095)
        at oracle.adfinternal.view.faces.lifecycle.LifecycleImpl._executePhase(LifecycleImpl.java:389)
        at oracle.adfinternal.view.faces.lifecycle.LifecycleImpl.render(LifecycleImpl.java:255)
        at javax.faces.webapp.FacesServlet.service(Unknown Source)
        at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:280)
        at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:254)
        at weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:136)
        at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:346)
        at weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:25)
        at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:79)
        at oracle.adf.model.servlet.ADFBindingFilter.doFilter(ADFBindingFilter.java:192)
        at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:79)
        at oracle.adf.view.page.editor.webapp.WebCenterComposerFilter.doFilter(WebCenterComposerFilter.java:117)
        at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:79)
        at oracle.adf.share.http.ServletADFFilter.doFilter(ServletADFFilter.java:77)
        at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:79)
        at oracle.adfinternal.view.faces.webapp.rich.RegistrationFilter.doFilter(RegistrationFilter.java:105)
        at org.apache.myfaces.trinidadinternal.webapp.TrinidadFilterImpl$FilterListChain.doFilter(TrinidadFilterImpl.java:502)
        at oracle.adfinternal.view.faces.activedata.AdsFilter.doFilter(AdsFilter.java:60)
        at org.apache.myfaces.trinidadinternal.webapp.TrinidadFilterImpl$FilterListChain.doFilter(TrinidadFilterImpl.java:502)
        at org.apache.myfaces.trinidadinternal.webapp.TrinidadFilterImpl._doFilterImpl(TrinidadFilterImpl.java:327)
        at org.apache.myfaces.trinidadinternal.webapp.TrinidadFilterImpl.doFilter(TrinidadFilterImpl.java:229)
        at org.apache.myfaces.trinidad.webapp.TrinidadFilter.doFilter(TrinidadFilter.java:92)
        at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:79)
        at oracle.sysman.eml.app.EMRepLoginFilter.doFilter(EMRepLoginFilter.java:800)
        at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:79)
        at oracle.sysman.eml.app.MBeanServerConnFilter.doFilter(MBeanServerConnFilter.java:43)
        at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:79)
        at oracle.adf.library.webapp.LibraryFilter.doFilter(LibraryFilter.java:202)
        at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:79)
        at oracle.sysman.emInternalSDK.core.model.utils.EntityManagerFilter.doFilter(EntityManagerFilter.java:46)
        at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:79)
        at oracle.sysman.eml.app.ContextInitFilter.doFilter(ContextInitFilter.java:561)
        at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:79)
        at oracle.adfinternal.view.faces.caching.filter.AdfFacesCachingFilterImpl.doFilter(AdfFacesCachingFilterImpl.java:142)
        at oracle.adf.view.rich.webapp.AdfFacesCachingFilter.doFilter(AdfFacesCachingFilter.java:66)
        at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:79)
        at oracle.sysman.core.app.exception.EMRuntimeExceptionFilter.doFilter(EMRuntimeExceptionFilter.java:150)
        at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:79)
        at oracle.security.jps.ee.http.JpsAbsFilter$1.run(JpsAbsFilter.java:137)
        at java.security.AccessController.doPrivileged(Native Method)
        at oracle.security.jps.util.JpsSubject.doAsPrivileged(JpsSubject.java:315)
        at oracle.security.jps.ee.util.JpsPlatformUtil.runJaasMode(JpsPlatformUtil.java:460)
        at oracle.security.jps.ee.http.JpsAbsFilter.runJaasMode(JpsAbsFilter.java:120)
        at oracle.security.jps.ee.http.JpsAbsFilter.doFilter(JpsAbsFilter.java:217)
        at oracle.security.jps.ee.http.JpsFilter.doFilter(JpsFilter.java:81)
        at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:79)
        at oracle.dms.servlet.DMSServletFilter.doFilter(DMSServletFilter.java:220)
        at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:79)
        at weblogic.servlet.internal.RequestEventsFilter.doFilter(RequestEventsFilter.java:27)
        at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:79)
        at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.wrapRun(WebAppServletContext.java:3436)
        at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3402)
        at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
        at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:120)
        at weblogic.servlet.provider.WlsSubjectHandle.run(WlsSubjectHandle.java:57)
        at weblogic.servlet.internal.WebAppServletContext.doSecuredExecute(WebAppServletContext.java:2285)
        at weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java:2201)
        at weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:2179)
        at weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1575)
        at weblogic.servlet.provider.ContainerSupportProviderImpl$WlsRequestExecutor.run(ContainerSupportProviderImpl.java:255)
        at weblogic.work.ExecuteThread.execute(ExecuteThread.java:311)
        at weblogic.work.ExecuteThread.run(ExecuteThread.java:263)

Diagnostic Dumps
----------------

Name: jvm.threads
Dump Arguments: context=true, timing=true
Dump Files: jvm_threads276_i1191.txt

Name: adf.AdfcDiagnosticDump
Application Name: emgc
Dump Files: adf_AdfcDiagnosticDump281_i1191.txt

Name: adf.ADFConfigDiagnosticDump
Application Name: emgc
Dump Files: adf_ADFConfigDiagnosticDump282_i1191.txt

Name: adf.ADFConfigPropertiesDump
Application Name: emgc
Dump Files: adf_ADFConfigPropertiesDump284_i1191.txt

Name: dms.metrics
Dump Arguments: dump=LocalDMS
Dump Files: dms_metrics277_i1191.txt

Name: dfw.samplingArchive
Dump Files: dfw_samplingArchive279_i1191.JVMThreadDump.txt,dfw_samplingArchive279_i1191.readme.txt

Name: odl.logs
Dump Arguments: match_all=false, timestamp=1514282842187, ecid=005OJhmavbmFg485zj1FiW0009Nm0006cv
Dump Files: odl_logs280_i1191.txt

Name: adf.DiagnosticsJarsVersionDump
Dump Files: adf_DiagnosticsJarsVersionDump283_i1191.txt

 
GATHER DETAILS

By querying the OMS database get details about the latest alert before any changes

ALTER SESSION SET NLS_DATE_FORMAT='DD-MM-YYYY HH24:MI:SS';
SET line 300
SET pages 999
COL MESSAGE FOR A150

SELECT * FROM (
SELECT TO_DATE(COLLECTION_TIMESTAMP,'DD-MM-YYYY HH24:MI:SS') "COLLECTION_TIMESTAMP", MESSAGE 
FROM   MGMT_VIEW.MGMT$ALERT_NOTIF_LOG 
WHERE  MESSAGE LIKE '%ADFC-00032%' ORDER BY 1 DESC
) WHERE ROWNUM < 3; 

COLLECTION_TIMESTAM MESSAGE
------------------- ------------------------------------------------------------------------------------------------------------------------------------------------------
26-12-2017 13:07:43 Incident (ADFC-00032 [ADFc]) detected in /orastage/oem13100/gc_inst/user_projects/domains/GCDomain/servers/EMGC_OMS1/adr/diag/ofm/EMGC_DOMAIN/EMOMS/alert/log.xml at time/line number: Tue Dec 26 13:07:22 2017/11012.

26-12-2017 13:07:43 Incident (ADFC-00032 [ADFc]) detected in /orastage/oem13100/gc_inst/user_projects/domains/GCDomain/servers/EMGC_OMS1/adr/diag/ofm/EMGC_DOMAIN/EMOMS/alert/log.xml at time/line number: Tue Dec 26 13:07:20 2017/11003.
					
COL SYSTIMESTAMP FOR A35 
SELECT SYSTIMESTAMP FROM DUAL;

SYSTIMESTAMP
-----------------------------------
26-DEC-17 02.17.43.885503 PM +03:00

Total errors

SET LINES 300
COL TARGET_NAME FOR A50 
COL TARGET_TYPE FOR A10 

SELECT TARGET_NAME, TARGET_TYPE, COUNT(*) "GENERATED ERRORS"
FROM   MGMT_VIEW.MGMT$ALERT_NOTIF_LOG 
WHERE  MESSAGE LIKE '%ADFC-00032%'  
GROUP BY TARGET_NAME, TARGET_TYPE ORDER BY 1;

TARGET_NAME                          TARGET_TYP GENERATED ERRORS
------------------------------------ ---------- ----------------
dbpilot.net:4889_Management_Service  oracle_oms             1011

Number of errors by month

ALTER SESSION SET NLS_DATE_FORMAT='MM-YYYY';

SELECT TO_DATE(COLLECTION_TIMESTAMP,'MM-YYYY') "MONTH",
       COUNT(*) "GENERATED ERRORS"
FROM   MGMT_VIEW.MGMT$ALERT_NOTIF_LOG 
WHERE  MESSAGE LIKE '%ADFC-00032%'  
GROUP BY TO_DATE(COLLECTION_TIMESTAMP,'MM-YYYY') ORDER BY 1;

MONTH   GENERATED ERRORS
------- ----------------
03-2017               72
04-2017               65
05-2017              107
06-2017               20
07-2017               25
08-2017               45
09-2017               48
10-2017              100
11-2017               55
12-2017               45

Number of errors by target name

ALTER SESSION SET NLS_DATE_FORMAT='MM-YYYY';

SET LINES 300
COL TARGET_NAME FOR A60 
BREAK ON TARGET_NAME
SELECT TARGET_NAME,TO_DATE(COLLECTION_TIMESTAMP,'MM-YYYY') "MONTH",
       COUNT(*) "GENERATED ERRORS"
FROM   MGMT_VIEW.MGMT$ALERT_NOTIF_LOG 
WHERE  MESSAGE LIKE '%ADFC-00032%'  
GROUP BY TARGET_NAME,TO_DATE(COLLECTION_TIMESTAMP,'MM-YYYY')
ORDER BY 1,2;

TARGET_NAME                             MONTH   GENERATED ERRORS
-------------------------------------- ------- ----------------
dbpilot.net:4889_Management_Service    03-2017               72
                                       04-2017               65
                                       05-2017              107
                                       06-2017               20
                                       07-2017               25
                                       08-2017               45
                                       09-2017               48
                                       10-2017              100
                                       11-2017               55
                                       12-2017               45

 
SOLUTION

A) Perform a full backup of all components (OMS, Database, Agent)

B) Apply patches

    For EM 13.2 Cloud Control:
    Apply the following PSU for the EM 13.2 Cloud Control OMS (or later):
    EMBPPSU 13.2.0.0.170418, , or later
    Alternatively, apply Patch 24742172 to the EM 13.2 OMS Oracle Home according to the Patch README file.

    For EM 13.1 Cloud Control:
    Apply the following PSU for the EM 13.1 Cloud Control OMS (or later):
    EMBPPSU 13.1.0.0.170718, Patch 25904755, or later
    Alternatively, apply Patch 24742172 to the EM 13.1 OMS Oracle Home according to the Patch README file.

 
REFERENCES

EM 13c: ADFC-00032 [ADFc] Error Messages Reported Frequently in Enterprise Manager 13c Cloud Control (Doc ID 2141011.1)
EM 13c: Enterprise Manager 13.2 Cloud Control Repeatedly Reports Log Monitor Incident with ADFC-00032 Errors (Doc ID 2240806.1)

NOTES for EM 13.1 Cloud Control

(a) Patch 23069794 (Doc ID 2141011.1) does not fix the ADFC-00032 [ADFc] issue
(b) Before installing patch 24742172 (Doc ID 2240806.1) refer to 2335997.1
EM 13: EM Middleware (EMGC_GCDomain) Targets and System Infrastructure Server Target show down/pending with Incidents after EM PSUs/OMS/Agent Bundle Patches are applied using opatch 13.9.2.1.0 (Doc ID 2335997.1)
(c) Don’t apply any patches with faulty 6880880 (OPatch Version: 13.9.2.1.0) until Opatch patch 27155076 is applied.

    The patch 24742172 must be installed in the following order :
    1. OMS, Update OMSPatcher utility by installing patch 19999993 (OMSPatcher Version: 13.6.0.0.4).
    2. OMS and Agent, Update OPatch utility by installing patch 6880880 (OPatch Version: 13.9.2.1.0)
    3. OMS and Agent, Apply the OPatch Patch 27155076
    4. OMS, Apply the patch 24742172
    These steps were actually at the time of the post writing (17:20 28.12.2017)
 
In this test case I’ve applied 24742172 on EM 13.1 Cloud Control and it’s fixed the issue.
 
(d) Added later, after 1 month [16:21 31.01.2018]
The 24742172 patch hasn’t fixed the issue completely, but a number of generated errors have noticeably decreased.

TARGET_NAME                                                  MONTH   GENERATED ERRORS
------------------------------------------------------------ ------- ----------------
s001as-oem13c.sibur.local:4889_Management_Service            04-2017               65
                                                             05-2017              107
                                                             06-2017               20
                                                             07-2017               25
                                                             08-2017               45
                                                             09-2017               48
                                                             10-2017              100
                                                             11-2017               55
                                                             12-2017               50
                                                             01-2018                8

 
 

Version  : 17:20 28.12.2017
Platform : Red Hat 6.8
OEM      : 13.1.0.0.0