ORACLE MANAGEMENT AGENT COLLECTION STATUS IS COLLECTIONS_HALTED
The emctl status agent command shows the Agent is Running and Ready. However, the agent's collection status is COLLECTIONS_HALTED.
oracle@dbpilot|09:23|~/scripts:emctl status agent Oracle Enterprise Manager Cloud Control 12c Release 5 Copyright (c) 1996, 2015 Oracle Corporation. All rights reserved. --------------------------------------------------------------- Agent Version : 12.1.0.5.0 OMS Version : 13.3.0.0.0 Protocol Version : 12.1.0.1.0 Agent Home : /agent12c/agent_inst Agent Log Directory : /agent12c/agent_inst/sysman/log Agent Binaries : /agent12c/core/12.1.0.5.0 Agent Process ID : 21684 Parent Process ID : 21574 Agent URL : https://dbpilot.net:3872/emd/main/ Local Agent URL in NAT : https://dbpilot.net:3872/emd/main/ Repository URL : https://oem13c-r3.net:4903/empbs/upload Started at : 2019-07-23 10:22:59 Started by user : oracle Operating System : Linux version 2.6.39-400.215.10.el5uek (amd64) Last Reload : (none) Last successful upload : 2019-07-23 10:23:24 Last attempted upload : 2019-07-23 10:23:24 Total Megabytes of XML files uploaded so far : 0.03 Number of XML files pending upload : 4,911 Size of XML files pending upload(MB) : 2.48 Available disk space on upload filesystem : 49.52% Collection Status : [COLLECTIONS_HALTED (UPLOAD SYSTEM Threshold - unable to purge files in upload system)] Backoff Expiration : 2019-07-23 10:23:56 Heartbeat Status : Ok Last attempted heartbeat to OMS : 2019-07-23 10:23:20 Last successful heartbeat to OMS : 2019-07-23 10:23:20 Next scheduled heartbeat to OMS : 2019-07-23 10:24:21 --------------------------------------------------------------- Agent is Running and Ready
All targets monitored by the problematic agent are in DOWN state.
1. Shutdown the OMS
$OMS_HOME/bin/emctl stop oms -all -force
2. After the OMS is stopped verify no processes are left running
{
ps -ef | grep EMGC_ADMINSERVER | grep -v grep
ps -ef | grep EMGC_OMS1 | grep -v grep
ps -ef | grep java | grep -v grep
ps -ef | grep opmn | grep -v grep
}
If there are any left OMS java processes, kill them by using the kill command
kill -9 PID
where PID is a process identifier for a left process
3. Restart the OMS repository database and the database listener
-- Shutdown the database
sqlplus / as sysdba
SQL> shutdown immediate;
SQL> exit
-- Restart the listener
lsnrctl stop
lsnrctl start
-- Start up the database
sqlplus / as sysdba
SQL> startup;
SQL> exit
4. Login to the OMS repository database and bounce the job subsystem
-- Connect as the SYS user
SQL> CONNECT / AS SYSDBA
Connected.
SQL> SHOW USER
USER is "SYS"
-- Save the current value of the job_queue_processes parameter
SQL> sho parameter job_queue_processes
-- Set the job_queue_processes parameter to 0
SQL> alter system set job_queue_processes=0;
-- Connect as the SYSMAN user and disable the job
SQL> connect sysman/your_sysman_password
Connected.
SQL> sho user
USER is "SYSMAN"
SQL> exec emd_maintenance.remove_em_dbms_jobs;
SQL> commit;
-- Connect as the SYS user and restore the job_queue_processes parameter
SQL> CONNECT / AS SYSDBA
Connected.
SQL> SHOW USER
USER is "SYS"
SQL> alter system set job_queue_processes=1000 scope=BOTH;
-- Connect as the SYSMAN user and enable the job
SQL> connect sysman/your_sysman_password
Connected.
SQL> sho user
USER is "SYSMAN"
SQL> exec emd_maintenance.submit_em_dbms_jobs;
SQL> commit;
5. Start up the OMS
$OMS_HOME/bin/emctl start oms
6. When the OMS is Up and Running, run the following commands on all hosts with the agent's collection status COLLECTIONS_HALTED
{
${AGENT_HOME}/bin/emctl stop agent
${AGENT_HOME}/bin/emctl clearstate agent
${AGENT_HOME}/bin/emctl start agent
${AGENT_HOME}/bin/emctl upload agent
}
The agent may have many files to upload, and you might need to issue the upload command several times before all files have been uploaded.
7. Verify the issue with the agent is gone
oracle@dbpilot|10:14|~:${AGENT_HOME}/bin/emctl status agent Oracle Enterprise Manager Cloud Control 12c Release 5 Copyright (c) 1996, 2015 Oracle Corporation. All rights reserved. --------------------------------------------------------------- Agent Version : 12.1.0.5.0 OMS Version : 13.3.0.0.0 Protocol Version : 12.1.0.1.0 Agent Home : /agent12c/agent_inst Agent Log Directory : /agent12c/agent_inst/sysman/log Agent Binaries : /agent12c/core/12.1.0.5.0 Agent Process ID : 31671 Parent Process ID : 31488 Agent URL : https://dbpilot.net:3872/emd/main/ Local Agent URL in NAT : https://dbpilot.net:3872/emd/main/ Repository URL : https://oem13c-r3.net:4903/empbs/upload Started at : 2019-07-23 11:12:26 Started by user : oracle Operating System : Linux version 2.6.39-400.215.10.el5uek (amd64) Last Reload : (none) Last successful upload : 2019-07-23 11:15:04 Last attempted upload : 2019-07-23 11:15:04 Total Megabytes of XML files uploaded so far : 0.74 Number of XML files pending upload : 1 Size of XML files pending upload(MB) : 0 Available disk space on upload filesystem : 49.51% Collection Status : Collections enabled Heartbeat Status : Ok Last attempted heartbeat to OMS : 2019-07-23 11:14:22 Last successful heartbeat to OMS : 2019-07-23 11:14:22 Next scheduled heartbeat to OMS : 2019-07-23 11:15:22 --------------------------------------------------------------- Agent is Running and Ready
As the result the Collection Status of the agent is changed from the [COLLECTIONS_HALTED] to the Collections enabled.
EM 13c Agent Unreachable - Halted - UPLOAD SYSTEM Threshold - Unable To Purge Files In Upload System (Doc ID 2462039.1)
- Accounts
- Auditing
- AWR
- Bash Scripts
- Datapump
- Default Category
- Demos
- Directory Objects
- Environment Variables
- Initialization Parameters
- Iptables
- Java Program
- Memory Usage
- Metadata API
- Networker
- NLS Settings
- Optimizer Statistics
- ORA-00942
- ORA-01031
- ORA-01720
- ORA-28001
- ORA-31671
- Oracle Database
- Oracle Enterprise Manager
- Performance Tunning
- Postfix
- Privilegies
- Processes
- Queries
- Red Hat Enterprise Linux
- Redo Logs
- Session Tracing
- Sessions
- SQL Trace
- SQLPlus
- Statspack
- Tablespaces
- UTL_FILE
- UTL_FILE_DIR
- Wait Events
- Yum