APSYNCH AP Errors Requiring TPS Reset, Finally Unable to Boot Applications Software, Hostmonitor Timeout after 700 Secon…

APSYNCH AP Errors Requiring TPS Reset, Finally Unable to Boot Applications Software, Hostmonitor Timeout after 700 Seconds See Long Description This is a 9.1 Goldseal system. The customer started Getting "APSYNCH" AP Errors requiring a TPS reset, which didn't help. The FE arrived and power cycled the TPS and rebooted the system which also didn't help. Then he tried reseating the MEDCAM board in the TPS chassis. Finally the system was unable to boot applications software, and popped up a "Hostmonitor Timeout after 700 Seconds". The FE didn't attempt to run any diagnostics during this troubleshooting session. See error sequence below: The initial event: Thu Jan 18 08:40:32 2007 2228010 1 tbjmr NSP TPS:APSYNCH 10 Thu Jan 18 08:40:32 2007 2228016 2 tbjmr NSP TPS:ap_log_err 0 Internal AP software error. Please consult file /usr/g/service/log/aperror.log for more information. Thu Jan 18 08:40:32 2007 2228010 1 tbjmr NSP TPS:rc_recnp1_seg 80 AP Error Thu Jan 18 08:41:06 2007 2228718 2 tbjmr NSP TPS:as_event_mgr 10 Failed in cm_rec(). Thu Jan 18 08:41:09 2007 2224666 1 tbjmr psc PscMainControl.cc 1322 Auto Prescan failed, Autoshim failed in Fermi filter creation. Thu Jan 18 08:41:14 2007 2224709 2 tbjmr NSP TPS:aps_frame_range_check 10 Low signal detected. Please check the signal level for receiver 3 (or receiver 4 while using Manual Prescan). Note that for 3 coil scans, one coil will cause a low signal detected message to occur. Thu Jan 18 08:41:36 2007 2227557 1 tbjmr sdm sdm_main.c 213 *** Reset/Download TPS In progress. *** Thu Jan 18 08:41:36 2007 2219668 1 tbjmr scn MsaScanMgr.m 836 Please restart the prescription after the TPS reset has completed. Thu Jan 18 08:41:36 2007 2218636 1 tbjmr scn StateMachine.c 95 The event: 3 is invalid in state 8. Thu Jan 18 08:43:27 2007 200002354 4 tbjmr hostMonitor ssProcStop.c 1118 An unexpected death of RtpHost caused it to be restarted. Thu Jan 18 08:43:27 2007 2227555 1 tbjmr sdm sdm_main.c 307 *** Reset/Download TPS was successful. *** (There were several repeats of the above sequence as the customer, and later the FE, kept trying. None of their retries and further TPS resets and system reboots changed the symptoms) ----------------------------------------------------------------------------------------- (Later, after the FE had reseated the MEDCAM board in the TPS chassis, the symptoms changed. It got worse -- now the system wouldn't even boot, reporting a "Hostmonitor Timeout after 700 Seconds". Thu Jan 18 11:13:09 2007 200002368 4 tbjmr hostMonitor ssProcStart.c 1099 A set of processes are deadlocked waiting for each other to start. "svat, spectro, scn, rtcProxy, resetServer, lais, hgp, displayServer, autoview, app, WLServer, TIR, SITT, RtpHost, PPSproxy" are the likely culprits. This deadlock has caused the system to shutdown! Thu Jan 18 11:13:09 2007 200002321 4 tbjmr hostMonitor ssProcStop.c 1403 No processes could be shutdown, skipping precursor limitations and trying again. Thu Jan 18 11:13:09 2007 200002322 4 tbjmr hostMonitor ssProcStop.c 1416 No processes could be shutdown, so exiting. Thu Jan 18 11:13:09 2007 200002348 4 tbjmr hostMonitor ssProcStop.c 1531 The System Software has terminated.