|
Errors generated were most likely caused by recent power issues |
|
scottl |
03/12/2013 07:55 AM |
Site has been up, correcting the state on the new TASS. |
Site Operation |
thumphrey |
05/27/2013 09:39 AM |
Manually pushed the delinquent files |
|
thumphrey |
06/03/2013 08:42 AM |
MiniMon was exited briefly at ~0735 PST 6-3-2013 to delete a zip file. Files CFF13153.H1 to CFF13154.B1 were manually pushed to PTAGIS. There will be a brief file gap. No activity was observed on PC2 during PC1's downtime. No further problems to report. |
|
Alan |
06/06/2013 07:58 AM |
MiniMon was exited briefly at ~0655 PST 6-6-2013 to delete a zip file. File CFF13157.E2 covers the time while PC1 was down. Files CFF13155.C1 to CFF13157.C1 were manually pushed to PTAGIS. No further problems to report. |
|
Alan |
06/10/2013 08:18 AM |
MiniMon was briefly exited at ~7:10 PST 6/10/2013 to delete a zip file. Files CFF13161.A1 to .C1 were manually uploaded to PTAGIS. File CFF13161.D2 covers the time while PC1 was out of data collection. No further problems to report. |
|
Alan |
06/13/2013 12:16 PM |
A generator overheated today causing a brief power outage. The other unit came online, and restored power. I’ll be going up there tomorrow morning to fix the problem.
Jerod Bartholomew
Klickitat Maintenance Manager – YKFP
|
|
Alan |
06/13/2013 02:48 PM |
Change of plans gentlemen. I'm at Castile now. There will be some short power outages as I perform the repairs.
Jerod.
|
|
Alan |
09/28/2013 10:45 AM |
These errors were generated dunring the remodel, not od concern |
01: Reader Set To Master.
01: Sync. Input Not Present.
02: Reader Set To Master.
02: Sync. Input Not Present. |
scottl |
10/01/2013 08:33 AM |
Buffered transceiver data from all three transceivers show that no detections occurred during the file gaps on 9/23 and 9/27 2013. |
|
Alan |
10/02/2013 08:10 AM |
For unknown reasons, Windows rebooted on the primary machine at 00:54 PST 10-2-2013, creating an~ 4 minute file gap. The secondary PC remained online. No detections were recorded during the primary machine's downtime. |
|
Alan |
10/02/2013 09:43 AM |
one-time alarm |
show errors |
Alan |
10/02/2013 09:43 AM |
clearing TASS |
01: Noise Level High. |
Alan |
10/03/2013 12:22 PM |
At the site to install two new computers, touch up the comms cabinet (installed a DIN rail mounted switch) and to test the RedLion DSP.
No problems to report. |
|
thumphrey |
10/04/2013 08:20 AM |
This will be investigated during the next site visit. |
01: Reader Set To Master.
02: Reader Set To Master.
01: Sync. Input Not Present.
02: Sync. Input Not Present. |
Alan |
10/07/2013 07:47 AM |
Noted, will monitor |
01: Noise Level High. |
scottl |
10/07/2013 12:14 PM |
Platform maintenance caused six file gaps from 10/3/2013 to 10/6/2013. Examination of secondary M4 PC data shows that no interrogations occurred during those times. |
|
Alan |
10/17/2013 09:11 AM |
For unknown reasons, a three minute data gap occurred on 10/10/2013 at 0218. The event log on the primary PC showed no information as to why the gap occurred. Examination of secondary PC data showed that no detections occurred during the gap. |
|
Alan |
11/13/2013 08:35 AM |
Reboots on both PCs from windows update caused a three minute data gap at ~0315 11-13-2013. The buffers will be downloaded during the next site visit. Windows automatic updates was remotely set to the PTAGIS standard "download but do not install updates" on both PCs. |
|
Alan |
11/21/2013 07:53 AM |
Examination of buffered data from all three transceivers shows that no detections occurred during the three minute data gap at ~0315 11-13-2013. |
|
Alan |
03/03/2014 09:13 AM |
The PSMFC .115 network is showing RED in Zymon. Talked with Jerod of the YIN and it was agreed to hold off any action to see if it returns on its own. The 116 network is still up and data submission should not be effected. Will monitor and take action if necessary. |
|
scottl |
03/04/2014 01:00 AM |
Unchecked the DST box in both PC1 and 2, Restarted services for changes to take effect |
|
scottl |
03/25/2014 11:31 AM |
Restarted M4 background service on both PCs. No activity observed on SEC while PRI out of service. |
|
Alan |
04/02/2014 11:38 AM |
Loaded TextPad on PC1 and 2, Total Cmdr was already installed. |
|
scottl |
04/02/2014 11:38 AM |
Traveled to site for pre-season maintenance. Issued full tune on transceivers, changed account name on both PC's to standard convention. Rebooted PC1 at 10:21:50, re-booted PC2 at 10:29:39 PST. No problems to report. |
|
scottl |
04/03/2014 07:51 AM |
Attempting to clear alarms again. See previous PIT event. |
02 |
rclark |
04/03/2014 07:52 AM |
Alarms caused by PSMFC personnel during site visit. |
01 |
rclark |
04/03/2014 03:22 PM |
Examination of SEC data shows that do detections occurred during the PRI reboot on 4-2-2014. |
|
Alan |
04/14/2014 01:16 PM |
DCP maintenance was performed at this site.
Updates were found to be set to download only.
Updates were brought up to date on each PC and then Updates were turned off.
The DOS name was changed on both PC's.
This was the only other item on the check list that needed to be address.
PTAGIS Kennewick staff verified each PC.
The patch files while PC1 was being updated are CFF-2014-104-S-013, S-014, and S-017.
A site GMC was performed.
Each transceiver was checked for tune.
03 exhibited high signal level than 01 and 02.
A full tune was performed and the noise dropped ~ 2%.
Upon further investigation it was found that the water through 03 is very turbulent
compared to that of 01. 02 is turbulent but not as bad.
A video of each was sent to PTAGIS Kennewick Staff for review.
No problems to report. |
|
thumphrey |
04/16/2014 08:39 AM |
Examination of secondary data shows that no detections occurred during the four, two and five minute data gaps on 4-14-2014. |
|
Alan |
04/20/2014 07:13 AM |
The PC's are not loading. Event log shows a FTP error. The Secondary PC was rebooted in an attempt to get uploads working again. The error has been reported to the Sys admin. |
|
thumphrey |
04/21/2014 08:05 AM |
Manually pushed data files to PTAGIS. Will monitor for reoccurring file load failures. |
|
scottl |
05/08/2014 08:05 AM |
This will be investigated today during a site visit. |
01: Noise Level High.
03: Noise Level High.
01: Virtual Tag Test Failed.
02: Virtual Tag Test Failed.
03: Virtual Tag Test Failed. |
Alan |
05/08/2014 02:05 PM |
Noise occurrence noticed this morning was caused by some replacement converters the YIN had installed yesterday. Replaced the new units with one old per Mike B. He indicated that he will get replacement converters and have us test them prior to deployment. |
|
scottl |
05/09/2014 07:41 AM |
These alarms were addressed yesterday during a site visit. The offending device was removed and will be replaced with a device that does not cause noise on the system. |
01: Noise Level High.
03: Noise Level High.
01: Virtual Tag Test Failed.
02: Virtual Tag Test Failed.
03: Virtual Tag Test Failed.
01: Reader Set To Master.
02: Reader Set To Master.
01: Sync. Input Not Present.
02: Sync. Input Not Present. |
thumphrey |
06/05/2014 07:44 AM |
Noted |
01: Virtual Tag Test Failed.
01: Noise Level High. |
thumphrey |
06/06/2014 07:31 AM |
The noise on all transceivers elevated for ~10 hour period yesterday. PSMFC will monitor. |
01: Noise Level High. |
thumphrey |
06/08/2014 09:26 AM |
Periodic noise events. |
01: Noise Level High |
thumphrey |
06/09/2014 02:28 PM |
M4 backend service was stopped and restarted on both PCs at ~1415 6-9-2014. No activity occurred during the PRI PC's downtime. |
|
Alan |
06/17/2014 07:55 AM |
On 6-13-2014, the site was visited to attempt remedy for the intermittent file upload problems.The platform and network equipment were power cycled during this visit. Two small data gaps were created by this activity. No interrogations on the secondary PC while the primary PC was out of service.
On 6-9-2014 at ~1515 PST the secondary PC was set to NOT use passive FTP for file uploads. This was done so that the performance could be compared to the primary PC (which still uses passive FTP to upload files).
The file upload issues appear to be related to the ISP. The YKFP are working to get a technician from E&E on site to improve ISP performance. |
|
Alan |
06/17/2014 10:57 AM |
Remotely turned unique mode off on all three transceivers in order to be able to detect fallback. |
|
rclark |
07/08/2014 02:27 PM |
Had to reboot the sonicwall to get emails to work on the YIN network. All testing is completed.
Emails will be sent when power has been off or on for 3-5 mins (had to stagger the times so that all the emails didn't send at once) |
|
thumphrey |
07/08/2014 02:27 PM |
Installed a power strip inside the enclosure that houses the sonicwalls and sat modems. This was so that I could put the din rail mounted switch that ties our PC's and the PLC into the YIN network would be on battery back up. I also moved the wireless router to UPS power so I could communicate with the PLC will simulating a power outage. After I did this I was not able to get emails out on the YIN network. I can get them out on our network now but not the other...? |
|
thumphrey |
07/08/2014 02:27 PM |
New PLC program loaded at site. Testing was conducted, please disregard power outage emails as these were the result of testing. |
|
thumphrey |
09/24/2014 07:18 AM |
In response to Zymon turning RED, Restarted the .116.2 interface. The only explanation for the dropped interface is the SW for this network was unplugged for a short period of time while doing other maintenance and the interface did not reconnect. All lights are now green. Also, a fan kit was installed in the comms cabinet for cooling purposes. |
|
scottl |
01/28/2015 09:26 AM |
Errors generated were most likely caused by recent power issues |
01: Reader Set To Master.
01: Sync. Input Not Present. |
scottl |
01/29/2015 06:28 AM |
Alarm due to power outage. |
01: Reader Set To Master.
01: Sync. Input Not Present. |
rclark |
02/03/2015 08:37 AM |
This may be related to ongoing power outages. Will continue to monitor. |
01: Reader Set To Master.
01: Sync. Input Not Present. |
dlwarf |
02/03/2015 11:11 AM |
Power is restored and should clear future alarms. Will monitor. |
01: Reader Set To Master.
01: Sync. Input Not Present. |
dlwarf |
02/04/2015 08:24 AM |
A power outage occurred at ~2000 2-2-2015. UPS2 (secondary PC, xcvrs 02 & 03) ran until ~0634 2-3-2015 (10.5 hours). UPS1 (primary PC, xcvr 01) ran until ~0900 2-3-2015 (13 hours). Power was restored at ~1100 2-3-2015. No data exist to fill the remaining file gap. |
|
Alan |
02/09/2015 06:57 AM |
Alarms due to power outage. |
01: Reader Set To Master.
01: Sync. Input Not Present. |
rclark |
02/09/2015 07:03 AM |
This message was received by Site personnel on Saturday evening.
Klickitat O&M will have power restored by 10:00 am, on Monday, 2-9-15. |
|
dchase |
02/09/2015 09:32 AM |
A power outage occurred at ~22:48 2-7-2015. UPS2 (secondary PC, xcvrs 02 & 03) ran until ~09:16 2-8-2015 (10.5 hours). UPS1 (primary PC, xcvr 01) ran until ~11:41 2-8-2015 (13 hours). Power was restored at ~08:53 2-9-2015. No data exist to fill the remaining file gap. |
|
Alan |
06/04/2015 08:33 AM |
Not sure why CFF-2015-154-P-015.xml did not load. It was taken from the interrogation\failed folder, made into CFF-2015-154-X-015.xml without modification, and submitted to PTAGIS without incident. |
|
Alan |
06/08/2015 02:58 PM |
Jerod from the YIN restarted both PC's at our request. He was onsite. This was done to clear the connections. Verified both PC's came back up and verified data collection. |
|
scottl |
06/09/2015 07:08 AM |
No activity occurred on the secondary PC during the three minute data gap when the primary PC was rebooted on 6-8-2015. |
|
Alan |
07/21/2015 01:11 PM |
Disabled the primary native port for PSMFC due to no internet connection. Initiated file upload process and was successful. Still something wrong with network 1, will be one site in the AM. |
|
scottl |
08/10/2015 11:54 AM |
File CFF-2015-219-P-013.xml was rejected for unspecified reasons. The file loaded without incident when resubmitted unmodified. |
|
Alan |
09/21/2015 07:16 AM |
Edited Zymon to reflect the new WAN address for the new HughsNet Sat. Service |
|
scottl |
09/22/2015 01:06 PM |
PSMFC was onsite today to remove the ability to remotely cycle power to the Yakama Nations equipment on site, per Mike Babcock. Control Relay's 4,6,7&8 were removed and a jumper wire was installed around the relay socket. |
|
scottl |
09/22/2015 01:06 PM |
Re-established the internet connection and VPN tunnel. Sent file from PC1 and 2 without issues. Internet connection is very good, substantially better than the previous.
Inventory check list was also completed. Transceivers were checked for tune, UPS's were checked for functionality. No problems to report. |
|
scottl |
09/23/2015 08:14 AM |
PSMFC has removed all remote circuit toggle abilities to the YKFP equipment that had previously existed between the YKFP and PSMFC networks also described in a subsequent PIT event. This action occurred on 9/22/2015 with the YKFP's complete knowledge and agreement. The two networks are completely separated with no interaction.
The duration of the data upload outage ranged from 7/21/2015 to 9/22/2015 This was caused by a d-funked SAT provider. System is back online and full access and connectivity has been restored. |
|
scottl |
09/23/2015 10:58 AM |
The facility was shut down due to forest fire from 8-28-2015 to 9-9-2015. File CFF-2015-240-X-023.xml was created from secondary data and was submitted to help fill the gap. No further data exist to fill the remaining gap. |
|
Alan |
12/10/2015 04:25 PM |
Alarms caused by power outage. |
01: Reader Set To Master.
01: Sync. Input Not Present. |
rclark |
12/14/2015 08:51 AM |
A power outage occurred 12-9-2015 at ~2000 PST. The transceivers and data collection platform remained on UPS power until 12-10-2015 at 0712 PST. The primary PC and xcvr 01 continued on UPS power until 12-10-2015 at 0942 PST. Power was restored on 12-10-2015 at 1116 PST. No data exist to fill the remaining 94 minute gap. |
|
Alan |
02/27/2016 09:35 AM |
Noted |
01: Noise Level High |
scottl |
02/27/2016 09:35 AM |
Noted |
01: Noise Level High |
scottl |
03/08/2016 07:05 AM |
High noise caused by noise event of unknown origin. Noise levels are now back to normal. |
01: Noise Level High.
01: Virtual Tag Test Failed.
02: Noise Level High.
02: Virtual Tag Test Failed.
03: Noise Level High.
03: Virtual Tag Test Failed. |
rclark |
03/17/2016 06:57 AM |
Noise event due to electrical work. |
01: Noise Level High.
02: Noise Level High.
03: Noise Level High. |
rclark |
05/18/2016 01:51 PM |
The DCP upgrade has been complete and is on the ptagisdcp domain. UPS2 is now on the 115 network and is sending mail. Replaced sonicwall w1th TZ300 , had to drill and tap back panel in network cabinet. Removed network drops for the 116 PLC from the red lion switch. This 116 network has been defunked since the separation with the YIN. PLC could be readdressed to the 115 subnet and again be accessible. Cleaned up the PSMFc side of the network cabinet as it is a cobbled mess on the YIN side. No problems to report. |
|
scottl |
05/18/2016 02:00 PM |
File CFF-2016-139-X-012 was created from secondary data to fill a gap caused by platform maintenance. |
|
Alan |
05/24/2016 08:20 AM |
The primary config was edited to create 60 minute files. The new config was uploaded to PTAGIS. |
|
Alan |
07/27/2016 07:30 AM |
Changed alerting to repeat once per hour until problem is cleared rather than the default every two minutes. |
|
scottl |
05/05/2017 07:02 AM |
Short noise event. Noted. |
01: Noise Level High.
02: Noise Level High.
03: Noise Level High. |
rclark |
07/14/2017 10:11 AM |
4 instances, will continue to monitor. |
03: Noise Level High.
03: Virtual Tag Test Failed. |
Alan |
07/30/2017 09:21 AM |
This noise may require a site visit as it has become more appearent, will monitor |
01: Noise Level High.
01: Virtual Tag Test Failed. |
scottl |
09/25/2017 07:25 AM |
High noise event noted. |
01: Noise Level High.
01: Virtual Tag Test Failed.
02: Virtual Tag Test Failed.
03: Noise Level High.
03: Virtual Tag Test Failed. |
Mleonard |
09/26/2017 07:35 AM |
High noise noted |
01: Noise Level High.
01: Virtual Tag Test Failed.
02: Virtual Tag Test Failed.
03: Noise Level High.
03: Virtual Tag Test Failed. |
Mleonard |
09/27/2017 02:12 PM |
High noise note. Will investigate next site visit |
01: Noise Level High.
01: Reader Set To Master.
01: Sync. Input Not Present.
01: Virtual Tag Test Failed.
02: Noise Level High.
02: Reader Set To Master.
02: Sync. Input Not Present.
02: Virtual Tag Test Failed.
03: Noise Level High.
03: Virtual Tag Test Failed. |
Mleonard |
09/27/2017 03:49 PM |
Examination of secondary data shows that no detections occurred during the 15 minute gap on 9-27-2017.
|
|
Alan |
09/27/2017 04:02 PM |
PSMFC was on site today to perform the BPA inventory of equipment. The PC's were hard booted as per requested. The noise levels on the readers were elevated to almost 60%, several lights were turned off to see if that was the source, no source was discovered. I was unable to get into the tank building or downstairs. PSMFC will talk with YIN mechanic to see if a joint visit needs to be scheduled to find source. |
|
dchase |
09/28/2017 08:03 AM |
High noise levels are more likely caused by high noise evens that are currently being discussed. |
01: Noise Level High.
01: Reader Set To Master.
01: Sync. Input Not Present.
01: Virtual Tag Test Failed.
02: Reader Set To Master.
02: Sync. Input Not Present.
02: Virtual Tag Test Failed.
03: Noise Level High.
03: Virtual Tag Test Failed. |
scottl |
09/29/2017 07:11 AM |
Sync alarms caused by remote diagnostics. Transceivers are in a high noise state that will be addressed on the next site visit. |
01: Noise Level High.
01: Reader Set To Master.
01: Sync. Input Not Present.
01: Virtual Tag Test Failed.
02: Reader Set To Master.
02: Sync. Input Not Present.
02: Virtual Tag Test Failed.
03: Noise Level High.
03: Virtual Tag Test Failed. |
rclark |
09/30/2017 08:21 AM |
High noise to be investigated next week. |
01: Noise Level High.
01: Reader Set To Master.
01: Sync. Input Not Present.
01: Virtual Tag Test Failed.
02: Reader Set To Master.
02: Sync. Input Not Present.
02: Virtual Tag Test Failed.
03: Noise Level High.
03: Virtual Tag Test Failed. |
rclark |
10/01/2017 08:15 AM |
High noise to be investigated on Tuesday. |
01: Noise Level High.
01: Reader Set To Master.
01: Sync. Input Not Present.
01: Virtual Tag Test Failed.
02: Reader Set To Master.
02: Sync. Input Not Present.
02: Virtual Tag Test Failed.
03: Noise Level High.
03: Virtual Tag Test Failed. |
rclark |
10/02/2017 07:06 AM |
Site visit scheduled for Tuesday, 10/2. |
01: Noise Level High.
01: Reader Set To Master.
01: Sync. Input Not Present.
01: Virtual Tag Test Failed.
02: Noise Level High.
02: Reader Set To Master.
02: Sync. Input Not Present.
02: Virtual Tag Test Failed.
03: Noise Level High.
03: Virtual Tag Test Failed. |
scottl |
10/03/2017 07:09 AM |
Noise events are being addressed today. |
01: Noise Level High.
01: Virtual Tag Test Failed.
02: Noise Level High.
02: Virtual Tag Test Failed.
03: Noise Level High.
03: Virtual Tag Test Failed. |
scottl |
10/03/2017 02:45 PM |
File CFF-2017-276-P-011 had an invalid xml character. The offending character was removed, CFF-2017-276-X-011 was submitted. No detections were affected.
|
|
Alan |
10/04/2017 07:11 AM |
The outside light on the tank bldg was the source of interference and was disconnected from the lighting circuit. Light will not be replaced until next year per YIN |
01: Noise Level High.
01: Virtual Tag Test Failed.
02: Reader Set To Master.
02: Sync. Input Not Present.
02: Virtual Tag Test Failed.
03: Noise Level High.
03: Virtual Tag Test Failed. |
scottl |
12/27/2017 02:15 PM |
Both PCs were rebooted at ~1110 PST 12-27-2017. Examination of secondary data shows that no detections occurred during the time the primary PC was out of data collection. |
|
Alan |
02/22/2018 11:22 AM |
Could not find an issue with B2, all other transceivers functioning normally. Replaced lithium batteries , erased buffers, cleared M4 alerts. no problems to report. |
|
scottl |
02/22/2018 11:22 AM |
YIN personnel report several alarms and multiple faults. Jerod to keep us informed of the situation |
|
scottl |
02/23/2018 11:00 AM |
A power outage occurred 2-22-2018, the DCP and transceivers remained on UPS power until ~1900. Power was restored 2-23-2018 at ~0915. No data exist to fill the 859 minute gap in data. |
|
Alan |
10/17/2018 10:41 AM |
Short duration noise event. |
01: Noise Level High |
Alan |
12/12/2018 07:39 AM |
For unknown reasons, file CFF-2018-345-P-017 had a noise report from xcvr 03 that occurred an hour prior to the file opening. That noise report was deleted, file CFF-2018-345-X-017 was submitted. |
|
Alan |