EVENT LOG FOR SITE B2J Most recent activity to be listed first. First line of each entry lists date, time & who is reporting. Subsequent lines of each entry aligned under Time column. Do not use tabs, only spaces. Date Time Who What ---------- ----- -------- ----------------------------------------------- 11/09/2001 12:07 Dave Modified site entry in etc/sites to disable check for data receipt. 11/09/2001 10:32 Scott_Li Data collection computers were shutdown for the season at 1030hrs Nov. 9,01 10/31/2001 15:49 Dean_Bal Bonneville JBS was switched to bypass at 0700 on 31 October. Sampling and PIT tag detections have concluded for the 2001 season. 10/29/2001 14:40 Dave_Mar All Multimon installations reacted to the end of Daylight Savings Time, even though the DST option was deselected in the Win98 Clock dialog box. All sites running Multimon closed their data files at 2:00AM on 10/28 (after generating four time stamps for 02:00:00), and have been generating files on an accelerated 3-hr schedule ever since (2:00, 5:00, 8:00, and 11:00 AM and PM). This occurred with all Multimon V7.04 sites, and with the V8.0 installation at MCJ. PTTP Uploader was also affected by the OS time shift, and has also been sending files one hour earlier than expected since 2:00AM. It's obvious that Win98 is "adjusting" the internal clock time even though, by deselecting the DST check box, we've told the OS to ignore displaying that change. We might try "fixing" this behavior by using the "Arizona" time zone setting, or modify the timers to use display times rather than the OS' internal timers. 10/23/2001 10:29 Carter_S -------- Original Message -------- Subject: Re: Problems with Bonneville Int Site Connections Date: Tue, 23 Oct 2001 09:49:27 -0700 From: "Todd Kaehler" Organization: PSMFC To: Dave Marvin CC: Carter Stein References: <3BD59E16.2787FB@ptagis.org> Sprint called an said they were having problems with the T1 connection. I gave them to OK to take it down for diag. testing. --Todd Dave Marvin wrote: >We haven't received data files from BWL, B2A, or B2J since yesterday >afternoon. I've spent the last hour trying to pcA into the three >sites. All three of them connect, as indicated by opening up in a >remote control window, but never paint the screen with the remote >desktop. Meanwhile, Big Brother shows intermittent connection problems >with those three servers since around 1600h yesterday. Anyone got any >ideas? > 10/22/2001 10:03 Dave SMP personnel report that the PIT tag override of the sample gate was deactivated on October 17, when the sample rate dropped from 25% to 16.67%. No fish are currently being diverted into the SbyC tanks, as so there is no net effect on facility operations or tag disposition. 10/08/2001 16:32 Darren_C I have been trying to get B2J to send files for about 1.5hrs. I get so far in the process and then PTTP appears to hang. The connection is acting very choppy. B2A has finally sent its files but after viewing the history of PTTP it appears that it to was trying to send its files but was unsuccessful until about 3p.m. BWL I am having trouble getting into I will call Todd K. in the morning and find out if we are having router problems or what is going on. 09/25/2001 11:03 Carter_S B2J was locked up this AM. I called Dean Ballinger and had him re-boot the utility computer. I noticed B2J util was hung through the BB interface. --Carter 09/21/2001 08:53 darren@p 91:ReaderReset,91:TestTagFailure,92:InputSyncDetect,92:InputSyncLost,92:Re aderReset,93:InputSyncDetect,93:InputSyncLost,93:ReaderReset: These alarms were caused by PTOC personnel performing transceiver replacement. Transceiver 92 was replaced. 09/21/2001 08:41 Darren_C On Tuesday (9-18-01) transceiver 92 was replaced. It had been performing rather oddly in the past 5 days. It had numerous reader resets and had shutdown for two days and then came back to life. This unit will be checked out in the lab before being sent back to Destron-Fearing. 09/18/2001 06:59 darren@p 92:InputSyncDetect,92:ReaderReset,93:InputSyncDetect,93:InputSyncLost: This alarm will be addressed today. The transceiver may be replaced. It appears to be an intermitten problem. I will be able to elaborate on the condition after site is visited. 09/17/2001 08:04 darren@p 92:InputSyncDetect,92:ReaderReset,93:InputSyncDetect: Alarm was caused by transceiver 92 coming back to "life" I will investigate this problem. We will be onsite tomorrow for maintenance. We suspect a possible fiber optic problem. 09/14/2001 10:35 Darren_C I researched the RAW files and discovered that coil 92 was last heard from at 22:00 on the 12th. At 23:30 the status report for coil 93 showed no sync input. I believe that the coil has failed internally. We will investigate and replace the transceiver if necessary on tuesday the 18th due to the High efficiency of the monitor and low volume of fish passing through we will still maintain a high level of detection until then. We will also be at the AFF performing maintenance on coil 4B at the request of NMFS and maintenance on coil 02 at B2A. The coils around 02 are performing well and it also can sustain until we arrive. 09/14/2001 08:33 dlwarf@p 92:InputSyncDetect,92:ReaderReset,93:InputSyncDetect,93:InputSyncLost: Ongoing problem. 09/12/2001 07:57 darren@p 92:InputSyncDetect,92:ReaderReset,93:InputSyncDetect,93:InputSyncLost: This is an ongoing situation that is being addressed by PTOC personnel and Destron-Fearing. 09/11/2001 11:05 darren@p 92:InputSyncDetect,92:ReaderReset,93:InputSyncDetect,93:InputSyncLost: This is an ongoing situation that is being addressed by PTOC personnel and Destron-Fearing. 09/09/2001 21:31 dlwarf@p 92:InputSyncDetect,92:ReaderReset,93:InputSyncDetect,93:InputSyncLost: Ongoing problem. 09/06/2001 08:07 darren@p 92:InputSyncDetect,92:ReaderReset,93:InputSyncDetect,93:InputSyncLost: Alarm was generated by a random reader reset.This situation is being addressed by PTOC personnel and Destron-Fearing. 09/03/2001 16:18 Dean_Bal The Juvenile Bypass System was placed in bypass mode on 28 August from 0700 to 1600 to work on the PDS. 08/31/2001 08:09 darren@p 92:InputSyncDetect,92:ReaderReset,93:InputSyncDetect,93:InputSyncLost: Alarms on coil 93 were caused by the random reader reset from coil 92. This is still being investigated by PTOC personnel and Destron-Fearing. 08/27/2001 08:05 darren@p 92:InputSyncDetect,92:ReaderReset,93:InputSyncDetect,93:InputSyncLost: Alarm was generated by a random reader reset. This is an ongoing situation that is being addressed by PTOC personnel and Destron-Fearing. 08/22/2001 21:50 Darren_C I went to the site because of the faulty coil 00. When I arrived I discovered that the exciter was fine but that the loop tuner was bad it was replaced and the timer tag was fired with no problem. We will monitor the site for activity and efficiency. I also stopped in at B2J and checked the platform. All was well. 07/13/2001 08:03 darren@p 92:InputSyncDetect,92:ReaderReset,93:InputSyncDetect,93:InputSyncLost: Alarm was generated by a random reader reset. This has been an ongoing event and is being checked into by Destron-Fearing and PTOC personnel. We will watch this monitor for further alarms. 07/12/2001 12:00 Dave I replaced the Multimon map file "b2j01191.map" with "b2j01193.map". The new map file replaces apparent DUs 19 and 20 (physical DU's 18-19) with apparent DUs 19-22. The change was necessary to correct a problem with the previous implementation. DUs 19 and 20 in "b2j01191.map" used SP6 and addressed three distinct date ranges each (for a total of six date ranges). Within each DU I referenced Action Codes 87 and 88, with "Seasonal Max" values of 21 and 20, respectively. I anticipated that this would result in the collection of up to 41 fish per sample range. I learned, after implementing this map file, that the "Seasonal Max" \ value is used to truncate tag diversion regardless of the active range. In this regard, the "Seasonal Max" could be construed as a "DU Max". I corrected the problem by separating the two Action Codes, and two sets of three date ranges, into distinct DUs, and using the "Range Max" value to apply the same, repeating AC limit to all sample date ranges in each DU. Thus, in "b2j01193.map", DU 19 still specifies SP6 and references three separate date ranges, but it now includes a single Action Code definition for AC87. In that AC definition, I've reset the "Seasonal Max" value to 0, and now specify a "Range Max" value of 21 in the DU header. I've made the same modifications to the second set of three sample date ranges in DU 20. I copied the new DUs 19 and 20 to DUs 21 and 22, replacing the references to AC87 with AC88, and setting the "Range Max" value to 20. I replaced the map files on the B2J PCs prior to the first sample, which started at 10:00 (PST) today (7/12/01). I cleared the old map from active memory by using the "Load Blank Map-of-Site file" option before loading the new map file. I did this to (hopefully) prevent any mis-assignments of counter limits caused by the re-assignment of DU definitions. However, while editting and implementing the new map file on my test bed, I noticed that Multimon generated a message similar to "Counters don't match map definitions" when I first loaded Multimon from DOS; I assume this resulted from the presence of legacy values in the "default.cfg" file for the old definitions of AC88 in DUs 19 and 20. If the opportunity presents itself, I recommend completely resetting the Multimon environment, including deleting and regenerating "default.cfg", to ensure the proper implementation of the U of I fall chinook SbyC collection. (BTW, I avoided this potential problem atGRJ by using six DUs, each one specifying a single date range and two ACs.) 07/12/2001 08:01 darren@p 92:InputSyncDetect,92:ReaderReset,93:InputSyncDetect,93:InputSyncLost: Alarm was generated by a random reader reset. This action is being addressed by PTOC personnel and Destron-Fearing. 07/11/2001 14:10 Dave I updated Multimon with "grj01191.db" and "grj01191.map". The database and map files are configured to divert Lyon's Ferry fall chinook (Action Codes #87 and #88) to the right at the SbyC gate (and then to the west SbyC sample tank). U of I will collect fish for physiological monitoring. The current schedule is set to collect a maximum of 41 fish over a two-day period each week, starting tomorrow (July 12) and reiterating six times, through September 17. I also confirmed the clocks on both PCs were within 10 seconds of each other and less than 2:45 different than "real" time, well within our maintenance standard of 30 seconds and five minutes, respectively. 07/10/2001 08:01 Darren_C Repair was done to the Sample Monitor. The transceiver on coil 51 was discovered to be down. No timer tags or detections were reported on Monday. When I arrived I found that the RAM Module for the +VE was only able to supply 1.1v this failure has been found in the past. The transceiver was replaced and the monitor was tested. The rest of the site was checked and no problems were found. 07/08/2001 10:46 scottl Cumulative efficiency report indicates that coil 51 , Sample room monitor is not reading tags. Will respond 7/9/01 for repair 07/08/2001 10:41 scottl@p 93:InputSyncDetect,93:InputSyncLost: 93:InputSyncDetect,93:InputSyncLost: ongoing problem 07/08/2001 10:41 scottl@p 93:InputSyncDetect,93:InputSyncLost: ongoing problem 07/08/2001 10:40 scottl@p 92:InputSyncDetect,92:ReaderReset: ongoing problem 06/27/2001 07:54 darren@p F1:ReaderReset,F1:TestTagFailure,F2:InputSyncDetect,F2:InputSyncLost: Alarms were generated by PTOC personnel performing maintenance on transceiver F1. The battery was replaced to solve the "low lithium" alarm then the transceiver was reset to clear the alarm. 06/26/2001 11:55 darren@p F1:LowLithium: Alarm will be taken care of today. Battery will be replaced. 06/25/2001 07:51 darren@p F1:LowLithium: This alarm will be taken care of on the next site visit. 06/22/2001 08:12 darren@p F1:LowLithium: Alarm will be taken care of on the next site visit. 06/21/2001 07:56 darren@p F1:LowLithium: Alarm will be taken care of on the next site visit. 06/20/2001 07:52 darren@p F1:LowLithium: Alarm will be taken care of on next site visit. 06/19/2001 07:53 darren@p F1:LowLithium: Alarm will be taken care of on the next visit. We should be there by next week. 06/17/2001 09:36 dlwarf@p F1:LowLithium: To be replaced on the next visit. 06/15/2001 07:47 darren@p F1:LowLithium: This alarm will be addressed on the next site visit. We will change the battery. 06/14/2001 07:51 darren@p F1:LowLithium: This alarm will be addressed on the next site visit. The battery will be replaced. 06/12/2001 17:09 Dave Dean Ballinger reports: As of 1630 today, repairs have been made to the 3-way rotating gate and rotational capability to the S by C east tank has been fully restored. The oiler to the "east" piston was frozen, thus, rotation was sluggish or non-existent to the SbyC east tank. This problem has been corrected and the 3-way gate appears fully operational. 06/12/2001 12:55 scottl General maintenence check found that the 3 way gate was not rotating to the East postion. Tried to free the gate by multiple manual rotations but no luck. Informed Dean Ballinger to initiate the repair order with the COE via Jennifer Sturgill. Dean will follow up with the repair and inform when gate has been repaired. 06/10/2001 08:24 darren@p 92:InputSyncDetect,92:ReaderReset,93:InputSyncDetect,93:InputSyncLost: Alarm was generated by a random reader reset. This problem is being addressed by PTOC personnel and Destron-Fearing 06/09/2001 06:23 darren@p 92:InputSyncDetect,92:ReaderReset,93:InputSyncDetect,93:InputSyncLost: Alarm was generated by a random reader reset. This problem is being addressed by PTOC personnel and Destron-Fearing. 05/31/2001 07:53 darren@p 92:InputSyncDetect,92:ReaderReset,93:InputSyncDetect,93:InputSyncLost: Alarm was generated by a random reader reset. This problem is being addressed by PTOC personnel and Destron-Fearing. 05/30/2001 13:16 scottl General maintenence check found no problems to report. SByC diversion test on the 3-way gate was 100 percent successful diverting tags to the east and west tanks. 05/23/2001 15:21 Dave I updated the B2J Multimon map file to "b2j01143.map". This version contains two additional two-day sample periods for the UofI study. I also verified that the inter- and intra-site timestamps on PC1 and PC2 were within PTOC tolerance levels. 05/22/2001 08:04 darren@p 92:InputSyncDetect,92:ReaderReset,93:InputSyncDetect,93:InputSyncLost: Alarm was generated by a random reader reset. This problem is being addressed by PTOC personnel and Destron-Fearing. 05/17/2001 08:08 Darren_C All sites were visited for site maintenance. No problems were found. Due to the high volume of fish at the juvenile facility no test sticks were thrown. The sample rate was at 1% (36 seconds an hour). All computers were labeled with PTOC labels. 05/10/2001 15:20 Dave The first SbyC subsample of CSS fish for U of I began at 10:00am (PST) today. 05/09/2001 20:44 Don_Warf Stick tested both directions of the 3 way gate with 100% efficiency. Looked for possible routes for tagged fish to end up in the wrong SbyC tank. No apparent routes were found. Flush water below the gate does not appear to be the problem. Nor does the flipper gate. Site personnel report that the site was watered down to repair the cracked two way gate tube. The site was reported to be down from 9:00 to 13:00 for the repairs. 05/07/2001 11:27 Dave I updated the configuration at B2J this morning. I uploaded new map and database files "b2j01124.map" and "b2j01124.db", respectively. These files contain AC records and instructions for the diversion of "lefty" stick tags (AC 253) at the three-way rotational gate. The map file also contains DU definitions for 10 U of I subsamples of CSS file, beginning on May 10. Since these subsamples begin and end at 10:00am (PST), and I was unable to use SP7 to explicitly start and stop those subsamples, I coded all the subsamples using SP6, and modified the hardware files to reset the counters (and DU definitions) at 10:00). The new hardware files on B2J- PC1 and PC2 are named "b2j01127.hw1" and "b2j01127.hw2", respectively. In addition to changing the counter reset time from 00:00 in both hardware files, I also changed the remote data path from "\NUL" to "\NULL" in "b2j01127.hw1", to make this value consistent with all the other remote paths referenced in the B2J-PC1 and PC2 hardware files. As is my SOP when I load a significantly different map file (and any time I modify the hardware file), I deleted the "default.cfg" file and started the Multimon application from scratch, thereby clearing all memory buffers and counters. This means that the cumulative tag counters were reset to zero. Prior to that reset, I observed that 459 unique fish from the B2JOPE study (AC 81) had been observed at B2J to date. This value should be added to any further cumulative queries for that study. None of the CSS fish of interest (ACs 11, 14, or 15) had been observed at B2J prior to the counter reset. The B2J configuration update went smoothly, with no disruption in the detection of tagged fish. 05/02/2001 07:58 Scott_Li The three way gate open time is set for one second. 05/01/2001 10:23 Scott_Li Maintenance check found that the Three-way gate was not rotating to the left to divert fish into the East SBC tank. Appeared the may be binding or needed an adjustment. Jim Simmonson was due to check the gate late yesterday and repair whatever was causing the malfunction. It was also noticed that just upstream from the Three-way gate, at the Flange joint that connects the gate and the flume section, a extreme amount of silicone was used to stop a leak and was causing a very abrupt edge and disrupting the flow of water thru the gate. This is a bad thing and need to be corrected... This condition will hurt the effectivness of the diversion gate and cause poor diversion efficiency... Modified the DB and map files to divert 20 stick tags to the East SBC tank. 04/28/2001 19:34 Don_Warf Due to low diversion efficiency I stepped the site Bio through setting the open time of the three way gate. The open time was increased from .8 to 1.6 sec. This was set using the wet lab Dtam. 04/27/2001 14:29 Dave There's a problem with the 3-way (SbyC) gate at B2J. SbyC for the NMFS BO2-OPE study was initiated on Tuesday, April 24, with an Action Code of "81". The first fish from that study were detected (and diverted into the East SbyC Tank) just before midnight on the 24th. Yesterday, Dean Ballinger attempted to alerted me by e-mail that there were fish in BOTH SbyC tanks. I talked with Dean today and reviewed the raw data files and verified that two PIT-tagged fish had indeed been detected on coils F1-F2 en route to the West SbyC Tank. Dean estimates that there are about 10 fish total in the West SbyC Tank. I verified that the MultiMon db and map files are configured correctly, and that the software is attempting to divert all "81" fish into the East SbyC Tank. I contacted Don Warf and suggested a mechanical problem with the rotational gate itself that might be "leaking" fish into the West SbyC Tank flume. Subsequent to that conversation, I reviewed the SBCA reports for B2J and was surprised to see only 39 of the B02-OPE fish tracked so far. I have enumerated a total of 92 unique tags with "Divert_81" actions at the 3-way gate, and was surprised to see that 37 of those (40%) were NOT routed to the East SbyC Tank, but were instead detected downstream on coils 81 and 82, as the fish returned to the river bypass flume. I then enumerated the unique tags from the BO2-OPE study and their disposition, with the following results. SbyC Gate (coils A1-A4): 92 tags, all with DIVERT_81 actions. East SbyC Tank (E1-E2): 51 tags. West SbyC Tank (F1-F2): 2 tags. SMP Sample Tank (51-52): 2 tags (the sample overrides SbyC, so this is expected). River Exit (81,82): 37 tags. The fish mis-routed to the West SbyC Tank were mixed in with fix that were correctly routed to the East SbyC Tank. The "missed" fish that returned to the river were similarly mixed with fix that were correctly diverted to the East SbyC Tank. It appears to me that the routing problem is intermittent and that the 3-way gate did not suddenly stop working correctly. Perhaps it's sticking. 04/25/2001 10:03 Scott_Li Re-connected phone lines to both computers. 04/25/2001 09:05 Scott_Li General maintenence check found no problems to report. 10 stick tags were thrown on the separator coils and successfully diverted thru the SByC leg. 04/23/2001 17:11 Dave I replaced the "b2j-gen.db" file with the expanded list of ISO stick tags. I also loaded a "b2j-gen-all.exe" self extracting archive of all of the generic Multimon configuration files for this site (b2j-gen.db, b2j- gen.map, b2j-gen.hw1, b2j-gen.hw2, and ignor.tag). Now that they're no longer in use, I deleted the "b2j-base.db" and "b2j-base.map" files; the "gen" versions of these files are now appropriate to use in the absence of any Separation by Code. 04/23/2001 17:05 Dave I updated the map and db files at B2J with "b2j01113.map" and "b2j01113.db", respectively. This update accommodates the NMFS/COE BO2 Orifice Passage Efficiency (OPE) study, led by Bruce Monk (NMFS), that starts tomorrow evening. All BO2OPE fish (AC 81) encountered at the 3-way gate will be diverted to the right and routed to the East SbyC tank for condition evaluation. According to Bruce, the study will last through May. 04/19/2001 09:10 Darren_C A General Maintenance Check was performed on both sites. At B2J 20 sticks were passed through the separator monitor and diverted into the West tank. The results are as follows: A1-20 A2-20 A3-20 A4-20 F1-20 F2-20. 04/13/2001 09:00 Dave SMP personnel report: "Modifications and repairs to the sample distribution flumes and SDS have been completed slightly ahead of schedule. Sample collections and PIT tag detection activities resumed at 1630 [on] 12 April." 04/10/2001 11:05 Darren_C Here are the results of the stick evaluation of the coils performed on 3-30-01. SITE: B2J COIL EVALUATED DETECTED EFFICIENCY A1 20 20 100.000% A2 20 20 100.000% A3 20 20 100.000% A4 20 20 100.000% MONITOR COIL EVALUATED DETECTED EFFICIENCY 51 20 20 100.000% 52 20 20 100.000% MONITOR COIL EVALUATED DETECTED EFFICIENCY E1 20 20 100.000% E2 20 20 100.000% MONITOR COIL EVALUATED DETECTED EFFICIENCY F1 20 20 100.000% F2 20 20 100.000% MONITOR COIL EVALUATED DETECTED EFFICIENCY *91 NO TEST *92 NO TEST *93 NO TEST MONITOR * These coils were not tested because of the head tanks in which sticks must be thrown. PTOC personnel was unsure of lodging sticks in flume. 04/03/2001 08:20 Darren_C The program ABOUT TIME was removed from both PC1 and PC2. The system clock was adjusted for PST. Sticks were thrown to check timing of the 3way gate and the DTAM was adjusted. 03/30/2001 06:06 darren@p 92:InputSyncDetect,92:ReaderReset,93:InputSyncDetect,93:InputSyncLost: Alarm was caused by a random reader reset. This problem is being addressed by PTOC personnel and Destron-Fearing. 03/23/2001 08:57 Darren_C On 3-22-01 the NEW sample monitor was occupied with 2 ISO coils and transceivers. The platform was changed to recognize the new coil I.D.'s 51,52. Communications were tested and established. 03/20/2001 15:54 Dave With Darren on-site today, we retrieved PC2 data files for the period from midnight-0900h on March 14. The corresponding PC1 files had been deleted (overwritten) due to a restart of the PC1 installation following the removal of the DEFAULT.CFG file. The Multimon installation on PC2 was active and current when Darren accessed the machine, but Windows suffered a GPF during this process. On reboot, the installation appeared normal and stable. 03/12/2001 17:30 Dave SMP personnel report that the BO2 Juvenile Fish Facility (JFF) was switched from primary to secondary bypass at 5:30pm on March 12, 2001. Sampling and PIT tag detection occur only when the facility is in secondary bypass. This will be the normal mode of operation until the end of October. 03/09/2001 15:08 Scott_Li Generic Hardware config files for these sites have been uploaded to the PSMFC FTP server for archival. These files contain the new sample messaging enhancement and are the lastest revision and should be the last. 03/09/2001 11:25 Scott_Li The pittag detection equipment has been restored and is operational. Interrogation platform has also been reinstalled and is uploading data to ptagis. The new sample monitor was not completed and integrated into the system due to delay's by the mechanical contractor. The electrical and fiber optics to this monitor are installed and waiting for the mechanics to be completed. Monitor should be completely functional in the next two weeks or so... 03/08/2001 18:02 Don_Warf Loaded modified PLC program to use messaging. 03/08/2001 08:34 scottl@p 91:ReaderReset,92:InputSyncDetect,92:ReaderReset,93:InputSyncDetect,93:Inp utSyncLost,93:ReaderReset,A1:ReaderReset,A2:InputSyncDetect,A2:ReaderReset ,A2:TestTagFailure,A3:InputSyncDetect,A3:ReaderReset,A3:TestTagFailure: Alarms were generated by PTOC personnel while doing installation of all monitoring equiptment. there will be more alarms created today (3-8-01). 01/05/2001 18:00 Dave Manually created new event log for 2001. ####End of Event Log for B2J###