bu_cms_history/DCCSoftware

SiteMap (Historical BU CMS wiki main page)

DCC Testing Blog

2009-03-10 (hazen, BU)

Reviving work on new DCC expert view. See NewDccExpertView page on TWiki.

2008-09-23 (hazen, BU)

Install 7_0_0

Installing 7_0_0.

Fix 6_6_2

Fixed a few things, now it works.

In cfgScript, change:

  ============
  Global {
 FirmwareDatabase = "mysql://WebUser@cms2/CFGBRICKDB?PASSWORD=hcalweb"

 if (FUNCTION == "MonLogger") {
    ConditionsDatabase="ascii:///home/daqowner/monlog"
 }

 }
 MonLogger {
  CollectorAccess=""
  CollectorRef="cms1.bu.edu:40000"
 }
 =========
 to
 =========
 Global {
  FirmwareDatabase = "mysql://WebUser@cms2/CFGBRICKDB?PASSWORD=hcalweb"
 }

Per Arno's suggestion, I changed the line in /opt/share/hcaljc/conf/xdaqd.conf:

 < XDAQ_HOSTNAME=cms1.bu.edu
 ---
 > XDAQ_HOSTNAME=localhost

Clean up log files to xdaqd can start/restart:

 1) xdaqd stop
 2) rm log files  in /opt/xdaq/log
 3) xdaqd start

This was needed too:

  wget http://pcephc356.cern.ch/hcalsw/installBase.sh
  bash installBase.sh

(had to do the second as root).

2008-09-22 (hazen, BU)

Try to make it work. According to Jeremy:

   "You should no longer have the HyperMonVis or MonLogger in your DuckCAD configurations.  
   As of ~6_4_3, these are services run by the job control script.  You'll need to re-install the job control setup.

   wget http://pcephc356.cern.ch/hcalsw/installBase.sh
   bash installBase.sh

Try this, but fail with lots of permission errors. Re-run with sudo.

2008-09-02 (hazen, BU)

Install 6_6_2.

  daqowner> rm dist
  daqowner> mkdir dist
  daqowner> wget http://pcephc356.cern.ch/hcalsw/release/installDAQ_6_6_2.perl
  daqowner> emacs installDAQ_6_6_2.perl   (uncomment anonymous CVS line)
  daqowner> perl installDAQ_6_6_2.perl --mode=teststand | tee install_6_6_2_log.txt

Notice a bunch of errors, but ingore them... eventually it finishes. Re-run install:

  daqowner> perl installDAQ_6_6_2.perl --mode=teststand | tee install_6_6_2_log_try2.txt

Now DCCdiagnose at least runs.

2008-06-02 (hazen, BU)

Updated to HCAL 6_3_0 release.

  install 6_2_4 (took a few tries, creating /tmp/daqowner, delete dist/sw etc)
  install 6_3_0 on top per Jeremy's advice

  install 'ownsource' tree with packages=all
  cd hcal; make

Took two tries, AOK now. Need to merge in changes from my 6_2_1 tree.

First, try running DAQ with out-of-the-box 6_3_0. AOK, except that first EvN from both HTRs was wrong, otherwise AOK.

Files for CVS:

  DCCExpertViewItems2c.csv
  DCC_Logicboard....dat
  hcalDCCItem.cc
  DCCdiagnose.cc
  hcalDCCManager.cc
  DCC.hh
  DCC.cc

2008-06-02 (hazen, BU)

Testing DCC v0x2c2b with SLink readout. Running at 49kHz with Slink readout. See a regular pattern of EvN mismatch which looks like a DCC firmware problem.

http://joule.bu.edu/~hazen/CMS/results_02Jun08_1.zip

Using special HTR firmware:

https://ehazen.web.cern.ch/ehazen/CountingHouse/HTR/firmware/HTR_62/DEBUG_v0/

(version 10064. See 'Notes.txt' in the directory)

Cycle the power, problem goes away.

2008-05-23 (hazen, BU)

Interesting loss-of-sync with short HTR event at EvN = 0x92ed. Run at 100kHz with SLink readout but very long TTS latency of 1.6ms (delay before L1A stop when backpressure happens).

http://joule.bu.edu/~hazen/CMS/23May08_run1.zip

Now move on to v0x2c29 with "error-capture" feature.

2008-05-22 (hazen, BU)

Seemingly consistent LoS at 65kHz random triggers. Files used: 22May08.zip .

Firmware: HTR 0005A, DCC 0x2c28.

2008-05-20 (hazen, BU)

Try to reproduce Terry's high-rate problems. Run with DCC v2c28 and HTR v0005A.

Update DCCdiagnose.cc a bit to properly display 32 and 64-bit counters.

After a few minutes at 49kHz there are no errors.

2008-05-16 (hazen, BU)

Update to DCC 0x2c26 and HTR 0x62 firmware.

Files changed for CVS in ~hazen/hcal_6_2_1:

  DCC_logicboard2c.dat
  DCCExpertViewItems2c.csv

2008-05-14 (hazen, BU)

Testing with HTR 5A firmware (2 time samples), DCC 0x2c25 firmware. Run with 49kHz random rates. No errors up to 3e8 events.

2008-05-12 (hazen, BU)

Preparing to test 0x2c25 firmware. First reprogram MIP2 with v27 LRB firmware. Reprogram Xilinx with 0x2c25.

2008-05-01 (hazen, BU)

Got monitoring updates by e-mail from Jason. Files now with pending changes:

  hcalDCCMonitoring.cc (Jason)
  DCC_logicboardv4_2c.dat (Jason)  
  DCCExpertViewItems2c.csv (me)
  hcalDCCItem.cc (me)

Fix various bugs working with Jason. Test with 2 spigots active; expert view and monlogger seem to report correct data. Commit to CVS above 4 files.

-----

Working to merge in updates before disk crash. Recursive diff with ~hazen/hcal_6_1_1 reveals the following changed files:

  DCC_logicboardv4_2c.dat  (keep 6_2_0 version)
  DCC.hh  (keep 6_1_1 version)
  DCC.cc  (keep 6_1_1 version)

  hcalDCCExpertView.hh (keep 6_1_1 version)
  hcalDCCExpertView.cc (use 6_1_1 version)

AOK now, with partial new expertview. Working to finish it.

2008-04-30 (hazen, BU)

Terry has installed 6_2_1. Initialize, configure using "bu_New" configuration. Only Spigot 0 sees events (0,1 are enabled). Swap HTR cables. Now sp. 1 counts. So, for some reason the bottom FPGA in the slot 15 HTR isn't sending anything. The top guy has some wierd firmware... reprogram with 0005A and try again. Oops... the LRBs have funny firmware, too. Reprogram them back to the standard v27.

OK, now we can do runs with SLink readout (Slink to cms2).

Recompiling from ownsource area.

  perl installDAQ... --mode=teststand --ownsource=/home/hazen/hcal_6_2_1 --packages=all
  cd hcal_6_2_1/hcal
  make clean
  make install

As daqowner

  cd $XDAQ_ROOT/
  tar -czvf dist_6_2_1_orig_lib.tar.gz lib/
  cp /home/hazen/hcal_6_2_1/hcal/lib/linux/x86/libhcal* .

Try a run. AOK.

Adding updateLRBData() calls to ExpertView and monitoring

Edit hcalDCCItem.cc to add above call. Edit DCCExpertViewItems2c.csv to remove log1 and log2 items.

Looks better, but counts appear on "LRB1" line. Fixed DCCExpertViewItems2c.csv again, now it's ok.

2008-04-04 (hazen, BU)

Synopsis: Updated to 6_1_0, 6_1_1. HD crash, recovery. Re-writing hcalDCCExpertview.cc. A few changes in DCC.cc and DCC.hh.

Now DCCdiagnose.exe and other .exe crash with segfault on exit. Doesn't happen with 6_1_0 tree (??).

2008-03-19 (hazen, BU)

See Post_GRuMM_Tests .

2008-03-17 (hazen, BU)

DCCdiagnose.cc changes were merged on 2/14, so hcal_3_11_4 is current. Adding new features to control i.e. TTS output state regs nicely.

2008-02-13 (hazen, BU)

Update to 3.11.4 on cms1. Various changes in 3.11.3 tree in dist_saved_3_11_3 need to be merged. See oSLBDevelopment .

2007-12-10 (hazen, BU)

Updated files in 3.9.14 (on CMS2)

''These need to be merged to cms1 3.11 series!''

2007-12-05 (hazen, BU)

2007-12-04 (hazen, BU)

Updating to 3.9.14. Copy dist tree to dist_before_3_9_14_update. Run install script per usual. After install, had to copy

  etc/runctrl.properties
  hal/OSLB_v2.dat

from older releases. Now the command-line tools and runcontrol seem to work OK.

2007-10-25 (hazen, BU)

Fixing up things after 3.9.11 install.

Copied to dist tree and updated in CVS:

  DCC.cc, DCC.hh  (add synch control via hcalDCCConfigInfo)
  hcalDCCManager.cc  (add synch control cfgScript option)

Insufficient Karma to update this one...

  hcal/hcalTrig/hal/TTCviAddressMap.dat  (add CSR1)

So didn't update this either for now...

  DCCdiagnose.cc  (add DCC/OPEN, DCC/DUMP, DCC/SYNCH, TTC/CMD)

2007-10-24 (hazen, BU)

Changes in dist/src tree since update to 3.9.10:

  DCC.hh, DCC.cc, DCCdiagnose.cc

Preparing for 3.9.11 update:

  mv dist dist_3_9_10_before_update

Installing 3.9.11.2. Success (no build errors, etc)

2007-10-16 (hazen, BU)

I think the below changes (block address table on MB full) are in CVS. Doing a full audit of on-system code vs 3.9.9 CVS prior to update to 3.9.9.

Diff summary (hcalDCC subtree):

Committed above (I think!) and updated to 3.9.10

Diff summary (hcalHW subtree)

Updating to 3.9.7

Move ~daqowner/dist tree to dist_saved_3_9_7

mkdir dist (empty dir)

run install script

2007-09-24 (hazen, BU)

Start to add support for new mode where event builder is blocked by monitor buffer full. Edit address table for DCC in:

  /home/hazen/hcal_3_9_7/TriDAS/hcal/hcalDCC/hal/DCC_logicboardv4_2c.dat

Next we need to add CfgScript support for the new bit, and expertview support.

2007-09-14 (hazen, BU)

Committed to CVS:

And also

2007-09-03 (hazen)

TODO:

file and use default action

2007-08-28 (hazen)

Updated software to 3.9.7. New stuff which is only in Eric's source trees hcal_3_9_6 and hcal_3_9_7:

2007-07-09 (stjohn)

Update to hcal_xdaq_3_9_4, now with DCC monitoring example!

'2007-07-02' (stjohn)

Updated to xdaq_3_9_3:

 cd /home/daqowner/
 cp dist dist_not_here
 cd -rf dist
 su wget http://pcephc356.cern.ch/hcalsw/release/installDAQ_3_9_3.perl

Had to su - daqowner and then

 cd dist/src/TriDAS/hcal
 make clean; make install

Some minor complaints about hcalRBX; ignoring. Then

 cp lib/linux/x86/libhcal*.so ~/dist/lib 
 cp ~/dist_before_3_9_2_udpdate/etc/runctrl.properties ~/dist/etc/.
 sudo /etc/rc.d/init.d/xdaqjc start

And voila! runcontrol executes. Looks like there's monitoring.

'2007-06-15' (stjohn)

The jobcontrol is a xdaq process which must be running for the xdaq to work. He's the way to start it, and then evidence that it is running:

 cms2
 /tmp > sudo /etc/rc.d/init.d/xdaqjc start 
 Starting XDAQ JobControl:                                  __OK__

 cms2
 /tmp > ps auxwww | grep xdaq
 root     27075  0.4  0.5 75792 10628 pts/0   S    10:04   0:00 xdaq.exe -h localhost -p 39999 -e /etc/xdaqjc-profile.xml
 stjohn   27112  0.0  0.0  4780  676 pts/0    S    10:05   0:00 grep xdaq

-> Re-install of 3_9_2

since the TriDAS directory, all the HCAL software seems to be missing. As daqowner ooon cms2:

 rm -r dist/etc/rpmdb
 rm -r dist/src
 perl installDAQ_3_9_2.perl --mode=teststand --source
 cd src/TriDAS/hcal
 make clean; make install

That was messy. Tried again from ~/dist/src/TriDAS/hcal (which is actually /home/daqowner/dist/src/HCALdaq because of symbolic links), and this time LOTS of successful make messages.

'2007-06-11' (stjohn)

Solved the runcontrol problems with jobcontrol! Modified (corrected) the value of HCALDAQ_SW_LOC in the file /etc/xdaqjc.conf:

 JC_ROOT_ALLOWED=NO
 JC_USERS_ALLOWED=:daq:hcalpro:hcalsw
 XDAQ_ROOT=/home/daqowner/dist/src/TriDAS
 HCALDAQ_SW_LOC=/home/daqowner/dist
 XDAQ_USER=root
 XDAQ_LOG=/var/log/xdaqjc.log
 XDAQ_PORT=39999
 CORE_DIR=/tmp
 XDAQ_HOST=localhost

Then restarted the jobcontrol:

 cms2
 /tmp > sudo /etc/rc.d/init.d/xdaqjc start
 Starting XDAQ JobControl:                                  __OK__

...and then runcontrol works without a hitch, HyperDAQ, Monitoring, and everything. Just in time for the release of XDAQ 3.9.2 (for which this jobcontrol problem will not need to be fixed again, it seems). Now we just need to get the DCC monitorables to show up.

'2007-05-06' (stjohn)

Updated cms2 to 3_9_1_1 which is supposed to be the right choice monitoring devopment.

runcontrol gives the usual popup, but selecting a RunType causes the following message to appear (after a lengthy pause):

     Exception in setRunType()
     net.hep.cms.xdaqctl.XDAQException: Configure an executive at http://localhost:40000

Same behavior with a small difference when selecting ''Start XDAQ'' from the XDAQ menu:

    Exception in startXDAQ()
    net.hep.cms.xdaqctl.XDAQException: Configure an executive at http://localhost:40000

and then clicking Setup Run

    Exception in setupRun()
    net.hep.cms.xdaqctl.XDAQException: Failed to send a SOAP command, ParameterQuery, to http://localhost:40000.

Does the jobcontrol (installJC.perl) need to be reinsatlled, as J. Mans suggested? (see The release 3.9.0 email )

2007-05-06 (hazen)

Working from home. Got runcontrol almost working after fixing slot numbers. Hangs at 2 events, sometimes with "offset error" complaint.

Trying some things:

maybe needs cfgScript item now?)

2007-05-04 (hazen)

Request from Tullio for control of initial BcN after reset. Easiest is to add as a new cfgScript parameter. Will do this, but first....

Update cms1 to 3_7_9 (just can't keep up!). Save source tree as dist/src_before_..... All is well in command-line-land after a slight tweak to ~hazen/src/DCC/test/Makefile to add new lib hcalAux.

OK. The config stuff is a mess. There is a nice hcalDCCConfigInfo class in hcalDCCManager. Why not pass one of these to prepareForRun() instead of the arbitrary handful of stuff passed now?

This is fine, but will require enough fiddling that it would be best to make sure the runcontrol works first.

2007-05-03 (hazen)

Wu has made new firmware for PCI1, PCI2 which seems to fix the 'spigot 8' reset problem. Seems to work here, at least.

Meanwhile, trying to get 3_7_8 working on cms2. After installing:

     wget http://pcuscms16.cern.ch:8080/hcaldaq/release/installDAQ_3_7_8.perl
     perl installDAQ_3_7_8.perl --mode=teststand --source

cd $XDAQ_ROOT/hcal and 'make'. Fails with some complaint about -lCDFROOT. Jeremy says:

   Please go to "dist/src/CDFROOT/src" and run "make".  I should automate this, 
   but it's pretty far down the list.  I've put it on the TODO list, though.

OK. Back to $XDAQ_ROOT/hcal, 'make'. It works!

Making a private source tree:

  > mkdir my_source
  > cd my_source
  > cp ~daqowner/installDAQ_version.perl
  > perl installDAQ_version.perl --ownsource=/path/to/my_source

This works fine if you have a CVS account at CERN with the same username as your login name, and you know the password.

If your username is different, add to your '''.ssh/config'''

   Host *.cern.ch
       User ehazen

If you need to use anonymous CVS, modify the 'installDAQ....perl' script, commenting out the two lines indicated:

      if ($ownsource) {
  #       $sourceLocs{"HCALdaq"}=[":ext:isscvs.cern.ch:
  #    } else {  
          $sourceLocs{"HCALdaq"}=[":pserver:anonymous:98passw
      }

After this, running the install script with the --ownsource option should create a TriDAS tree with all the relevant links in your my_source directory.

Next:

  setenv XDAQ_ROOT /path/to/my_source/TriDAS
  cd $XDAQ_ROOT/../CDFROOT/src
  make
  cd $XDAQ_ROOT/hcal
  make install

and all should be well. Also, to resolve dependencies for stand-alone tools (like DCCprogrammer) I had to make a link from my ~dist/lib to ~daqowner/dist/lib. Not sure if this is the best solution.

Now I can build and run the DCC tools in $XDAQ_ROOT/hcal/hcalDCC/tool.

2007-04-30 1pm (hazen, stjohn)

DCC test setup won't start (test_all.exe). Hangs after initialization with a count of '1' in the HTR#8 block count (this spigot is not used). There is a reputed "Spigot 8 problem" at CERN too.

Try downgrading DCC to v2c0c firmware... no change.

Try downgrading DCC to v2b06 firmware... no change.

2007-04-09 10am (hazen) Success!

Muon bits seen in oSLB output! Using HTR firmware 0x1131. Set HTR input LUTs to

     0x400   0x420   0x440   0x460
     0x401   0x421   0x441   0x461
     0x402   0x422   0x442   0x462
     0x403   0x423   0x443   0x463
       ...
     0x41F   0x43F   0x45F   0x47F

(identity 1:1 table with muon bits set everywhere). Set HTR output LUTs to all zeros (use NULL command in htr.cc). Set input RAMs on HTR to a pulse every 50 clocks, 4 clocks wide. Pedestal is exp=0, mant=3. Pulse is exp=2, mant=31.

Set oSLB register 0 to 0x0.

Capture input data from receiving HTR:

 486:    0x3ffff    0x3ffff    0x3ffff    0x3ffff    0x10000    0x10000    0x10000    0x3ffff
 487:    0x3ffff    0x3ffff    0x3ffff    0x3ffff    0x1f000    0x1f000    0x1f000    0x3ffff
 488:    0x3ffff    0x3ffff    0x3ffff    0x3ffff    0x10100    0x10000    0x10100    0x3ffff
 489:    0x3ffff    0x3ffff    0x3ffff    0x3ffff    0x11402    0x17400    0x11402    0x3ffff
 490:    0x3ffff    0x3ffff    0x3ffff    0x3ffff    0x10000    0x10000    0x10000    0x3ffff
 491:    0x3ffff    0x3ffff    0x3ffff    0x3ffff    0x1f000    0x1f000    0x1f000    0x3ffff
 492:    0x3ffff    0x3ffff    0x3ffff    0x3ffff    0x10000    0x10000    0x10000    0x3ffff

With the non-zero values like at (488, 489) spaced every 100 words (50 BX clocks). This is what we expect, more or less. There are no rotating BCN or parity bits because this HTR firmware 0x1131 isn't for HO and doesn't send this info.

2007-04-04 10am (hazen)

Working on HTR pattern RAMs. Updating all HTRs to version 31 (actually, 0x1131). New '-a' option to htr.cc doesn't work because our HTR VME chips are out of date.

Making some htr scripts in ~hazen/src/htr. set_rams_1.htr sets various constant values in RAMs. Seems to work. Also works with pulses. Now to try to get something sensible from the Level 1 path.

Quit for today. Next to do: program LUTs, look at TP output, see if muon bits are set sensibly.

2007-03-29 9am (hazen)

Aha! Can reproduce CERN problem:

  1. Erase Xilinx sector (DCCprogrammer -x XILINX)
  2. Load old version into LOG3 (DCCprogrammer -p LOG3 pci3va.hex)
  3. cycle power

Now all attempts to access motherboard dump core! If we can just fix this...

Something somewhat subtle with the U2 initialization is going on. Drop it for now. There are a couple of potentially helpful test programs in ~hazen/src/DCC/SuperDebug:

DCCDebug.cc - instantiate and initialize a motherboard

DCCDumpVrai.cc - dump the non-zero regs of U2 VRAI

-----

All commandline tools abort immediately and dump core. Crate was off. Switched on and tried /etc/init.d/daq restart. No good. Added some more try...catch blocks to DCCdiagnose. getAdapterFor was throwing an exception regarding the CAEN VME library. Tried again:

  /etc/init.d/daq/stop
  /etc/init.d/daq/start

No it's OK. No idea why.

  TODO:  commit DCCdiagnose.cc
         fix TTC FIFO readout in expert view

2007-02-16 10:00 (hazen)

DAQ runs OK, but no data is collected to root file.

to latest versions. Installed SLink transmitter. Moved GIII (SLink receiver) from cms2 to cms1 (bounced both machines).

2007-02-14 16:30 (hazen)

in ExpertView. clean up usage().

2007-02-07 07:30 (hazen)

Release 3_7_4 is out... may install later today. Check with me before depending on the system to work!

Fixing up Expert View for DCC v2c0c firmware. Edits to:

2007-02-05 11:00 (hazen)

Testing DCC::testTTS() - doesn't seem to work (oops!). Fixed, committed to CVS.

2007-02-01 10:30 (hazen)

Testing out DCC expertview, etc.

Created documentation page DCC_CfgScripts for config script parameters with link from countinghouse repository.

2007-01-31 16:30 (hazen)

2007-01-30 13:30 (hazen)

Next things on my ToDo list:

2007-01-25 13:22 (hazen)

Current state of the software (on cms1):

cause setup to crash.

Preparing to test release 3_7_3. Saved current tree as /tmp/HCALdaq_25Jan07.tar.bz2. Here it goes:

  cms1
 ~/dist/src $ cvs_esh co -r Release_3_7_3 TriDAS/hcal/hcalHW
  ? TriDAS/hcal/hcalHW/lib
  ? TriDAS/hcal/hcalHW/src/linux
  cvs checkout: Updating TriDAS/hcal/hcalHW
  cvs checkout: Updating TriDAS/hcal/hcalHW/doc
  cvs checkout: Updating TriDAS/hcal/hcalHW/doc/html
  cvs checkout: Updating TriDAS/hcal/hcalHW/doc/rose
  cvs checkout: Updating TriDAS/hcal/hcalHW/hal
  U TriDAS/hcal/hcalHW/hal/Fanout_v4_r2_a.dat
  cvs checkout: Updating TriDAS/hcal/hcalHW/interface
  U TriDAS/hcal/hcalHW/interface/hcalDCCVMEReadout.hh
  U TriDAS/hcal/hcalHW/interface/hcalHLX.hh
  U TriDAS/hcal/hcalHW/interface/hcalHTRManager.hh
  U TriDAS/hcal/hcalHW/interface/hcalHTRMezzanineSite.hh
  U TriDAS/hcal/hcalHW/interface/hcalSLB.hh
  U TriDAS/hcal/hcalHW/interface/hcalTTCFanout.hh
  cvs checkout: Updating TriDAS/hcal/hcalHW/src
  cvs checkout: Updating TriDAS/hcal/hcalHW/src/common
  RCS file: /local/reps/tridas/TriDAS/hcal/hcalHW/src/common/hcalDCCItem.cc,v
  retrieving revision 1.4
  retrieving revision 1.5
  Merging differences between 1.4 and 1.5 into hcalDCCItem.cc
  TriDAS/hcal/hcalHW/src/common/hcalDCCItem.cc already contains the differences between 1.4 and 1.5
  U TriDAS/hcal/hcalHW/src/common/hcalHLX.cc
  U TriDAS/hcal/hcalHW/src/common/hcalHTRManager.cc
  U TriDAS/hcal/hcalHW/src/common/hcalHTRMezzanineSite.cc
  U TriDAS/hcal/hcalHW/src/common/hcalSLB.cc
  U TriDAS/hcal/hcalHW/src/common/hcalTTCFanout.cc
  cvs checkout: Updating TriDAS/hcal/hcalHW/test

Now doing 'make clean; make install'