SVT Database Use: What goes in Hardware, Run Conditions, or Trigger database? (WJA, 2001-11-01)

• Background

• See Marco's slides

• Example hwset and mapset files

• From my perspective, database serves two purposes:

• Bookkeeping, i.e. knowing which configuration was downloaded for every run. A necessary and sufficient check that bookkeeping was a success is whether offline svtsim can reproduce SVT output.

• File serving, i.e. a mechanism to access SVT configuration data from an offline job running anywhere that offline jobs run.

• Another possible purpose of the database that has been discussed lately (and whose implementation is still being discussed) is central control and tracking of changes to trigger subsystems' downloads, by the Trigger Working Group.

• Minimal functionality needed:

• Experts create immutable (non-editable) mapset and hwset files and store them in SVT_DATA table in Hardware DB.

• Special rows in SVT_DATA indicate the name of the "current" hwset and mapset files to use and their CRC values.

• At start of run, R_C fetches names of hwset and mapset to load, then fetches the hwset and mapset data and sends them to the crates via smartsockets.

• Bill Badgett already implements a snapshot of the Hardware DB at the start of each run. This looks to an offline user as if it were an entry in the Run Conditions DB. Thus, an offline job can ask, for a given run number, what are the name and CRC of the hwset and mapset used to download SVT.

• This solves the "bookkeeping" problem, no matter what mechanism is actually used to move the pattern files, etc. to the crates themselves, because the mapset file contains CRCs of all of the memory images that are derived from its contents.

• It seems feasible to transmit all pattern files, etc., from R_C to the VME crates as part of the configuration message, but some optimization may be needed. Marco and Subir found that it takes about 2 seconds to read each pattern file from the database; transmitting each pattern file over smartsockets takes about 0.3 seconds, which is comparable to the network speed.

• If we do this, though, we lose the cache capability that we have used so far to speed up SVT initialization.

• The details of moving the data online are transparent to all but a few experts, so let's address bookkeeping first.

• The "file serving" problem is solved by storing all files (hwset, mapset, pattern, superstrip, and fit constants) as BLOBs in the SVT_DATA table. Offline support for BLOBs is already needed for other applications and is expected some time in December. In the meantime, we can serve the files by e.g. HTML and use the database for bookkeeping.

• All SVT configuration data would be stored in the Hardware DB.

• Link to Trigger DB?

• We first note that the plan described above will make our implementation similar to XFT's. XFT requires expert intervention to change configurations, but uses the hardware database to enforce the bookkeeping by storing names/numbers to tag what should be loaded.

• One could imagine having the SVT experts agree that at all times, N different "valid" sets of constants will be nominated, e.g. one "SVX only" set, one "wide roads, big beam" set, one "narrow roads, tuned to beam" set, and one "narrow roads, tuned to beam, allow one SVX miss" set. The trigger table could then indicate which of these N types of constants it requires. Then the trigger table would track major changes such as enabling "4/5" pattern recognition, while only the Hardware DB would track minor changes, such as tuning to a new beam spot.

• If we let the trigger table choose an SVT configuration, then we need either to make an explicit entry in the Run Conditions DB to indicate what hwset and mapset were actually loaded or to read the Trigger DB from svtsim.

• If we let the trigger table choose an SVT configuration, then we need to partition the constants more carefully into ones that the TWG wants to control and ones for which the TWG prefers not to have to make a new trigger table (e.g. masking one ladder in a HF vs changing TF chisq cut or HF cluster charge cut).

• I think the database scheme that we are now implementing has enough flexibility that if we decide in the future that various instances of SVT and XFT configurations should be tracked by the trigger table, that option can be handled as a small perturbation.

• Would be nice:

• We should make sure it is easy to search the run database for runs in which SVT mapset or hwset name contains a given substring.