text:
# A Readout System for the STAR Time Projection Chamber

## I Introduction

The Solenoidal Tracker at RHIC (STAR) is a large detector, designed to study relativistic heavy ion collisions at center of mass energies up to $`\sqrt{S_{NN}}=200`$GeV per nucleon, for nuclei ranging from protons to gold. The main purpose of the experiment is to search for signals of the quark-gluon plasma (QGP). A wide variety of QGP signatures have been proposed. STAR is optimized to study produced hadrons over a large fraction of phase space. This imposes many requirements on the central time projection chamber (TPC) that is the central detector for STAR and on its front end electronics (FEE). The TPC must track and identify particles over as large an acceptance as possible. Because central heavy ion collisions produce high-multiplicity final states, it must have good pattern recognition and reconstruction capabilities in regions that are densely packed with tracks. The tracking must be efficient at finding and identifying low-momentum particles, and have good momentum resolution for high-momentum ones. It must be able to accurately point tracks back to the primary vertex and be able to detect secondary and tertiary vertices such as $`K^0`$, $`\mathrm{\Lambda }`$, $`\mathrm{\Xi }`$ and $`\mathrm{\Omega }`$ decays. It must be able to identify particles by $`dE/dx`$ below the relativistic rise. In order to measure the source size via Hanbury-Brown Twiss correlations, it must have excellent two-track separation. In addition to the search for the QGP, RHIC will also study collisions of polarized protons and also ultra-peripheral electromagnetic interactions involving heavy ions.

The STAR TPC is a 4 meter long cylinder surrounding the beam pipe. The TPC covers from 50 cm to 200 cm in radius. The gas is 90% argon, 10% methane (P-10) at 1 atmosphere; 50% helium, 50% ethane is a possible upgrade to reduce multiple scattering. The electronics is designed to accommodate either gas.

Electrons from ionized gas drift toward the nearest endcap, where they are read out. The TPC readout is divided into 24 sectors, 12 at each end. Each sector is divided into inner and outer subsectors. As Fig. 1 shows, the readout chambers have 3 wire planes: a gated grid, ground plane, and anode wires. The inner and outer sector geometries are somewhat different, to emphasize 2-track separation at small radii, and good $`dE/dx`$ and moderate channel count at large radii. Table I compares the inner and outer sector characteristics.

The gated grid is normally closed, minimizing buildup of positively charged ions in the drift volume. When a trigger is received, the voltages are switched and it becomes transparent. The ground plane, anode wires and TPC pad plane form a multi-wire proportional chamber. Electrons drift to the wire where they initiate an avalanche, leaving a cloud of positively charged ions remaining around the wire. The 136,608 pads on the TPC pad plane image this charge; this image charge goes to the electronics.

The size and shape of the ion cloud depends on the number of primary ions, drift distance and diffusion, and gas gain. These factors are quite different for the inner and outer sectors. The signal $`S`$ reaching the electronics depends on the capacitative coupling $`C_c`$ between the ion cloud and the pad:

$$S=N_{pri}GC_cT$$
(1)
where $`N_{pri}`$ is the geometry-dependent number of primary electrons, $`G`$ is the gas gain, and $`T=0.63`$ accounts for the convolution of the diffusion-spreading with the preamplifier shaping. The value 0.63 is the convolution of a Gaussian distribution for a 2 meter drift distance with the actual amplifier response function. Table 1 compares the other values for the inner and outer sectors.

The gas gains depend on the wire voltages $`V`$. These gas gains were measured in chamber prototypes using an $`{}_{}{}^{55}Fe`$ source. The inner ($`G_i`$) and outer ($`G_o`$) sector gains may be parameterized

$`G_i=\mathrm{exp}[0.01267(V520.1)]`$ $`G_o=\mathrm{exp}[0.009341(V628.7)].`$ (2)
Since the signal has a long tail as the ions drift away from the wire, the measured gain depends on the shaping time used in the amplifier. The quoted gains are for the 200 ns FWHM shaper used in STAR.

To minimize the charge deposited on the wires, and maximize the chamber lifetime, the anode wires are operated at the lowest possible gain consistent with a 20:1 signal to noise ratio: about 3770 in the inner sectors, and only 1230 in the outer sectors. Since the noise levels are very similar, the two geometries produce similarly sized minimum ionizing particle signals, about 22,000 electrons.

The width of the signals depends on both the diffusion and on the response of the wire chamber. In P-10 gas in the full 0.5 T magnetic field, the transverse and longitudinal diffusion are 185 $`\mu m/\sqrt{cm}`$ and 320 $`\mu m/\sqrt{cm}`$ respectively. After a typical 1 meter drift, an electron cloud measures 1.8 mm transversely by 3.2 mm longitudinally. This cloud is further broadened by the track dip angle and crossing angles. The TPC resolution is roughly given by the diffusion divided by $`\sqrt{N_{pri}}`$ where $`N_{pri}`$ is the number of primary electrons, typically 100. After accounting for the dip and crossing angles and averaging over chamber position, the resolution is about 350 $`\mu `$m transversely by 700 $`\mu `$m longitudinally. The TPC TPC pads have widths far larger than this transverse resolution. Because of this, a multi-pad fit is required to reach the diffusion limit.

The STAR system is similar to the design used for the EOS and NA-49 TPC electronics. In fact, NA-49 and STAR share the SCA/ADC chip design. However, the STAR system has a larger dynamic range, lower noise, and a much faster readout rate. A slightly modified version of this electronics is also used for the 22,000 pads in the 2 STAR forward TPCs (FTPCs).

## II Electronics Requirements & Specifications

Because of the high probability of multiple hits on a single TPC pad, each pad must be read out with a waveform digitizer. If the system takes several samples for each ionization cluster reaching the pad, then a 3-point fit can be used to localize the hit to within a fraction of a time sample. For a 3-point fit, calculations show that the contribution of the electronics noise to the resolution is comparable to the intrinsic chamber resolution when the signal to noise ratio is 20:1. This requires that the electronics noise must be less than 1,000 electrons. Noise decreases with increasing shaping time; the electronics is best matched to the TPC if the shaping time is comparable to the width of the signals. With P-10, the drift velocity is about 5.5 cm/$`\mu `$s (36$`\mu `$s total), so signals are about 200 ns FWHM. Helium-ethane has a much slower, 2.5 cm/$`\mu `$s drift velocity (80$`\mu `$s total), but less diffusion, so the change in signal length is moderate. To accommodate the variation, a shaper with a adjustable shaping time was chosen. It should be noted that the chamber resolution could be improved slightly by increasing the signal to noise ratio further.

The other significant analog challenge came from the wire response function. The signals have a long-lived ($`20\mu `$s) tail due to the time it takes for the positive ion cloud to disperse from around the wire. This tail is different for P-10 and helium-ethane. This tail is removed with an electrically adjustable tail cancellation circuit. The response was fine tuned by using digitized waveforms of the actual chamber wire response function.

The number of digitized samples is set by the diffusion spreading and two-track resolution. If the signal is spread over 3 buckets, then the arrival time can be found to $`20\%`$ of a bucket; this requires about 500 time samples. The electronics accepts a maximum of 512 samples (with an option to upgrade to 1024). In the summer 2000 run about 380 time-buckets were filled. The multi-pad and multi-time sample fits used to find the position of the ionization clusters impose fairly stringent requirements on channel to channel gain variation ($`<5`$%) and crosstalk ($`<0.3`$%).

The dynamic range is set by the energy loss ($`dE/dx`$) of produced particles. The maximum ionization expected was from a 200 MeV/c proton, which produces a signal that is 10 times minimum ionizing. We allow a further factor of 4 to account for non-zero dip angles (which increase the path length) and Landau fluctuations. With the 20:1 signal to noise ratio, this leads to an 800:1 dynamic range, or 10 bits.

Finally, the system must work in a 0.5 T magnetic field. Cooling considerations and the need to maintain the pad plane at a constant temperature (for $`dE/dx`$ stability) limit the power consumption to less than 100 mW/channel. The system must be read out in 10 msec. Although STAR will write less than 100 events/second to tape, the faster readout rate will allow a larger event sample to be examined by the level 3 trigger.

The FTPC FEE requirements were similar to those for the main TPC. However, the signals were considerably slower, and so the signal shaping required a FWHM of about 400 ns. The longer shaping time reduced both the signal slewing rates and the preamplifier noise. Only 256 time-buckets were needed in the system. However, because of the limited cooling available, power consumption was a problem. Because of the slower rise times, we were able to run the SAS preamplifiers and output buffers at considerably lower currents than in the main TPC, 40 mW/channel.

## III System Architecture

Figure 2 shows how the system is divided into two components. Small, 32-channel FEE cards plug into the TPC pad plane, and contain all of the circuitry for amplification, shaping and analog to digital conversion. The FEE cards are supported by larger readout boards, which provide power and control signals, read out the data, and send it to the data acquisition system.

The FEE card contains two custom chips: a STAR preamplifier/shaper (SAS) chip and a switched capacitor array/ADC chip (SCA/ADC); each chip contains 16 channels. The SAS contains low noise integrating preamplifiers, 2-pole shapers and buffers to drive the SCA chip. The SCA/ADC includes 512 time-bin switched capacitor arrays and 12-bit Wilkinson ADC’s, plus an output buffer and multiplexer. Both chips are implemented in a 1.2$`\mu `$m 2-poly, 2-metal CMOS process. A key attribute of this process was the ability to make high-quality (good dielectric) capacitors for the SCA.

Up to 36 FEE cards are controlled and read out by a readout board (RDO). The board multiplexes the digital signals from the FEE cards and sends the data to the data acquisition system (DAQ) on a 1.2 Gbit/sec fiber optical link. A trigger system initiates events, including several types of calibrations, and distributes the SCA acquisition clocks. The readout board also includes a 1-event memory for event storage (mostly for debugging), a slow controls link that can turn parts of the system on and off and monitor temperatures and voltages. It also includes voltage regulators for itself and the FEE cards.

In the FTPCs, the FEE cards each read out 64 channels; a single FEE card is connected to a cable. By using smaller chip packages, the FTPC FEE cards are about the same size as their main TPC counterparts, despite the increased channel count.

### A SAS chip

Fig. 3 shows a block diagram of the SAS chip. The input is a folded cascode integrator with a switched reset. The input FET capacitance is about 10 pF, optimized to match the TPC pad capacitance. The FET measures 7200 $`\mu `$m by 1.2$`\mu `$m, and runs with 2.9 mA drain current. This gives it a noise of 560 electrons + 14 electrons per pF of input capacitance. The typical input capacitance was about 25 pF, from the pad, connecting trace and connector. This led to a typical noise level of 910 electrons.

The integration capacitor $`C_i`$ is 1.6 pF, big enough to handle a maximum input charge of 1.1 pC, or 300 minimum ionizing particles. A resistive discharge was considered for the integrator, but was abandoned because of the difficulty of obtaining consistent, appropriately high resistance. When a trigger occurs, the reset switch opens in parallel with the TPC gating grid, so no data is lost due to the opening time.

The chip includes input protection diodes for protection against sparking. Although the FEE cards include a place for external protection diodes for additional protection, external diodes were not installed. If an anode wire sparks to a pad, damage will usually be limited to that channel, with some chance of destroying the entire 16-channel SAS chip. This was considered an acceptable risk, compared to the problems of leakage currents in the external diodes.

The two-pole shaper and tail correction are both electrically adjustable to accommodate either P-10 or helium-ethane gas. The shaper peaking time is adjustable from 60 to 150 ns. The tail correction timing is similarly adjustable. The tail correction is adjustable so that, for either P-10 or He-Eth, after $`1\mu `$s, the remaining uncorrected tail is less than 1% of the maximum input.

The resistors are implemented with MOSFETs biased in their linear region. The resistances are adjustable by changing the gate voltage; these voltages are externally adjustable. The drawback of this approach is that the resistors are only linear over a limited range, usually $`<1`$ volt. So, the resistors were implemented with 4 MOSFETs in series. With this circuit, the width changes by $`<5\%`$ going from small signals to full scale. The chip-to-chip variation in timing constants and chip gain are both less than 4%. Unfortunately, the chip-to-chip gain variation is largely correlated with the variation in pulse width, so that the integrated output voltage varies by a larger factor, up to 8%.

The shaper amplifier uses the ‘cold resistor‘ technique for differentiation. This reduces the noise referred to the input. In the SAS, almost all of the noise comes from the input FET.

The overall chip gain is 16 mV/fC, so a minimum ionizing signal is about 40 mV; the maximum output voltage is 2 volts. The crosstalk specification is $`<`$ 0.36%. This required using very heavy metallization.

The chip includes a charge-injection calibration system. Charge can be injected into each input through a capacitor connected to an external voltage. A serially loaded shift register determines which channel(s) are pulsed.

The output buffer is designed to drive a 50 pF load (2 SCA’s). The reason for this is that the FEE card includes an option to add 2 additional SCA/ADC chips to the back of the board, to expand the system to 1024 time samples.

The SAS die measures 2.6 mm by 3.6 mm and was packaged in an 68 pin PLCC. The yield was about 70%, in accord with expectations based on the die size. Dice were also mounted in an TQFP80 package for use in the FTPC readout.

### B SCA/ADC chip

As Fig. 4 shows, each channel of the SCA/ADC chip is made up of a 512-capacitor array which serves as an analog storage unit, and a 12 bit ADC. During sampling, each capacitor is connected to the chip input in turn, by a switch. The 512 switches are closed in turn by a shift register.

The SCA performance is dependent on the process used to make the chip. We used a 1.2 $`\mu `$m CMOS process; the 0.7 pF capacitors are made from two metal layers separated by a polysilicon dielectric. This gives them a low dielectric loss angle.

The ADC is a conventional 12-bit Wilkinson rundown ADC; STAR uses only 10 bits. In STAR, it counts on both edges of a 90 MHZ clock signal. Counting takes about 6$`\mu `$s, with an additional $`4`$ $`\mu `$s allowed for the op amp settling and switching between capacitors. Running 16 converters in parallel introduces significant economies of scale. Each individual channel requires only a comparator and a latch. A single voltage ramp feeds all 16 comparators and a single 12-bit counter feeds all 16 latches. The 12-bit counter is implemented as 3 4-bit Grey code counters. This design was a tradeoff between circuit complexity and minimizing the number of transitions on any given clock edge. During SCA testing, the chips are tested with a slow ramp. Any problems in the Grey code counter will show up as non-monotonic behavior on the ramp, and the chip will be rejected. The 16 latches are read out by a double-buffer, a 16:1 multiplexer, a Grey code to binary converter, and an output buffer. The buffer was designed to allow analog to digital conversion to proceed during readout of the preceding time samples. We do not use this feature because it introduces considerable additional noise.

The system has a dynamic range of up to 4 volts; we use a 2 volt range because it offers improved linearity, $`<2`$%. One ADC count corresponds to 2 mV. The SCA noise is about 1 mV, relatively independent of operating condition. This noise is probably due to charge injection in the switches. The pedestal varies by about 4 mV from time-bin to time-bin; this pedestal is subtracted on-line by the data acquisition system.

The analog performance of the SCA/ADC was studied with a variety of methods. The dynamic linearity was studied with fast (5 MHz) triangle waves, and was found to match the static behavior. The effective aperture time was studied with an input step function. The phase of the step with respect to the sample clock was adjusted in 1 ns steps, and the changing output monitored. The measured rise time was about 5 ns, comparable to the rise time of the input step. This was more than satisfactory for STAR needs. Crosstalk was observed only between adjacent channels, but most of this is probably external to the chip.

The biggest chip-to-chip variation was in the SCA/ADC gain, about $`\pm 15\%`$. The channel-to-channel gain variation within a chip was small, typically less than 0.4%. The larger chip-to-chip gain variation was likely in the ADC ramp generator. The chip-to-chip gain exceeded our specifications, so we sorted the chips by gain, into 6 different classes. A different external resistor was used in the ramp generator for each class, to greatly reduce the gain variation.

Because of the DAQ zero suppresion design, all pedestals were required to be less than 256 ADC counts (512 mV). Most pedestals were in the range of 125-175 ADC counts. Within a channel, each time-bin had a slightly different pedestal; typical variation was a few mV. These pedestals were determined and subtracted by the data acquisition system. The pedestals have been very constant with time, varying by a small fraction of an ADC count.

If the system is dormant for too long (10’s of seconds), leakage allows the voltages on the SCA capacitors to drift, eventually reaching saturation. When they reach saturation, a single readout cycle may not be enough to fix the capacitors to the correct input voltage. Because of this, we do not accept the first event after the sytem has been halted or paused. During operations, we generate fake triggers as needed to avoid saturation.

The SCA die measures 5 mm by 8 mm and was packaged in an 84 pin PLCC. The yield was about 30%. Chips used in the forward TPC (FTPC) readout were mounted in PQFP100 packages.

### C FEE Card

Fig. 5 shows a photo of the FEE card. The large chips are the SAS and SCA’s. The card also includes a 90 MHz ADC clock and a 2:1/4:1 multiplexer. Signals are read out over a 50-pin ribbon cable. Most of the control signals are distributed over this cable, along with regulated $`\pm 5`$ VDC. Control voltages for the SAS and SCA are also distributed on this cable, but with on-board buffers for the nodes with significant current draw. The 90 MHz ADC digitization clock is generated on the FEE card, to avoid high-frequency signal distribution.

In addition to the pad signals, the 44-pin FEE connector also provides an 8 bit geographical address. These addresses are etched into the pad plane, and uniquely identify each of the 181 FEE cards in a sector. These identifiers can be read out to the data acquisition system through a geographical address event. These geographical events make it easy to identify cabling errors.

Since the TPC pad plane must be maintained to $`\pm 0.7\mathrm{deg}`$ C to keep the wire gain constant and maintain the $`dE/dx`$ calibration, the FEE cards needed to be water cooled. The cards are cooled through a mounting bracket which is glued and screwed to the back of the card. This bracket mounts to a 3/8” square aluminum cooling channel which has water running through it. This connection to the cooling channel also serves as a ground.

Because the FTPC signals are slower, the FTPC system can be run at lower currents. Both the SAS input current and SAS buffer current are significantly reduced. This allows the FTPC FEE cards to be air cooled.

### D Noise

Besides the SAS, noise comes from 3 sources: dielectric losses in the TPC pad plane, charge injection noise in the SCA, and quantization noise in the ADC.

The G-10 pad plane is a poor dielectric, with a loss angle $`\mathrm{tan}(\delta )0.014`$. The capacitance with a G-10 dielectric acts as a noise source, generating noise equivalent to a resistor with resistance given by the real part of the capacitance, $`R=\mathrm{tan}(\delta )/\omega C`$. For the STAR geometry and shaping, the noise is about 300 electrons.

The SCA charge injection noise of 1 mV is equivalent to 420 electrons. The SCA quantization noise is $`1/\sqrt{12}`$ ADC counts, or about 250 electrons. All of the noise sources are independent, so the total noise is their quadrature sum of 1075 electrons (for a relatively high capacitance, 25 pF pad).

Fig. 8 shows a histogram of the noise on the 5682 installed channels in one sector. The mean is 1.3 counts, or 990 electrons, meeting the specifications; there is no significant additional noise due to inadequate grounding, pickup, or other external sources.

### E Readout Boards

Figure 6 shows a block diagram of the readout board. Its main function is as a multiplexer, reading data from the FEE cards and sending it to the data acquisition system over a 1.2 Gbit/sec fiber optic link. Digitization on the FEE card proceeds synchronously with data transmission, so there is little buffering on the readout board.

As Fig. 7 shows, the FEE cards are divided into groups of 4, 2 boards on each of 2 cables. Each group is controlled by its own field programmable gate array (FPGA), a Xilinx 4008. Each group also has its own positive and negative voltage regulators, and can be turned on and off via the STAR control system. Each FPGA is also connected to a 40-bit wide 15 MHz tristate bus which transfers the data to the serializer chip and laser diode. The FPGAs contain state machines which control the 4 boards, and buffers to synchronize the FEE card readout with the data readout bus cycle. The data are sent to DAQ in an order that simplies the eventual on-line reconstruction. The ordering matches the requirements of the data acquisition system , but is essentially arbitrary from the FEE standpoint. This required ordering complicates the state machine.

The 40 bit wide bus feeds a 2:1 multiplexer/level shifter/latch which itself feeds a HP G-link serializer which sends the data to a Finnisar optical transmitter. The G-link serializer runs at 60 MHz, so that the data is sent to the DAQ system at 1.2 GHz; the link actually runs at 1.44 GHz after Manchester encoding. Because the DAQ system is less than 100 meters from the detector, we use relatively inexpensive 50$`\mu `$m diameter multi-mode fiber for the transmission.

The readout board also has a 1 Mword 10-bit wide memory that can store the data from a single event. This memory is intended mainly for diagnostic purposes, allowing the same event to be sent multiple times. It can also be written to and read out from the slow controls link discussed below.

Data collection, digitization and readout is controlled by a trigger bus, an 8 pair differential pECL bus that runs at 5 times the RHIC crossing frequency (about 50 MHz); one cable drives an entire sector of 6 readout boards. Every 105 ns (each RHIC crossing), the trigger transmits a 4-bit trigger action word, a 12-bit trigger token and a 4-bit DAQ action word. The trigger action word can initiate readout, abort the readout of an event in process (if it is rejected by a higher level trigger), initiate calibration readout with the SAS pulser, and write events to or read events from the event memory. The token uniquely identifies each event, insuring that, during event building, there is no mixing of pieces from different triggers. The DAQ word can be used to control DAQ functionality to allow for different type of processing for different types of triggers.

The readout boards are monitored and controlled by a hardware controls link. The system can monitor voltages, currents (via the voltage drop in the power cables), and temperatures. Thermistors for temperature measurement are placed on the pad plane itself. It can also independently control power to each group of 4 FEE cards. Finally, it can be used to read out the event memory. Because this readout takes 40 seconds/sector, is mainly for diagnostic purposes.

The link is based on the high-level data link control (HDLC) protocol. The physical link is RS-485 running at 1 MBit/second. On the readout board, the protocol is implemented in a plug-in card holding a Motorola 68302 communications processor, memory, and interface logic. The off-detector end is a commercial VME module. Because of some limitations on the commercial modules, the system includes ’looped back’ timing, giving a system that behaves closer to RS-422. Each sector of 6 readout boards is fed by a single HDLC cable. The same hardware and protocol is used by several other STAR subsystems.

The readout board is powered by $`+8`$ V and $`8`$ V ferroresonant power supplies. A fully populated readout board consumes about 15 amps at both $`+8`$ V and $`8`$ V; each readout board is powered a by Kepco Series C 300 watt power supply. Ferroresonant supplies were chosen for their reliability, moderate regulation and lack of switching noise. They regulate output voltage to within 0.7 V (for 25% to 100% load changes); linear regulators on the readout board provide better regulation, but the initial regulation reduces the on-detector power dissipation. The power cables are part of the system; $`+8`$ V, ground and $`8`$ V are each carried by two 12-gauge wires. The ground lead current is small because the positive and negative current consumption are quite closely matched. The voltages are monitored at the supply and on the readout board; the voltage drop (typically about 1V) is proportional to the current consumption. To avoid ground loops, the cable grounds are not grounded at the power supplies. The readout boards are water cooled, using a similar setup as with the FEE cards.

There is no zero suppression on the readout boards. The introduction of relatively inexpensive, fast fiber optic links, together with uncertainties about the algorithm for selecting non-zero sequences, led STAR to implement zero suppression in the data acquisition system.

As Fig. 7 shows, physically, the readout boards are split into master and slave pieces. The master contains all of the unique parts of the circuitry: fiber driver, trigger interface, event memory and slow controls interface, along with 5 FPGA FEE-card controllers, enough to control 20 FEE cards. The slave board contains 2, 3 or 4 FEE-card controllers, to control 8, 12 or 16 FEE cards. This multiple-sizing matches the readout boards to the number of nearby FEE cards and available physical space. On the inner sectors, the master and slave boards are physically separated by a 20 cm cable.

The readout boards also have geographical addresses. These are set via programmed connectors which are physically tied to the cooling structure.

The tight space constraints on the FTPC led to some design changes in the readout board. Readout boards were mounted some distance from the FEE cards, with cables up to 2 meters long, and connected with small (0.025”) pitch cables. To avoid digital crosstalk, the FEE cards were read out at half the speed of the main TPC. The optical link speed was maintained with the use of double buffering. A single board layout was used, instead of the master/slave setup for the main TPC.

### F Waveform Digitization Clock

Although the readout boards operate quite independently, the SCA signal acquisition must be synchronous across the entire detector to simplify the event reconstruction. The SCA acquisition frequency must be adjustable to account for changing gas mixtures, TPC cathode voltages, and possibly pressure. At the same time, acquisition must begin at a fixed phase with respect to the collision that causes the trigger. This is currently done by using the 10.5 MHz RHIC beam-crossing clock as the SCA clock. At this speed, the full drift time corresponds to about 380 time-buckets.

In the future, a programmable clock may be used. An analog prototype based on a delay-locked loop was built, but it proved to be overly temperature sensitive. A programmable clock would allow the frequency to be adjusted to make use of all 512 SCA time buckets. The clock will also be necessary if helium-ethane gas is ever used.

## IV Chip and Board Testing

The system required 10,000 SAS and SCA chips, mounted on 5,000 FEE cards. With the yields, 15,000 SAS chips and 30,000 SCA chips had to best tested. Testing was thus a substantial fraction of the total effort.

We developed a modular tester system, with similar circuitry used to test both chips and the FEE cards. Similar circuitry was used for both chip and board debugging and for production testing. We focus here on the production testing. Only a chip/board specific test card was different for each component. The testers were implemented on PC’s with National Instruments Lab-Windows. The PC communicated with the tester via a 12-channel MIO-16 analog board, with 12 inputs, each digitized to 12 bits, and 2 DAC outputs. Digital communication was via a 96 channel DIO-96 digital interface board.

Much of the ciruitry for the testers was common to all 3 designs. The common circuitry included a programmable signal generator to produce DC levels, pulses, and a sawtooth. Control signals were generated by FPGAs.

The testers were all double socketed. The chip or board plugged into a socket mounted on a small replaceable PC board, which plugged into a socket on the tester. This allowed for quick changes when the socket wore out.

To keep track of the large number of chips and boards, each chip and each FEE board was labelled with an individual bar code. All testing stations were equipped with bar code readers, allowing for automatic identification and easy establishment of a test database. Test results were stored in an EXCEL spreadsheet compatible format.

We decided to test the chips after they were packaged, eliminating the difficult problem of probing the bare wafers. The large chip count required automated testers. To speed the measurements, all 3 tests sequences began with measurements of DC currents and voltages, to quickly eliminate obviously bad components.

The SAS-specific board included a peak detector to measure the amplitudes of output pulses, a peaking time detector to measure the pulse rise time, a zero-crossing time detector to measure the fall time (and check on the tail-correction circuitry), and a stair-case generator to generate different sized input pulses. The analog outputs were fed to the MIO-16 board through a 16:1 multiplexer. After the DC measurements, the tester checked for dead channels and measured cross-talk by injecting pulses into each channel, and observing the output in that channel and the two adjacent channels. The RMS noise was measured, followed by the peaking and zero-crossing times, and the signal width found. Finally, the channel gains and linearities were measured. The complete test sequence for a good chip took about 1 minute.

The SCA tester included a board to inject signals into the SCA at adjustable times, to test different time buckets. After DC tests, the tester checked for dead channels and crosstalk by injecting pulses into each channel at different times. The tester looked for a correctly timed signal in the pulsed channel and no signals (i.e. crosstalk) in all of the other channels. Next, the RMS noise of the pedestal was calculated, and all channels and time-buckets scanned for glitches (as from a single bad storage capacitor). The gains and linearity were then tested by injecting 10 DC levels into the SCA. This test was used to classify the SCAs into groups based on their gain. To keep the SCA gains within a narrow (5%) range, the groups were used with different SCA ramp currents. The ramp current was adjusted by selecting one resistor on the FEE card. Finally, saw-tooth waveforms were injected into the SCA, and the results displayed on the PC monitor, for a final visual inspection by the operator.

The FEE card testing was quite similar to the SCA testing. DC currents and voltages were measured first. These voltages and currents were measured through a 10-pin test connector mounted on the FEE card. Channel functionality and cross-talk were checked. Linearity/gain were measured for 10 different amplitude inputs; the signals were checked at 3 different points on the waveform. The RMS noise was measured, and another glitch test performed. Finally, the pedestals were read out and displayed on the PC monitor, for operator inspection.

## V Conclusions

We have designed, built and commissioned a 136,608 channel 6-12 MHz 10-bit waveform digitization system to read out the STAR time projection chamber. The electronics was installed on the TPC in 1998-99, and has been exercised extensively on cosmic rays and during the summer, 2000 run. The system is working as designed and demonstrating excellent stability; over 99% of the channels are working within specifications.

We thank our STAR colleagues for their support. The gas gain measurements were done by Wayne Betts. Sergei Panitkin provided Fig. 8. The SAS chip was designed by Eric Beuville, the SCA primarily by Stuart Kleinfelder and the ADC by Oren Milgrome. This work was supported in part by the Division of Nuclear Physics of the Office of High Energy and Nuclear Physics of the U. S. Department of Energy under Contract No. DE-AC-03-76SF00098.


format: text

text:
# An Overview of a Grid Architecture for Scientific Computing

## 1 Introduction

This document assumes basic knowledge of the computing Grid concept, which is a paradigm for the modern distributed computing and data handling. For a general introduction to Grid computing the reader is referred to eg. . The most common starting point for constructing a computing Grid is the Globus Toolkit111Globus Project and Globus Toolkit are trademarks held by the University of Chicago. . This toolkit provides a Grid API and developing libraries as well as basic Grid service implementations.

The NorduGrid  project is a common effort by the Nordic222The term Nordic covers the countries: Denmark, Norway, Sweden and Finland. countries to create a Grid infrastructure, making use of the available middleware. Through the European DataGrid project (EDG)  the NorduGrid project has had extensive experience with the Globus Toolkit and with deploying and using a Grid Testbed. During this we have found some shortcomings in the Globus Toolkit and some problems with the EDG Testbed architecture that we would like to address on a Grid testbed in the Nordic countries. In this paper we present a proposal for a Grid architecture for a production testbed at the LHC  experiments. It is not the intent to define a general Grid system, but rather a system specific for batch processing suitable for problems encountered in High Energy Physics. Interactive and parallel applications have not been considered. It is our goal to deploy a Grid testbed based on this architecture for the LHC Data challenges in the Nordic countries.

We will start by describing the various components of the system. Focus will be given to the components which we have developed ourselves or taken from others, but heavily modified. After the description we will give a review of the task flow and the communication between the various components.

It is assumed that the reader is familiar with the major components of the Globus Toolkit since the NorduGrid Testbed architecture uses this as the foundation. An example is the underlying security model - the Grid Security Infrastructure (GSI) - which is a based on the X.509 certificate system and takes care of authentication and authorisation and the GRIS and GIIS of the information system (for an explanation of the GRIS, GIIS, VO terms see  and ).

One of the aims of the architecture was to make the system able to install on top of an existing Globus installation. Rather than modifying the Globus Toolkit, we have made a clear boundary between the two architectures, thus enabling them to coexist and try out this new system without destroying an already working Globus installation.

## 2 The NorduGrid Architecture Components

### 2.1 Computing Element - CE

The computing element (CE) is the backend of the system. It is where the execution of the application is performed. This is a very general element which can be anything from a single processor to complex computing clusters. In our case it is at present limited to simple PBS (Portable Batch System) clusters.

One of our basic ideas about Grid implementations is that it should not impose any restriction on the local site. Therefore it should be possible to use existing computing clusters and place them on the Grid with minor or no special reconfiguration. One thing that we have stressed, which should not be imposed on the site, is that the individual worker nodes (WN’s) of the cluster can not be required to have direct access to the Internet. This excludes dependence on eg. global filesystems like AFS and direct download from the Internet from any WN. Grid services will thus only be run from a front-end machine. This scenario does not however exclude the use of local network filesystems and NFS is actually often used for cluster-wide application distribution.

### 2.2 Storage Element - SE

Similar to the computing element, the storage element (SE) is the common term for another of the basic Grid resources - storage. The storage can be as simple as a standard filesystem or eg. a database. The authorisation is handled using Grid credentials (certificates). A SE can be local or remote to a CE. Here local means that access is done via standard filesystem calls (eg. open, read and close) and is usually realised by a NFS server. A remote SE is usually a stand-alone machine running eg. GridFTP  server with local file storage. Data replication is done by services running on the SE.

A dedicated pluggable GridFTP server is beeing developed for use on the SE. At the moment a simple file access plugin exists. The main reason for this is to have a way to provide a consistent certificate-based data access to the data. At least one other Grid solution to a certificate-based filesystem exists . One advantage of the GridFTP approach is that it is done entirely in user space and thus is very portable.

### 2.3 Replica Catalog - RC

The information about replicated data is contained in the  Replica Catalog (RC). This is an entirely add-on component to the system and as such is not a requirement.

### 2.4 Information System - IS

A stable, robust, scalable and reliable information system is the cornerstone of any kind of Grid system. Without a properly working information system it is not possible to construct a functional Grid. The Globus Project has laid down the foundation of a Grid information system with their LDAP-based Metacomputing Directory Service (MDS) . The NorduGrid information system is built upon the Globus MDS.

The information system described below forms an integral part of the NorduGrid Testbed Architecture. In our Testbed, the NorduGrid MDS plays a central role: all the information related tasks, like resource-discovery, Grid-monitoring, authorised user information, job status monitoring, are exclusively implemented on top of the MDS. This has the advantage that all the Grid information is provided through a uniform interface in an inherently scalable and distributed way due to the Globus MDS. Moreover, it is sufficient to run a single MDS service per resource in order to build the entire system. In the NorduGrid Testbed a resource does not need to run dozens of different (often centralized) services speaking different protocols: the NorduGrid Information System is purely Globus MDS built using only the LDAP protocol.

The design of a Grid information system is always deals with questions like how to represent the Grid resources (or services), what kind of information should be there, what is the best structure of presenting this information to the Grid users and to the Grid agents (i.e. Brokers). These questions have their technical answers in the so-called LDAP schema files. The Globus Project provides an information model together with the Globus MDS. We found their model unsuitable for representing computing clusters, since the Globus schema is rather single machine oriented. The EDG suggested a different CE model which we have evaluated . The EDG’s CE-based schema fits better for computing clusters. However, its practical usability was found to be questionable due to improper implementation.

Because of the lack of a suitable schema, NorduGrid decided to design its own information model (schema) in order to properly represent and serve its Testbed. A working information system, as part of the NorduGrid Architecture, has been built around the schema. Nevertheless, NorduGrid hopes that in the not-so-far future a usable common Grid information model will emerge. We think that the experience of the NorduGrid users gained with our working system will provide a useful feedback for this process.

#### 2.4.1 The Information Model.

The NorduGrid Testbed consists of different resources (they can be referred as services), located at different sites. The list of implemented Grid services contains computing resources (Linux clusters operated by PBS), SEs (at the moment basically disk space with a GridFTP server) and RCs. The designed information model naturally maps these resources onto a LDAP-based MDS tree, where each resource is represented by an MDS entry. In this tree, each NorduGrid resource operates as a separate GRIS. The various resources (GRIS’s) can be grouped together to form Virtual Organisations (VO) which are served by a GIIS (i.e. in our present Testbed configuration, the resources within a country are grouped together to form a VO). The structure created this way is called a hierarchical MDS tree.

The NorduGrid schema is a true mirror of our architecture: it contains information about computing clusters (nordugrid-cluster objectclass), storage elements (nordugrid-se objectclass) and replica catalogs (nordugrid-rc objectclass). In Fig. 1, a part of the NorduGrid MDS tree  is shown. The clusters provide access to different PBS queues which are described by the nordugrid-pbsqueue objectclass (Fig. 2 shows an example queue entry). Under the queue entries, the nordugrid-authuser and the nordugrid-pbsjob entries can be found grouped in two distinct sub-trees (the branching is accomplished with the nordugrid-info-group objectclass). The nordugrid-authuser entry contains all the user-dependent information of a specific authorised Grid user. Within the user entries, the Grid users can find, among other things, how many CPUs are available for them in that specific queue, what is the disk space a job can consume, what is an effective queue length (taking into account the local UNIX mappings), etc. The nordugrid-pbsjob entries (see Fig. 3 an example) describe the Grid jobs submitted to a queue. The detailed job information includes the job’s unique Grid ID, the certificate subject of the job’s owner and the job status.

The NorduGrid information system has been designed to be able to effectively serve the User Interface (UI) (job status query commands, free resource discovery utilities), the brokering agent (in the present implementation it is integrated with the UI’s job submission command) and a general Grid user who can access the Grid information either directly with a simple ldapsearch or through an LDAP-enabled Grid portal.

### 2.5 Grid Manager - GM

In our model, job management is handled by a single entity which we call the  Grid Manager (GM). It is the job of the GM to process user requests and prepare them for execution on the CE. It also takes care of post-processing the jobs before they leave the CE. In the Globus Toolkit context, the GM takes care of what is normally done by the Globus job-manager backend scripts. In fact it is installed in a similar way to a standard Globus jobmanager and can work perfectly together with already existing jobmanagers. Authorisation and authentication is still done by the Globus gatekeeper.

The status of each job is recorded in a special status directory which also contains informational files needed by the GM.

#### 2.5.1 Grid Manager tasks.

The Grid Manager uses a job ID assigned by the Globus gatekeeper/jobmanager to distinguish between job requests. It starts by creating a session directory reachable by the WN and parsing the job request. The job request is passed in a resource specification language. This specifies, among other things, the list of input files needed for the job execution. The Grid Manager then proceeds to download files to the session directory from remote SEs or copy files from local SEs as specified by the job request.Files can also be uploaded by the user submitting the job. Once all files are in place the GM submits the job to the local scheduling system (PBS). All information about job status must be retrieved through the MDS. Communication to the job such as cancellation is handled through the GM as a specific RSL requests. When the job leaves the WN(s) it is the responsibility of the GM to clean up afterwards. The GM also manages the output files and uploads them to local or remote storage elements and registers uploaded files to the Replica Catalog.

### 2.6 User Interface (UI)

In contrast to eg. the Grid implementation in the EDG, the NorduGrid  User Interface (UI) has significantly more responsibility. This is mainly due to the choice of having the resource broker placed at the UI rather than having it as a central service. We found that the EDG implementation with a central broker, where all job requests as well as data payload had to pass through, would be a single point of failure and non-scalable.

The NorduGrid UI is at present command-line driven, while a web based solution is foreseen in the future. The UI is responsible for generating the user request in a Resource Specification Language (RSL) based on the user input. The RSL we use has additional attributes to those provided by the Globus Toolkit . This xRSL has been enhanced to support enriched input/output capabilities and more specification of PBS requirements. All unneeded Globus attributes has been deprecated.

The UI matches the request or job options with the available resources as reported by the MDS, and returns a list of matching resources. Job options can reflect eg.:

* required CPU time
* required system type
* required disk space
* required runtime environment (eg. application software)
* required memory
* required data from SE’s

## 3 Task flow

In this section we describe the life of a job and expose the functions of the various components of the NorduGrid architecture. An overview of the system with task flow can be seen in Fig. 4.

The numbers on the figure refers to tasks which we describe below.

1. The User Interface does a filtered query against the GIIS and query the Replica Catalog to get the location of input data. Based on these responses and its brokering algorithm the Broker within the UI selects a remote resource.
2. The UI contacts the selected resource by submitting the xRSL file to the Gatekeeper of the resource, along with the (resolved by the Broker) physical file names.
3. The Gatekeeper does the required authentication and authorisation and passes the job request to the Grid Manager.
4. Grid Manager creates the session directory and downloads or copies files from Storage Elements to this directory
5. User Interface uploads input files and executables via GridFTP to the session directory of the job
6. After all files are pre-staged, the Grid Manager submits the job to the local resource management system (PBS)
7. PBS schedules and executes the job on the Worker Nodes in the normal way
8. On request, Information Providers collect job , user, queue and cluster information, disk usage information and writes the information to the MDS
9. Job status information is retrieved from the MDS, the User Interface monitors the job status by querying the Information System. E-mail notification to the user during the various job stages by the Grid Manager is also possible.
10. User Interface may cancel and clean jobs by sending RSL request with special attribute set through the Gatekeeper to the Grid Manager. The Grid Manager will then take care of stopping the job and removing the job session directory if requested
11. When the job finishes the Grid Manager moves requested output results to Storage Elements and does registration in the Replica Catalog as specified by the user. The Grid Manager takes care of cleaning up the session directory after its lifetime has exceeded, too While the directory exists, the User Interface can download the specified output files produced by the job.
12. GIIS queries GRIS on demand based on cache timeout values in order to provide fresh enough information. The single GIIS in the figure really represents a whole Virtual Organisation hierarchy. In NorduGrid testbed the hierarchy has a central NorduGrid GIIS which connects to country level GIIS’s. The country level GIIS’s either connects directly to resources or have an institutional layer in between.

In the figure data and control connections are specified. In designing the system we have been very conscious about the location of data bottlenecks and places where the system will have scalability problems. The only data intensive transfers occur between the Computing Element and the Storage Elements and the User Interface. In contrast to the EDG there is no single point where all data has to pass through. All transfers are truly peer-to-peer.

## 4 Conclusion and Outlook

A very early prototype implementation of the architecture exists and is being tested and further developed. Scalability tests still has to be performed.

In order to have a functional testbed within a short time-frame, there are some areas which we have not given much attention and have taken simple but still secure and functional solutions. In the future we plan to include, for example a more advanced authorisation and accounting system.

Once we get experience with the system we are going to make the Broker more sophisticated and make it able to perform user dependent choices such as RC location and preferred resource selection.

## 5 Acknowledgment

We are very grateful for the developers of the Globus Project for their willingness to answer questions and providing an open development environment with access to early version of their toolkit.

As participants in the European DataGrid we have been provided access to a Testbed and the collaboration has been very fruitful.

The NorduGrid project - A Nordic testbed for wide area computing and data handling  (http://www.nordugrid.org/) - is funded by the Nordunet2 program (http://www.nordunet2.org/.


text:
# Numerical studies of the two- and three-dimensional gauge glass at low temperature

## I Introduction

The gauge glass is a model which is often used to describe the vortex glass transition in high-temperature superconductors. Consequently there has been a substantial amount of theoretical work attempting to understand its properties. Nonetheless, there are still areas of disagreement. For example, in two dimensions there is an ongoing controversy as to whether a spin-glass transition occurs at finite temperature or not. In addition, in three dimensions, although it is known that the system exhibits a finite-temperature transition, see, e.g., Ref. Olson and Young, 2000, there is no consensus on the value of the stiffness exponent.

Recent results by means of resistively-shunted junction (RSJ) dynamics by KimKim (2000) claim evidence for a finite-temperature transition in two dimensions with a transition temperature $`T_c=0.22`$. Choi and ParkChoi and Park (1999) find the same value for $`T_c`$ by studying the scaling of the spin-glass susceptibility via Monte Carlo simulations. These claims for a finite $`T_c`$ are in contrast to results by GranatoGranato (1998) and Hyman et al.Hyman et al. (1995) who also use RSJ dynamics and find evidence of a zero-$`T`$ transition. In addition, Monte Carlo simulations by Fisher et al.Fisher et al. (1991a) and Reger and YoungReger and Young (1993) show evidence of a zero-temperature transition, although the simulations were not taken down to extremely low temperatures.

In this work we perform Monte Carlo simulations of the two-dimensional gauge glass, using parallel tempering Hukushima and Nemoto (1996); Marinari (1998) to go to significantly lower temperatures than was possible in earlier work. In particular we are now able to cover the temperature range where the claimed spin-glass transitionChoi and Park (1999); Kim (2000) takes place. We find strong evidence that $`T_c=0`$.

In addition, we study the gauge glass in three dimensions at very low, but finite, temperatures to provide a good estimate of the stiffness exponent $`\theta `$. The motivation for this is that earlier estimates, which were obtained from ground state methods, are inconsistent. One groupReger et al. (1991); Gingras (1992); Kosterlitz and Simkin (1997); Maucourt and Grempel (1997) finds values consistent with $`\theta 0`$, whereas anotherAkino and Kosterlitz (2002); Cieplak et al. (1992); Moore and Murphy (1994); Kosterlitz and Akino (1998) finds $`\theta `$ in the range $`0.26`$$`0.31`$. Ground state methods involve computing the ground state with two different boundary conditions, and one of the main reasons for the large range of estimates for $`\theta `$ is that different groups made different assumptions about the optimal form of boundary condition changes, see, e.g., Ref. Akino and Kosterlitz, 2002. This difficulty is avoided in the finite-$`T`$ Monte Carlo approach adopted here. We find that $`\theta =0.27\pm 0.01`$, which agrees with the results of the second group of ground state calculations mentioned above.

In Sec. II we introduce the model and observables as well as details regarding the Monte Carlo technique used. Some results from finite-size scaling are discussed in Sec. III. Our results in two and three dimensions are presented in Secs. IV and V, respectively, and the conclusions are summarized in Sec. VI.

## II Model, Observables And Equilibration

The Hamiltonian of the gauge glass is given by

$$=J\underset{i,j}{}\mathrm{cos}(\varphi _i\varphi _jA_{ij}),$$
(1)
where the sum ranges over nearest neighbors on a square lattice in $`D`$ dimensions of size $`N=L^D`$ and $`\varphi _i`$ represent the angles of the $`XY`$ spins. Periodic boundary conditions are applied. The $`A_{ij}`$ are quenched random variables uniformly distributed between $`[0,2\pi ]`$.Katzgraber and Young (2001) Because $`A_{ij}`$ represent the line integral of the vector potential between sites $`i`$ and $`j`$, we have the constraint that $`A_{ij}=A_{ji}`$. In this work we set $`J=1`$.

Traditionally one uses the Binder ratioBinder (1981) to estimate the critical temperature $`T_c`$. In this method one plots the ratio of the fourth moment and the second moment squared of the spin-glass order parameter as a function of temperature for different system sizes $`L`$. The crossing point of all curves identifies $`T_c`$. For the gauge glass the Binder ratio cannot exceed unity,Olson and Young (2000) thus the splaying of the curves is small and $`T_c`$ is difficult to establish.

In order to avoid this problem we use a method introduced by Reger and YoungReger et al. (1991) in which one calculates the current $`I`$ defined as the derivative of the free energy $`F`$ with respect to an infinitesimal twist to the boundaries, i.e.,

$$I(L)=\frac{1}{L}\underset{i}{}\mathrm{sin}(\varphi _i\varphi _{i+\widehat{x}}A_{ii+\widehat{x}}).$$
(2)
In this case the twist is applied along the $`\widehat{x}`$ direction. As the gauge fields $`A_{ij}`$ are uniformly distributed we expect $`[I(L)]_{\mathrm{av}}=0`$, where $`[\mathrm{}]_{\mathrm{av}}`$ represents an average over disorder and $`\mathrm{}`$ represents a thermal average. We calculate the root-mean-square current

$$I_{\mathrm{rms}}=\sqrt{[I_\alpha (L)I_\beta (L)]_{\mathrm{av}}}.$$
(3)
Here $`\alpha `$, $`\beta `$ denote two replicas of the system with the same disorder introduced to avoid any bias in the calculation of the average of the square of the currents. Unlike the Binder ratio, the currents are not restricted to converge to unity for $`T0`$, thus any potential splaying of the data is easily visible. In fact, for $`T<T_c`$ the currents scale as $`I_{\mathrm{rms}}L^\theta `$ where $`\theta `$ is the (positive) stiffness exponent, see Refs. Fisher and Huse, 1987; Huse and Fisher, 1987; Fisher and Huse, 1988 for a general discussion of the stiffness exponent in the context of the “droplet model” of spin glasses, and also Ref. Fisher et al., 1991b which discusses the stiffness exponent specifically in the context of the vortex glass.

Note that the stiffness exponent is also often defined in terms of the free energy change for a finite twist, by $`\pi `$ for example. We are assuming here that any reasonable measure of the stiffness will give the same exponent $`\theta `$, whether it be the free energy change due to a finite twist or, as here, the derivative with respect to an infinitesimal twist. This is consistent with the usual assumptions of universality for phenomena on long length scales, that the same exponent will be obtained for the relation between, say, energy and length scale, for any reasonable definition of energy. Presumably a plot of the free energy against twist angle $`\mathrm{\Theta }`$ will have quite complicated structure, with local regions varying quadratically as a function $`\mathrm{\Theta }`$ and proportional to $`L^{d2}`$ (where only spin-wave excitations are generated), but with “kinks” where which the system changes from one parabola to another due to a vortex changing position. Hence, in effect, we are assuming that the typical magnitude of the slope of this curve is of the same order in $`L`$ as the range of its variation with $`\mathrm{\Theta }`$. This requires that the typical spacing between kinks $`\mathrm{\Delta }\mathrm{\Theta }`$ satisfies $`L^\theta (\mathrm{\Delta }\mathrm{\Theta })^2L^{d2}`$, i.e. $`\mathrm{\Delta }\mathrm{\Theta }L^{(d2\theta )/2}`$.

For a finite-temperature transition, $`T_c>0`$, we expect $`\theta >0`$, since then the ordered state at $`T=0`$ is “stiff” on large scales, and so will presumably resist small thermal fluctuations. On the other hand for a zero-temperature transition, we expect $`\theta <0`$, because the system will then easily break up under the influence of thermal fluctuations.

In order to compare with the results of Choi and ParkChoi and Park (1999) we have also calculated the spin-glass susceptibility $`\chi _{_{\mathrm{SG}}}`$ defined by

$$\chi _{_{\mathrm{SG}}}=N[q^2]_{\mathrm{av}},$$
(4)
where $`q`$ is the spin-glass order parameter defined by

$$q=\frac{1}{N}\underset{j}{\overset{N}{}}\mathrm{exp}[i(\varphi _j^\alpha \varphi _j^\beta )]$$
(5)
and $`\alpha `$, $`\beta `$ are two replicas of the system with the same disorder. This susceptibility diverges at the transition. However, as we shall see in Sec. III, its finite-size scaling behavior is more complicated than that of $`I_{\mathrm{rms}}`$.

For the simulations we use the parallel tempering Monte Carlo methodHukushima and Nemoto (1996); Marinari (1998) as it allows us to study systems at lower temperatures than with conventional methods. Because the equilibration test for short-range spin glasses introduced by Katzgraber et al.Katzgraber et al. (2001) does not work for the gauge glass since the disorder is not Gaussian, we test equilibration by the traditional technique of requiring that different observables are independent of the number of Monte Carlo steps $`N_{\mathrm{sweep}}`$. Figure 1 shows data in 2D for the energy $`E`$, spin-glass susceptibility $`\chi _{_{\mathrm{SG}}}`$, as well as the average current squared $`I_{\mathrm{rms}}^2`$ as a function of Monte Carlo steps. One can clearly see that the different observables saturate at the same equilibration time. We show data for an intermediate size as it allows us to better illustrate the procedure by simulating with much longer runs than are necessary to equilibrate. For large sizes we equilibrate doubling the number of Monte Carlo sweeps between each measurement until the last three agree within error bars.

We also require that the acceptance ratios for the global moves which interchange the different temperatures in the parallel tempering scheme be greater than $`0.3`$ on average and roughly constant as a function of temperature. The equilibration tests for the 3D data have been described elsewhere.Katzgraber and Young (2001)

Tables 1 and 2 show the number of samples $`N_{\mathrm{samp}}`$ and the number of Monte Carlo sweeps $`N_{\mathrm{sweep}}`$ performed by each replica for each lattice size for $`D=2`$ and $`D=3`$, respectively.

In two dimensions the highest temperature is $`1.058`$, whereas the lowest temperature is $`0.13`$ (for $`L=24`$ the lowest temperature is $`0.20`$). The number of temperatures $`N_T`$ is chosen to give satisfactory acceptance ratios for the Monte Carlo moves between the temperatures. In three dimensions the lowest temperature studied is $`0.05`$ \[to be compared with $`T_c0.45`$, (Ref. Olson and Young, 2000)\] whereas the highest temperature is $`0.947`$.

Because the gauge glass has a vector order parameter symmetry, to speed up the simulations we discretize the angles of the spinsKatzgraber and Young (2001); Cieplak et al. (1992) to $`N_\varphi =512`$. To ensure a reasonable acceptance ratio for single-spin Monte Carlo moves, we pick the proposed new angle for a spin within a temperature-dependent acceptance window about the current angle. By tuning a numerical prefactor we ensure the acceptance ratios for these local moves are not smaller than 0.2 for each system size at the lowest temperature simulated.

## III Finite-Size Scaling

We review briefly some salient aspects of finite-size scaling theory, as applied to the present situation, paying particular attention to features which are special to a $`T=0`$ transition. Consider first the singular part of the bulk free energy per spin divided by $`T`$, a quantity with dimension $`(`$length$`)`$<sup>-D</sup>, where $`D`$ represents the space dimension. The basic assumption is that the only relevant lengthscale is the correlation length $`\xi `$, and so $`\beta f_s\xi ^D`$. Consider next a finite-size system where the bulk behavior is assumed to be modified by a function of $`L/\xi `$, where $`L`$ is the lattice size, i.e.,

$$\beta f_s=\xi ^D\stackrel{~}{f}\left(\frac{L}{\xi }\right).$$
(6)
Thus, we can write the scaling of the total singular free energy $`F_s(=L^Df_s)`$ as

$$\beta F_s=\stackrel{~}{F}\left(\frac{L}{\xi }\right)=\widehat{F}(L^{1/\nu }(TT_c)).$$
(7)
The last expression follows because $`\xi (TT_c)^\nu `$ and we have taken the argument of the function $`\stackrel{~}{F}`$ to the power $`1/\nu `$.

As $`I_{\mathrm{rms}}`$ is the derivative of $`F_s`$ with respect to the twist angle and since the twist angle is dimensionless, the scaling of $`I_{\mathrm{rms}}`$ is the same as that as of $`F_s`$, i.e.,

$$\beta I_{\mathrm{rms}}=\widehat{I}[L^{1/\nu }(TT_c)].$$
(8)
Note that we have been careful to maintain the factor of $`\beta `$ in our analysis. However, if $`T_c>0`$, then, since the critical region is close to $`T_c`$, we can replace $`\beta `$ by $`1/T_c`$ and incorporate this constant factor into the scaling function, i.e.,

$$I_{\mathrm{rms}}=\stackrel{~}{I}[L^{1/\nu }(TT_c)](T_c>0).$$
(9)
By contrast, if $`T_c=0`$ we have to keep the variation in $`\beta `$ and so

$`I_{\mathrm{rms}}`$ $`=`$ $`T\widehat{I}(L^{1/\nu }T)`$ (10)
$`=`$ $`L^{1/\nu }\stackrel{~}{I}(L^{1/\nu }T)(T_c=0),`$
where the scaling functions $`\widehat{I}`$ and $`\stackrel{~}{I}`$ are simply related to each other. Equation (10) indicates that the $`T=0`$ stiffness exponent $`\theta `$ is negative and equal to $`1/\nu `$. Equation (9) shows that if $`T_c>0`$ the curves for $`I_{\mathrm{rms}}`$ for different sizes intersect at the critical point, whereas Eq. (10) shows that if $`T_c=0`$, then the data decrease with increasing size at $`T=0`$.

According to standard finite-size scaling the spin-glass susceptibility, defined in Eq. (4), behaves as

$$\chi _{_{\mathrm{SG}}}=L^{2\eta }\stackrel{~}{\chi }_{_{\mathrm{S}G}}[L^{1/\nu }(TT_c)],$$
(11)
which means that at criticality it diverges with a power law, i.e.,

$$\chi _{_{\mathrm{SG}}}L^{2\eta }(T=T_c).$$
(12)
Equations (11) and (12) are valid whether the transition is at $`T_c=0`$ or $`T_c>0`$. The power law prefactor in Eq. (11) with an unknown exponent complicates the analysis of $`\chi _{_{\mathrm{SG}}}`$ compared with that for $`I_{\mathrm{rms}}`$ which does not have such a prefactor if $`T_c>0`$, see Eq. (9). Even if $`T_c=0`$, where a prefactor does arise \[see Eq. (10)\], the exponent in the prefactor is the same as that in the argument of the scaling function, so we do not have an extra exponent to determine.

## IV Results for $`D=2`$

In Fig. 2 we present results for $`I_{\mathrm{rms}}`$ in two dimensions. At all temperatures the data decrease with increasing $`L`$ indicating, from Eq. (9), that $`T_c`$ must be less than the range of temperatures studied. In fact, we shall find it impossible to scale the data for $`I_{\mathrm{rms}}`$ for any finite $`T_c`$, in contrast to the results of KimKim (2000) and Choi and Park,Choi and Park (1999) who find a finite-temperature transition at $`T_c=0.22`$.

In Fig. 3 we show a finite-size scaling plot of the data in Fig. 2 according to Eq. (10). Although the data do not scale well over the whole range, the data at low-$`T`$ do scale quite well, since the plot shows that the data collapse if $`L^{1/\nu }T`$ is small for all values of $`L`$, and over the whole range of $`L^{1/\nu }T`$ for the largest sizes. The data in Figs. 2 and 3 are therefore consistent with a zero temperature transition, but with significant corrections to scaling at intermediate temperatures. We estimate the stiffness exponent to be

$$\theta =0.39\pm 0.03.$$
(13)
The above error bar is estimated by varying $`\theta `$ slightly until the data do not collapse well. This result is consistent with recent work of Akino and KosterlitzAkino and Kosterlitz (2002) who find $`\theta =0.36\pm 0.01`$.

Figure 4 shows the current data according to Eq. (9) assuming the parameters found by Choi and ParkChoi and Park (1999): $`T_c=0.22`$, $`1/\nu =0.88`$. The scaling is very poor especially near the proposed $`T_c`$. By contrast, if the deviations are due to corrections to scaling, they should be smaller near $`T_c`$. Hence our data are incompatible with the claim of KimKim (2000) and Choi and ParkChoi and Park (1999) that $`T_c=0.22`$. In fact, we are unable to get a reasonable fit to the data for $`I_{\mathrm{rms}}`$ according to Eq. (9) for any finite $`T_c`$.

Next we present our data for the spin-glass susceptibility $`\chi _{_{\mathrm{SG}}}`$. Figure 5 is a log-log plot of $`\chi _{_{\mathrm{SG}}}`$ vs $`L`$ at several low temperatures. According to Eq. (12) the data should lie on a straight line at $`T_c`$. However, the data in the vicinity of $`T=0.22`$, the transition temperature claimed by KimKim (2000) and Choi and Park,Choi and Park (1999) show a strong downward curvature, indicating that this is actually above $`T_c`$. Only around the lowest temperature where we have data, $`T=0.13`$, is the curvature small, although it still greatly exceeds the error bars. This indicates that $`T_c<0.13`$, which is compatible with our data for $`I_{\mathrm{rms}}`$.

Figure 6 shows a scaling plot according to Eq. (11) for $`T_c=0`$, and $`1/\nu =0.39`$, the same parameters found in the scaling of $`I_{\mathrm{rms}}`$, together with $`\eta =0`$ which is expected at a zero-temperature transition in two dimensions. The data at low temperatures and for the largest sizes scale well, but the data away from this range show deviations. Allowing $`1/\nu `$ to vary we find $`1/\nu =0.50\pm 0.03`$. The inset shows data for the optimal value, where only the $`L=4`$ and 6 data are not part of the scaling function for all $`L^{1/\nu }T`$. The fact that the best values of $`1/\nu `$ are not precisely the same when obtained from $`\chi _{_{\mathrm{SG}}}`$ and $`I_{\mathrm{rms}}`$ presumably indicates that scaling is only valid for fairly low temperatures and large sizes, and that, despite our working at quite low temperatures, we have only a limited range of data which are fully in the scaling regime.

In Fig. 7 we show the same data as in Fig. 6 scaled with the parameters used by Choi and Park in Ref. Choi and Park, 1999. We note that the quality of the data collapse is poor near $`T_c`$, whereas if the deviations were due to corrections to scaling, we would expect it to be best in this region. We therefore do not consider this scaling to be acceptable. Furthermore, the scaling of $`I_{\mathrm{rms}}`$ shown in Fig. 4 is clearly much worse, indicating that $`I_{\mathrm{rms}}`$ is much better able to distinguish between a finite $`T_c`$ and $`T_c=0`$ than is $`\chi _{_{\mathrm{SG}}}`$ because of its simpler finite-size scaling form.

This is further illustrated in Fig. 8 which shows a scaling plot of $`\chi _{_{\mathrm{SG}}}`$ in which we took $`T_c`$ to be the lowest temperature in our simulations, i.e., $`T_c=0.13`$. The best fit with this $`T_c`$ has $`1/\nu =0.68`$ and $`\eta =0.19`$, where $`\eta `$ is determined by the requirement that the data scale as well as possible at $`T_c`$. By eye, the data scale fairly well, though the deviations are actually significantly greater than the error bars, as we noted in the above discussion of the unscaled $`\chi _{_{\mathrm{SG}}}`$ data at $`T=0.13`$. However, the data for $`I_{rms}`$, scale very badly with $`T_c=0.13`$ as can be seen in the inset of the figure. In fact, as noted above, the data for $`I_{rms}`$ do not scale according to Eq. (9) for any finite $`T_c`$.

The data for $`\chi _{_{\mathrm{SG}}}`$ enable us to determine the exponent $`\eta `$ as well as $`1/\nu `$. Assuming $`T_c=0`$, as obtained from the $`I_{\mathrm{rms}}`$ data, we find $`\eta =0.0\pm 0.1`$ from a scaling plot for $`\chi _{_{\mathrm{SG}}}`$. This is consistent with $`\eta =0`$, as expected at a $`T=0`$ transition in two dimensions, at least if the ground state is not highly degenerate.

## V Results for $`D=3`$

The critical region of the three-dimensional gauge glass has been investigated in detail by Olson and Young.Olson and Young (2000) In their work they obtain a lower bound for the stiffness exponent of $`\theta 0.18`$. Some previous resultsReger et al. (1991); Gingras (1992); Kosterlitz and Simkin (1997); Maucourt and Grempel (1997) find $`\theta `$ in the range $`0\theta 0.077`$ whereas othersAkino and Kosterlitz (2002); Cieplak et al. (1992); Moore and Murphy (1994); Kosterlitz and Akino (1998) find a much larger value, $`0.26\theta 0.31`$.

We can estimate $`\theta `$ from our data for $`I_{\mathrm{rms}}`$ since $`I_{\mathrm{rms}}L^\theta `$ when $`L^{1/\nu }(TT_c)`$, the argument of the scaling function in Eq. (9), tends to infinity. The data are shown in Fig. 9, in which one observes that the slopes in the log-log plot vary somewhat with temperature. Asymptotically, however, one expects the same $`L^\theta `$ behavior for all $`T<T_c`$, so the deviations from a constant slope presumably indicate that that the data are not yet at sufficiently large sizes and low enough temperature to be in the asymptotic region. To obtain an estimate of $`\theta `$ we therefore perform a linear least-squares fit of $`\mathrm{ln}I_{\mathrm{rms}}`$ against $`\mathrm{ln}L`$ for each temperature in order to obtain an effective stiffness exponent $`\theta _{\mathrm{eff}}(T)`$ which depends on the temperature. Figure 10 shows that $`\theta _{\mathrm{eff}}(T)`$ can be fitted well to a linear form at low temperatures. Extrapolating to $`T=0`$ we obtain

$$\theta =0.27\pm 0.01$$
(14)
which is clearly positive and consistent with the results of Refs. Olson and Young, 2000; Akino and Kosterlitz, 2002; Cieplak et al., 1992; Moore and Murphy, 1994; Kosterlitz and Akino, 1998.

## VI Conclusions

We have shown results from Monte Carlo simulations at low temperatures of the two- and three-dimensional gauge glass. In two dimensions our data for $`I_{\mathrm{rms}}`$ are consistent with a zero-temperature transition with $`\theta =0.39\pm 0.03`$. However, it is necessary to go to quite low temperatures and large sizes to see the expected scaling. Our data are incompatible with the prediction made by KimKim (2000) and Choi and ParkChoi and Park (1999) that $`T_c=0.22`$. The error bar we quote is purely statistical and systematic corrections to scaling can increase this. Indeed, our best estimate for $`1/\nu \theta `$ from the $`\chi _{_{\mathrm{SG}}}`$ data is $`0.50\pm 0.03`$, which differs from the value of $`1/\nu `$ from $`I_{\mathrm{rms}}`$ by more than the (statistical) error bars. However, our claim that $`T_c=0`$ is very robust and is not affected by corrections to finite-size scaling.

In three dimensions we report the first reliable estimate of the stiffness exponent from finite-temperature Monte Carlo simulations. Monte Carlo has the advantage over ground state methods that it computes the stiffness directly and is free from the difficulty of determining the optimal boundary condition changes, which has been quite controversial for the ground state approach. We find $`\theta =0.27\pm 0.01`$, which agrees with the results of Refs. Olson and Young, 2000; Akino and Kosterlitz, 2002; Cieplak et al., 1992; Moore and Murphy, 1994; Kosterlitz and Akino, 1998.

###### Acknowledgements.

We would like to thank C. Dasgupta, R. T. Scalettar, P. Sengupta and A. Slepoy for fruitful discussions, and R. T. Scalettar and T. Olson for comments on an earlier version of the manuscript. HGK acknowledges support from the National Science Foundation under Grant No. DMR 9985978. APY acknowledges support from the National Science Foundation under Grant No. DMR 0086287, and the EPSRC under Grant No. GR/R37869/01. He would also like to thank David Sherrington for hospitality during his stay at Oxford. This research was supported in part by NSF cooperative agreement ACI-9619020 through computing resources provided by the National Partnership for Advanced Computational Infrastructure at the San Diego Supercomputer Center and the Texas Advanced Computing Center. We would like to thank the University of New Mexico for access to their Albuquerque High Performance Computing Center. This work utilized the UNM-Alliance Los Lobos Supercluster. We would also like to acknowledge computer time on the UC Davis Undergraduate Computing Cluster and the UCSC Physics Graduate Computing Cluster funded by the Department of Education Graduate Assistance in the Areas of National Need program.