Observational Implications of Lowering the LIGO-Virgo Alert Threshold

The 2017 August detection of GW170817 was an event of many firsts. Not only was it the first binary-neutron-star (BNS) merger detected (Abbott et al. 2017e) by the Laser Interferometer Gravitational-Wave Observatory (LIGO)-Virgo detector network (Aasi et al. 2015; Abbott et al. 2016c; Acernese et al. 2015), it was the first gravitational-wave (GW) event confidently detected by both ground-based GW detectors and electromagnetic (EM) observatories (Abbott et al. 2017f). While the detection of GW170817 could be confidently established by GW-detector data alone, the joint EM detection enabled a vast array of rich physical insights, such as the association of short gamma-ray bursts with BNS mergers (Abbott et al. 2017c), a new procedure for constraining the value of the Hubble parameter H₀ (Abbott et al. 2017a), and evidence of heavy-element nucleosynthesis (Arcavi et al. 2017; Coulter et al. 2017; Lipunov et al. 2017; Soares-Santos et al. 2017; Tanvir et al. 2017; Valenti et al. 2017). The high signal-to-noise ratio (S/N) of GW170817 and the clarity with which it could be distinguished in both GW and EM data aided the wealth of scientific information extracted by studying it. However, given an astrophysical (uniform-in-volume) population of such sources, we expect that quieter GW candidates might also make a non-negligible scientific contribution to both the GW and EM communities. For example, if we were able to double the number of joint GW–EM detections by performing searches for low-significance candidates, we could decrease the uncertainty in GW-based measurements of H₀ by up to a factor of $\tfrac{1}{\sqrt{2}}$ (although the actual improvement may be lower on account of poor distance localization; Chen et al. 2017a).

In this Letter, we examine the extent to which searches for low-significance GW transients can augment the total ensemble of GW detections. For the purpose of establishing a baseline, let us assume that the minimum false-alarm rate (FAR) at which GWs can be confidently detected by LIGO-Virgo alone is 1 per 100 years. This FAR corresponds to the nominal LIGO-Virgo alert threshold proposed for issuing open public alerts in the third Advanced LIGO-Virgo observing run (Public LIGO document 2018). In effect, we will define any GW event with a FAR of greater than 1 per 100 years to be a low-significance event. Under this assumption, we cannot claim low-significance LIGO-Virgo events as confident detections unless they are jointly detected by EM observations at a convincing significance. In a sense, this method is the complement to the scenario where GW events with extremely small localization volumes enable the discovery of faint EM counterparts (Chen & Holz 2016).

However, there are potential opportunity costs that EM observers must weigh when considering how many GW candidates they can reasonably follow up. By definition, following up GW candidates at a higher FAR threshold means a greater number of false-alarm contaminations. Furthermore, low-significance candidates are inherently faint in GW detectors, implying that they may not be well-localized for EM observations. The combination of these two factors, along with finite observational resources, suggests that each EM follow-up campaign should determine its own GW-alert-purity threshold that optimizes its scientific goals. In the remainder of this Letter, we quantify the ingredients necessary for calculating these purity thresholds. This analysis is complementary to work optimizing EM follow-up using tiling, time allocation, and scheduling methods, e.g., (Coughlin & Stubbs 2016; Ghosh et al. 2016; Chan et al. 2017; Rana et al. 2017; Salafia et al. 2017).

All search algorithms for transient GW events follow the same basic hypothesis: the signatures of GW events in every GW detector should be morphologically identical (once projection effects are taken into account) and time coincident, while detector noise need not be. The noise in each GW detector is a superposition of a Gaussian bulk and non-Gaussian noise transients. With low probability, this noise can mimic the appearance of GW events, which forms a background for the various search algorithms. One such search algorithm is PyCBC (Usman et al. 2016; Nitz et al. 2018), which uses a bank of compact-binary-coalescence templates to rank GW detection candidates according to a network ranking statistic, ρ, that combines the candidate S/N with signal consistency tests (Allen 2005; Babak et al. 2013). The main results of this Letter are all extensions of the following points: (1) background event rates fall off exponentially as a function of ρ, while (2) GW event rates fall off less steeply as a power law.

The bulk of the background distribution of searches for GW transients falls off steeply as a function of ρ, meaning the FAR changes by orders of magnitudes over narrow ranges of ρ. To quantify this more precisely, we explore the background for the two LIGO detectors: one in Hanford, Washington, USA (H), and the other in Livingston, Louisiana, USA (L). The HL background rate (i.e., the FAR) decays roughly exponentially as a function of ρ for FARs between 1 per 100 years and 1000 per year. Thus, we can model the FAR as

$$\begin{eqnarray}&&\mathrm{FAR}={\mathrm{FAR}}_{8}\times \exp \left[-\displaystyle \frac{(\rho -8)}{\alpha }\right]\end{eqnarray} \tag{ 1 }$$

where FAR₈ is the FAR at ρ = 8, and α is scale-parameter that determines the steepness of the falloff. This behavior is observed both for searches for compact binaries (Abadie et al. 2012; Nitz et al. 2017) and searches for short-duration unmodeled GW events (Abbott et al. 2017b). The steepness of this exponential falloff is determined by how easy it is for background events to mimic GWs in a given search. Thus, searches for BNS events have a steeper falloff (smaller α) than searches for binary-black hole (BBH) events, because a known time-frequency evolution is observed over longer durations for BNS events than for BBH events. Likewise, searches for short-duration unmodeled GW events have less-steep exponential falloff than for either BNS or BBH events, because the former's time-frequency evolution is inherently unknown and thus less constrained.

For this Letter, we focus only on searches for BBH and BNS events (Abbott et al. 2016a, 2016b, 2017e), as both of these source types have already been detected by LIGO-Virgo. These detections have allowed for the sources' rates to be observationally established (Abbott et al. 2016a, 2016d, 2016e, 2017d, 2017e). As mentioned, BNS events have already been jointly detected by LIGO-Virgo and EM observers (Abbott et al. 2017f), making them the most anticipated targets for low-significance efforts. We do not present the results for short-duration unmodeled events, as we do not have any direct measurements of their rate. However, the relative results, obtained by normalizing out these unknown rates, are of similar magnitude to those for both BNS and BBH sources, resembling the results for BBH sources more closely.

We perform the exponential fit using the FAR versus ρ relationship reported in Nitz et al. (2017), which represents the PyCBC search background for the HL data during the first Advanced LIGO-Virgo observing run. The results of this fit are shown in Figure 1. For BBH we find α = 0.18 and ${\mathrm{FAR}}_{8}=5500\,{\mathrm{yr}}^{-1}$, while for BNS we find α = 0.13 and ${\mathrm{FAR}}_{8}={\rm{30,000}}\,{\mathrm{yr}}^{-1}$. We assume that the slopes of these fits are representative of the BBH and BNS HL searches in current and future observing runs. This assumption is based upon empirical results: we observe similar fits in published results for both Advanced (Nitz et al. 2017) and initial (Abadie et al. 2012) LIGO-Virgo observing runs. As we will soon see, the potential impact of searches for low-significance GW events is reduced as the slope of the searches' background distributions becomes steeper. Thus, any improvements that make the GW searches' backgrounds less heavily tailed (and hence steeper), such as adding a third detector like Virgo to potentially reduce the coincidence rate of high-ρ noise transients, may further reduce the case for low-significance GW science.

Zoom In Zoom Out Reset image size

We must likewise find a model to describe the rate of GW events versus ρ. Assuming that GW events are distributed uniformly in volume, the cumulative rate of GW events exceeding an S/N threshold should roughly scale as S/N⁻³ for Advanced-era GW detectors probing the low-redshift universe (Schutz 2011; Chen & Holz 2014; Vitale 2016; although this scaling relation will break down for third-generation GW detectors that probe higher redshifts (Vitale 2016)). By construction, we expect that the network ranking statistic ρ ∼ S/N for real GW events (Nitz et al. 2017). As the Advanced GW detectors improve in sensitivity, the overall rate of GW events being observed will increase accordingly. We can estimate the rate of GW events for a given observing epoch as

$$\begin{eqnarray}&&\mathrm{GW}\ \mathrm{Event}\ \mathrm{Rate}=\langle V\rangle \langle R\rangle \end{eqnarray} \tag{ 2 }$$

where $\langle R\rangle$ is LIGO-Virgo's empirically motivated rate-density estimate (Abbott et al. 2016a, 2016d, 2016e, 2017d, 2017e), and $\langle V\rangle$ is the average sensitive volume of the epoch's GW search. We estimate the cosmologically corrected sensitive volume for each epoch at an S/N threshold of 8, $\langle {V}_{8}\rangle$, using the online distance calculator provided by (Chen et al. 2017b), so that the average sensitive volume at a given ρ is given by

$$\begin{eqnarray}&&\langle V\rangle =\langle {V}_{8}\rangle \times {\left(\displaystyle \frac{\rho }{8}\right)}^{-3}\end{eqnarray} \tag{ 3 }$$

The three different observing epochs we consider are (Abbott et al. 2016c): the second Advanced LIGO-Virgo observing run (O2), the third Advanced LIGO-Virgo observing run (O3), and the eventual design sensitivity Advanced LIGO-Virgo observing runs. For BBH searches, we consider both uniform-in-log and power-law (with a power of −2.35) source-mass distributions (Abbott et al. 2016a, 2016d, 2016e, 2017d) when estimating $\langle V\rangle$ and $\langle R\rangle$. The differences in the results for these two distributions are negligible, thus we only quote the power-law results in this Letter. For the power-law distribution, we use $\langle R\rangle ={103}_{-63}^{+110}\,{\mathrm{Gpc}}^{-3}\,{\mathrm{yr}}^{-1}$ (Abbott et al. 2017d), and averaging over the sensitive volumes of the mass distribution we find $\langle {V}_{8}{\rangle }_{{\rm{O}}2}=0.22\,{\mathrm{Gpc}}^{3}$, $\langle {V}_{8}{\rangle }_{{\rm{O}}3}=0.66\,{\mathrm{Gpc}}^{3}$, and $\langle {V}_{8}{\rangle }_{\mathrm{Design}}=2.3\,{\mathrm{Gpc}}^{3}$. For BNS, we use the median total mass estimate of 2.8 M_⊙ for GW170817 (Abbott et al. 2017e) when estimating $\langle V\rangle$ and $\langle R\rangle$. We use $\langle R\rangle ={1540}_{-1220}^{+3200}\,{\mathrm{Gpc}}^{-3}\,{\mathrm{yr}}^{-1}$ (Abbott et al. 2017e) and $\langle {V}_{8}{\rangle }_{{\rm{O}}2}=0.002\,{\mathrm{Gpc}}^{3}$, $\langle {V}_{8}{\rangle }_{{\rm{O}}3}\,=0.007\,{\mathrm{Gpc}}^{3}$, and $\langle {V}_{8}{\rangle }_{\mathrm{Design}}=0.03\,{\mathrm{Gpc}}^{3}$.

Combining Equations (1)–(3), we compute the expected rate of GW events at each FAR threshold. In Figure 1, we show the fractional increase in GW events expected below each FAR threshold as compared to the baseline FAR threshold of 1 per 100 years. The most notable result is that increasing the FAR threshold by five orders of magnitudes from 1 per 100 years to 1000 per year only increases the number of detectable GW events by approximately a factor of 1.6 for BNS and 1.9 BBH. In other words, because the GW event rate scales as ρ⁻³, we need to change the ρ threshold by a factor of ${\left(\tfrac{1}{2}\right)}^{\tfrac{1}{3}}\sim 0.8$ to gain a factor of 2 in the number of detectable GW events. However, changing ρ from ∼10 (corresponding to a FAR threshold of 1 per 100 years) by a factor of 0.8 increases the FAR contamination by more than five orders of magnitude (see Figure 1). We emphasize that in practice these numbers only represent the increased number of real GW events as candidates but not necessarily as detections. To claim any of these additional low-significance GW events as confident detections, they would need to be jointly detected by EM observations. Thus, the actual increase in the total number of confident GW detections may be lower than this factor of two depending on the EM detection efficiency.

Because the uncertainties in the GW event rate amount to a constant normalization factor (see Equations (2) and (3)), they do not factor into Figure 1. The only uncertainties that affect this plot are therefore related to the background fit (the following applies to the area/distance/volume localizations discussed in Section 4 as well). We manually vary the normalization of the total background rate, FAR₈, by up to an order of magnitude, however this only results in negligible uncertainties in the expected rate of detectable GW events. The effect of the uncertainty regarding the slope, α, of our exponential background fits is illustrated by comparing the results for the steeper BNS background to those for the less-steep BBH background. The relative increase in the rate of detectable GW events is greater for BBH searches than for BNS searches because the range of ρ spanned at these FARs is greater for BBH searches (see Figure 1). Nevertheless, the results of both of these searches are of similar magnitude across all FARs. Thus, we would only expect the numbers in Figure 1 to change significantly if the backgrounds for any search were to become drastically more or less heavily tailed.

Although it is interesting to explore the fractional increase in GW event rate associated with each FAR threshold, we must take into account that these fractional increases come with absolute costs to observers. The quantity of interest when issuing alerts to EM astronomers is the fraction of alerts that are actual GW events. The GW community quantifies this alert purity by calculating the probability that a GW event is of astrophysical origin, P_astro. P_astro is measured by comparing the differential rates of GW and background events at a given value of a search statistic (Farr et al. 2015; Abbott et al. 2016a, 2016d, 2016e). This quantity has been produced by LIGO-Virgo in offline analyses (Abbott et al. 2016a, 2016e, 2017d); however, it may be difficult to provide in low-latency because its proper calculation requires a careful measurement of and/or marginalization over the searches' background and GW event rates. For ease, we approximate the average P_astro that we would expect to observe over all events exceeding a given threshold of ρ as

$$\begin{eqnarray}&&\langle {P}_{\mathrm{astro}}\rangle =\displaystyle \frac{\langle V\rangle \langle R\rangle }{\mathrm{FAR}+\langle V\rangle \langle R\rangle }\end{eqnarray} \tag{ 4 }$$

where FAR and $\langle V\rangle$ can be given in terms of ρ using Equations (1) and (3), respectively. In Figure 2, we plot $\langle {P}_{\mathrm{astro}}\rangle$ versus the FAR associated with the ρ threshold using the predicted sensitivities for the O2, O3, and design sensitivity observing epochs. We depict the uncertainties associated with the GW event rate as the shaded regions, while plotting the results for the median published rates as lines. We give the explicit values of some of these probabilities (corresponding to the median published rates) at several ad hoc FAR thresholds in Table 1.

Zoom In Zoom Out Reset image size

Table 1.  The Average Probability (in %) of GW Alerts Being Actual GW Events ($\langle {P}_{\mathrm{astro}}\rangle$), as Depicted in Figure 2, at Several ad hoc FAR Thresholds for Various LIGO-Virgo Observing Epochs

Epoch	1 per 100 years	1 per year	1 per month	1 per week	1 per day
$\langle {P}_{\mathrm{astro}}\rangle$ O2 BBH	99	93	56	24	5
$\langle {P}_{\mathrm{astro}}\rangle$ O2 BNS	99	66	15	4	1
$\langle {P}_{\mathrm{astro}}\rangle$ O3 BBH	99	98	79	49	13
$\langle {P}_{\mathrm{astro}}\rangle$ O3 BNS	99	87	39	13	2
$\langle {P}_{\mathrm{astro}}\rangle$ Design BBH	99	99	93	77	35
$\langle {P}_{\mathrm{astro}}\rangle$ Design BNS	99	97	73	40	9
FIGW BBH	1.0	1.3	1.5	1.6	1.8
FIGW BNS	1.0	1.2	1.3	1.4	1.6

Note. We also give the corresponding fractional increase of GW events (FI_GW) for these same thresholds. These values correspond to the median published rates for BBH and BNS events (Abbott et al. 2017d, 2017e). Note that the errors on these probabilities corresponding to rate uncertainties can be large (see Figure 2). For O2, the probability of a candidate being a GW could degrade by more than an order of magnitude for higher FAR thresholds, while still not doubling the number of GW events. This degradation is less severe as LIGO-Virgo improves and reaches O3 and design levels of sensitivity, suggesting that alert thresholds based on $\langle {P}_{\mathrm{astro}}\rangle$ provide more optimal and consistent information for EM follow-up than thresholds based on FAR.

Download table as:  ASCIITypeset image

At low FAR thresholds (like our baseline of 1 per 100 years), we have very high alert purity, meaning that there is a great likelihood of success to offset any observational costs incurred to EM observers. At higher FAR thresholds, the purity is strongly dependent upon the expected GW event rate, i.e., the sensitivity of the detectors. In O2, LIGO-Virgo observed relatively low GW event rates, meaning high-FAR GW alerts had relatively low probabilities of being real GW events. However, in more sensitive observing epochs, such as when Advanced LIGO-Virgo reaches design sensitivity, the expected event rate is large enough that even high-FAR GW alerts can have a high purity. For example, assuming the median event rates (Abbott et al. 2017d, 2017e), we would have needed a FAR threshold of two per year for BNS alerts in O2 to have a $\langle {P}_{\mathrm{astro}}\rangle$ of 50%, while at design sensitivity we could instead have a FAR threshold of 30 per year. Thus, we argue that alert thresholds based on FAR do not convey consistent purity information to EM observers. The more useful and consistent information needed by astronomers during their cost-reward analysis of follow-up thresholds is a measure of P_astro that explicitly balances GW event rates against noise rates. A deeper discussion of the statistical motivation and consequences of thresholding with P_astro can be found in (Farr et al. 2015). In the absence of explicit estimates of P_astro, Figure 2 can be used to map an alert's FAR into estimates of $\langle {P}_{\mathrm{astro}}\rangle$.

In Section 3, we describe how raising the FAR threshold decreases the purity of LIGO-Virgo alerts. However, there is an additional cost associated with low-significance GW events that may directly impact their EM follow-up: events are more poorly localized by the GW detectors as their S/N decreases. A simple Fisher matrix analysis (Cutler & Flanagan 1994) shows that the uncertainty in GW distance estimates, σ_D, roughly scales as ${\sigma }_{D}\propto {\rm{S}}/{{\rm{N}}}^{-2}$ (Fairhurst 2017). More in-depth calculations can be used to show that the angular area uncertainty in GW localization estimates, σ_A, roughly scales as ${\sigma }_{A}\propto {\rm{S}}/{{\rm{N}}}^{-2}$ (Wen & Chen 2010), which agrees with the findings of Monte Carlo studies (Berry et al. 2015). Thus, applying an additional factor of distance squared ($\propto {\rm{S}}/{{\rm{N}}}^{-2}$) to convert the angular area uncertainties to proper area uncertainties, we expect the total uncertainty in GW localization volume, σ_V, roughly scales as ${\sigma }_{V}\propto {\rm{S}}/{{\rm{N}}}^{-6}$, which again agrees with the findings of Monte Carlo studies (Del Pozzo et al. 2018).

We again assume that the network ranking statistic ρ ∼ S/N for all low-significance GW alerts. In Figure 1, we plot the fractional increase in the GW distance, angular area, and volume localizations for threshold events, as compared to the localization at our baseline FAR of 1 per 100 years. Similarly to the results for GW event rates (which also scales as a power law in ρ), we find that the relative increase in localization is a slowly varying function of FAR. Increasing the FAR threshold by five orders of magnitude from 1 per 100 years to 1000 per year degrades the distance and angular area localizations by less than a factor of 2 and the volume localization by less than a factor 4. The discussion of the uncertainty in these numbers is identical to the discussion for GW event rate in Section 2 above. Thus, we do not expect low-significance follow-up efforts to experience order-of-magnitude increases in localization costs as compared to those of current threshold events.

We will now discuss how these observational costs will realistically affect EM follow-up. Here we focus specifically on follow-up procedures, although it should be noted that a serendipitous coincident detection of GW candidates with high-energy telescopes like Fermi/Gamma-Ray Burst Monitor (GBM), INTErnational Gamma-Ray Astrophysics Laboratory/Spectrometer for INTEGRAL-Anti-Coincidence Shield (INTEGRAL/SPI-ACS), Konus/WIND (Abbott et al. 2017c) may affect the significance of GW candidates (Blackburn et al. 2015). These instruments have the advantage of continually monitoring a large fraction of the high-energy sky, meaning that they do not need to be run in follow-up mode. Additionally, they are usually subject to backgrounds that overall are quieter than the corresponding ones in optical bands. Thus, they present a low-cost means of potentially increasing the significance of GW events in near real time (as was the case with GW170817; Abbott et al. 2017f).

The EM follow-up observations of GW counterparts are undertaken in stages. Transients detected by imaging systems are assessed by spatial location (either two- or three-dimensional), broadband spectral characteristics, and light curve temporal evolution. These assessments can be accomplished with 2–4 m aperture telescopes. If a viable EM counterpart is detected, large-aperture (8–10 m class) spectroscopic observations are obtained.

We consider the impact of a higher FAR and poorer localization on EM follow-up efforts in three regimes:

• 1.  
  Wide-field surveys, for which follow-up observations amount to re-ordering the sequence in which regions of the sky are observed;
• 2.  
  Galaxy-targeted or other narrow-field imaging programs, which search for transients consistent with GW counterparts; and
• 3.  
  Large-aperture spectroscopic follow-up campaigns, which obtain spectra of individual sources of interest.

4.1. Wide-field Sky Surveys

4.2. Targeted Imaging Observations

4.3. Large-aperture Spectroscopy

The background in GW searches for binary systems with LIGO-Virgo falls off exponentially steeply as a function of the detection statistic, while the GW event rate only falls off as a power law. As a result, lowering the detection statistic enough to double the number of expected GW detections would inflict an increase of greater than five orders of magnitude in the false-alarm contamination. However, the FAR is not particularly well-suited for determining the scientific merits of follow-up efforts, as it contains no information about the expected rate of GW events during the observing epoch. Instead, a quantity that compares the relative rates of GW events and false alarms, such as the probability of an alert being astrophysical (P_astro), is the more appropriate metric for setting consistent follow-up thresholds. If LIGO-Virgo were to provide estimates of P_astro for alerts in real time, the observing facilities could then threshold on it directly. In lieu of such information, estimates of P_astro, such as $\langle {P}_{\mathrm{astro}}\rangle$ described in this Letter, could be used as a proxy.

Optimistically, the resources are available for the joint GW–EM community to roughly double the number of GW detections by following up low-significance GW candidates with EM observations. Nevertheless, the trade-off between the scientific value of these additional detections and the effort required to obtain them is not immediately or universally clear. The joint GW–EM community should always attempt to maximize the scientific output of its observational efforts. In the age of multi-messenger astronomy, it is now in the enviable position of having to quantify the point of diminishing returns for additional low-significance detections.

The authors acknowledge the support of the National Science Foundation and the LIGO Laboratory. LIGO was constructed by the California Institute of Technology and Massachusetts Institute of Technology with funding from the National Science Foundation and operates under cooperative agreement PHY-0757058. M.C. is supported by the David and Ellen Lee Postdoctoral Fellowship at the California Institute of Technology. We would also like to thank Hsin-Yu Chen, Kwan Yeung Ng, Yiwen Huang, Robert Eisenstein, Satya Mohapatra, Steve Drasco, Peter Shawhan, Alex Nitz, Tito Dal Canton, Christopher Berry, Thomas Dent, Reed Essick, and the LIGO-Virgo EM follow-up working group for useful comments and discussion. C.W.S. is grateful to Harvard University and to the US Department of Energy (through grant DE-SC000788) for their support of this effort. This is LIGO document number P1800042.