Sample Forward-Model Simulation and Mock Data

Our simulation bypasses the generation and analysis of light-curve data by directly gen- erating simulated observations of the {m_B,z,x₁,c} values assigned to each SN Ia. We assume a survey consisting of the union of two samples: a highly complete low-redshift sample (S1:ztrue≤0.1), and a high-redshift sample (S2: 0.05_.ztrue_.1.2) modeled after the DES-SN.

Color and stretch distribution shapes, and the selection function for SNe Ia from S1

andS2, are loosely based on 5,000 SNe Ia simulated in the conditions of the first DES-SN observing season, generated by the SuperNova ANAlysis package (SNANA; Kessler et al., 2009b) and its implementation of SALT2. Details of the DES-SN can be found in Kessler et al. (2015) and a description of the Dark Energy Camera (DECam) can be found in Flaugher et al. (2015). The DES-SN observes ten 3 deg2fields (eight “shallow” and two “deep”), roughly once per week, in thegrizfilters. The “shallow” fields are observed to an average depth of 23.5, while the “deep” fields are observed out to an average depth of 24.5 in each of thegrizbands (Kessler et al., 2015).

In what follows we detail the procedure for simulating the observed redshift, color, stretch, and peakB-band magnitude distributions fromθ andψ. Most of the procedure

is identical forS1andS2and performed for both populations. We assume the sizes ofS1

andS2are fixed such thatS1is 10% the size ofS2, i.e., the total number of SNe Ia in the population is 1.1 times the number of SNe Ia inS2. Variables drawn from the respective populations are denoted with the subscriptsS1 andS2.

Step 1: Draw the redshift distribution.

S1: Drawztrue_S1 ∼ U(0,0.1).

S2: Drawztrue_S2 from R_z= dN

dz = (

2.65×10−5(1+z)γ_{,0.05≤z≤1.0}

7.35×10−5,z>1.0 , (4.38) where the rate is given in units of SNe Mpc−3h3₇₀yr−1. Hereγ is a parameter typi-

cally between 1 and 3.8

Step 2: Drawxtrue₁ andctruefrom Eqs. 4.31 - 4.34 usingψ.

Step 3: Drawmtrue_B such that

mtrue_Bi ∼ N(M+∆M0+µ(ztrue_i ,Ωm,w0,H0) +α×x₁true_i +β×ctrue_i ,σ_int2 _mB) (4.39)

Step 4: Apply selection criteria to determine which SNe Ia will remain in the observed sample.

S1: We assume all of the low-redshift SNe Ia(z≤0.1)pass selection cuts and so we do not apply any additional selection.

S2: A model for selection that mimics the assignment of scarce spectroscopic fol- lowup time is applied. As in Section 4.4.2, we assume the f_j brightest objects are selected in each redshift bin. The selection function utilized for this example is described in Table 4.8.

8_{The exact form of this rate was chosen via private communication with R. Kessler, but is roughly based}

Table 4.8. Bins and selection fractions for example SN Ia selection function

Lower Bin Value fj ztrue=0.0 1.0 ztrue=0.1 0.95 ztrue=0.3 0.9 ztrue=0.5 0.8 ztrue=0.6 0.7 ztrue=0.7 0.6 ztrue=0.8 0.3

Step 5: Compile the remaining observed variables that pass selection cuts,mobs_B ,xobs₁ ,cobs, andzobs, from bothS1andS2for a full-forward model of the data.

Following this procedure, we generate a mock data set of 667 spectroscopically- confirmed SNe Ia using the parameter values listed in Table 4.9, and denote this asD3. To generate a mock data set of this size, we draw an initial high-redshift population of 1,200 SNe Ia and a low-redshift population of 120 SNe Ia; roughly 50% of the population remain in the observed sample after selection. We remind the reader that our mock data set is derived using the same model used in the simulations for the purpose of method validation. We also note that we keep track of how many SNe Ia are simulated and how many are selected so we can evaluate the binomial factor in the likelihood (Eq. 4.26).

The true and observed 1-D marginalized distributions ofD3are shown in Figure 4.16. As expected, the observed set of SNe Ia are bluer (more negative values ofc) and brighter (more positive values ofx1) than the population. The mean color shifts from 0.003 (true)

0.0 0.2 0.4 0.6 0.8 1.0 1.2 z 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 p( z) 7.5 10.0 12.5 15.0 17.5 20.0 22.5 25.0 mB 0.00 0.05 0.10 0.15 0.20 0.25 p( mB ) True Obs 3 2 1 0 1 2 x1 0.0 0.1 0.2 0.3 0.4 0.5 p( x1 ) 0.2 0.1 0.0 0.1 0.2 0.3 c 0 1 2 3 4 5 p( c)

Figure 4.16: 1-D marginalized distributions of the true (blue) and observed (orange) SN Ia light-curve parameters and redshifts inD3. The selection function used to construct the observed sample is given in Table 4.8.

Table 4.9. Parameter Values for Mock Data Parameter Input Value

Ωm 0.3 w₀ −1.0 γ 1.5 α −0.14 β 3.2 ∆M0 0.0 σintmB 0.15 ξx 0.5 ξc −0.05 ωx 1.0 ωc 0.1 φx −1.0 φc 1.0

4.5.2

Results

As in Section 4.4.2.1, we adjust theemceescale factor toa=1.2 to ensure proper chain mixing near the peak of the posterior region. The simulation of 100,000 SNe Ia and evaluation of the log likelihood at each point proposed by the sampler takes ≈2s. We use 80 walkers, over 20 compute cores, and run the chains for≈1.5 days. We find the MCMC chains take much longer to converge in this case, with typical autocorrelation times ofτ ≈80−100 for the twelve parameters of interest. Convergence of the MCMC

chain requires≈250,000 samples, including burn in.

Results usingD3are presented in Figure 4.17 and Table 4.10. For this particular real- ization, we recover each of the parameters inθ andψskewnorm within the 1σ uncertainty

regions. Many of the recovered posteriors are roughly Gaussian, despite our use of wide uniform priors. Skewness of the color and stretch are particularly difficult to constrain; thecsposterior is strongly non-Gaussian and skewed right. We recoverw0=−1.04+₋0₀._.085₀₈₀

Table 4.10. Best Fit Parameter Estimates UsingD3

Parameter Prior Input Value Best Fit σ− σ+

w0 U(−1.5,−0.6) −1.0 −1.04 0.08 0.085 Ωm U(0.1,0.6) 0.3 0.303 0.024 0.019 x_m∗ — −0.064 −0.024 0.046 0.061 x_v∗ — 0.682 0.642 0.049 0.056 xs∗ — −0.137 −0.144 0.06 0.043 c_m∗ — 0.006 0.009 0.006 0.005 c_v∗ — 0.007 0.007 0.001 0.001 c_s∗ — 0.137 0.088 0.081 0.112 ∆M0 U(−0.1,0.1) 0.0 −0.013 0.02 0.022 α U(−0.25,−0.05) −0.14 −0.139 0.008 0.013 β U(2,4) 3.2 3.195 0.075 0.059 σint U(0,0.25) 0.15 0.156 0.007 0.006

∗_{Note that priors are imposed on the hyperparameters and not the derived} parameters presented here. We impose wide uniform priors on the width of the latent stretch and color distributions informed by the widths of the observed distributions: ωx∼ U(0,2σ_xobs

1 ),ωc∼ U(0,2σcobs).

uncertainties, errors onΩmandw0are idealized and should not be compared to previous

measurements.

Given the observed parameter biases in Section 4.4.2, we expect to see parameter biases in the SN Ia cosmology model; how this bias will manifest, however, is unclear. We believe we must fully understand the source of bias presented in Section 4.4.2 before assessing biases in the cosmological model. Therefore, we present five realizations of the mock data as illustrative examples of cosmological parameter inference.

As we are primarily interested in cosmological parameter biases,w₀andΩmcontours inferred using each of the data realizations are displayed in Figure 4.18. We are able to recover bothΩm andw0within their respective 1σ uncertainty regions in each of the re-

Figure 4.17: 1σ and 2σ uncertainty regions for the twelve parameters inθ andψskewnorm.

Yellow stars indicate the true values used in the simulation input. Priors and best fit results for each of the parameters are listed in Table 4.10.

alizations. However, it appears that the median of the 1-D marginalized w₀ posterior is systematically smaller than the true value and that the median of the 1-D marginalized Ωmposterior is systematically larger than the true value. In a cursory analysis of the re- gression and hyperparameters, we find the skewness of the color and stretch distributions also appear to be systematically smaller than the truth, and that the intrinsic scatter is systematically larger than the truth. Of course, from these five realizations, we are un- able to determine if these perceived biases are a manifestation of the bias exhibited in Section 4.4.2 or just random fluctuations from limited statistics.

Figure 4.18: Contours of w₀ and Ωm posteriors using five realizations of mock data. Filled contours represent 1σ and 2σ uncertainty regions; the yellow star indicates the

true parameter values. Uncertainty regions inferred fromD3are shown in purple.

4.5.3

Conclusions

Using a simple SN Ia regression model and a mock realization of 667 observed SNe Ia, we have shown that BAMBIS can recover the input simulation parameters within their 1σ uncertainties, where cs and xs are the most difficult parameters to constrain. As we

are aware of an inherent bias in the algorithm (demonstrated in Section 4.4.2), we only realize five sets of mock SN Ia data for illustrative purposes. In these realizations, there appears to be a systematic bias inw₀in that the median of the inferred posterior is lower than the true value; a systematically larger values of Ωm are also observed. However, we cannot assess the significance of these biases without further testing. As discussed in Section 4.4.2.2, we will investigate if such bias can be attributed to the non-parametric estimate of the PDF in each of the redshift bins in future work.

Despite potential parameter biases, we can utilize these simple simulations to compare the uncertainties on the inferredw₀andΩmusing the BAMBIS algorithm and a standard analyticχ2likelihood. We use a χ2 based on the work of Kelly (2007) which models a

BHM with Gaussian latent variable distributions:

χ_i2=DT_iV−1Di, (4.40) where Di=   mobs_Bi −(M+∆M0+µ(zi) +αxm+βcm) xobs_1i −xm cobs_i −c_m   (4.41) and V =   α2xv+β2cv+σ_int2_mB αxv βcv αxv xv 0 βcv 0 cv  . (4.42)

We also assume a Gaussian likelihood such that p(mobs_Bi ,xobs_1i ,cobs_i |θ,ψ) = 1

2π|V|1/2 exp −1 2χ 2 . (4.43)

Figure 4.19 shows the difference in the inferredw₀andΩmposterior distributions be- tween the two techniques using theD3data set. As we have not included any parameters in the analytic model to account for selection effects, we expect the posteriors obtained using this technique to be biased; therefore, we scale the posteriors in Figure 4.19 by their respective medians for better comparison. As shown in the figure, although BAMBIS includes a complex selection function, stochastic model, and non-parametric density es- timates, it can more strongly constrainw₀ andΩmthan this particular analytic approach.

The larger size of the uncertainty regions obtained using the analytic χ2 is likely due

to 1) the assumption of a Gaussian likelihood for the selected data and 2) the choice to model the latent distributions as Gaussians, which we know to be incorrect. This second point is particularly important, as the lack of explicitcsandxsmodel dependence is likely allowing too much freedom in the parameter sampling.

0.2 0.0 0.2 0.4 0.16 0.08 0.00

Analytic

BAMBIS

Figure 4.19: Comparison ofw₀ and Ωm posteriors inferred with BAMBIS (blue) and a standard analytic χ2 used in a Gaussian likelihood (green). Contours represent 1σ and

2σ uncertainty regions; posteriors have been scaled by their respective median values.

While this comparison presents an interesting assessment of parameter uncertainties, it perhaps more clearly illuminates the dangers of incorrect analytic model and likelihood assumptions. Although the analytic χ2 used in this example is not as sophisticated as

those of other BHM analyses (Rubin et al., 2015; Shariff et al., 2016; Mandel et al., 2016), it illustrates how difficult it is to capture selection effects and non-Gaussian distributions in an analytic framework.