1ES1011+496, EBL studies with flare from 2014. Ready for PWG

[david.paneque]

Posted on behalf of Adiv Gonzalez (e-mail from 2015/01/22)

[julian.sitarek]

(it was send by e-mail on 26th of January and already discussed, but David told me to post it also here)

Nice paper, it reads quite smoothly, below some comments.

ABSTRACT:
12.27+2.75 −1.83 - too many significant digits (in general you have
somehow inconsistent number of significant digits, 1.58±0.32 but
2.0±0.1)

INTRODUCTION:
high-frequency BL Lac (HBL) ==> high-frequency peaked BL Lac (HBL)

Ansoldi ́c ==> Ansoldi

UV-optical an near infrared ==> UV-optical and near infrared

"For every event we calculated the image size
(number of photoelectrons) and the angular distance θ between
the source position and the reconstructed γ-ray arrival direction.
The arrival direction was estimated combining the information
from each telescope using the Disp method (Fomin et al. 1994;
Zanin et al. 2013). The separation between gamma and hadronic
events was performed using a random forest algorithm (Albert
et al. 2008). Energy look-up tables generated with Monte Carlo
simulations were used to estimate the energy of each event." - This is
just the standard analysis, expalined in MARS paper or in the upgrade
paper, I think you can safely skip it.

FIG1:
the blue dotted line ==> the red dotted line (?)


SECTION 3:
even while one can see it later in plots, when you give the deabsorbed
fit results you can specify immediately fit probability (as you did
for the observed one)

Also for each night the correlation energy was computed. - I think you
can skip this sentence. You do not plot this correlation energy
anywhere (and it also does not have any deaper physical meaning for
the source, because it depends on the observations), and even if you
normalize each spectrum at each own correlation energy for a PWL
spectrum this should not change the error of the spectral slope.


FIG3:
"The red line represents the fit to a simple line" - "... to a constant value"
"If the night of February 8th (third point from the left) is not
included in the fit, the probability is almos 50%.' - remove, there is
no apriori reason to remove this night, and it seems to me that is
simply has the smallest error and therefore all the small effects
(systematic or source intrinsic) will be mostly visible on it. If you
decide to keep the sentence after all, correct almos==>almost

There is also an alternative way to check if the source was varying.
You can plot the hardness ratio, F(E2<E<E3) / F(E1<E<E2) and see if it
is stable.
The problem is however that you have a broad zenith range, so you
could have a problem finding E1 which is working for all the nights.
Plus E2 is quite arbitrary.

SECTION4:
"the observed spectrum is determined" ==> "the observed spectrum is formed"

"A population of electrons are" ==> "A population of electrons is"

FIG 4:
did you try if it does not look better in log scale ?
otherwise you can clip the plot at 1.6 or something, because either
way one cannot see anything above this value.
I would suggest you to join Fig 4 and Fig 5 as two panels in one
figure, write also the EBL model in the figure caption.

SECTION5:
writing that cut-off energy can only be possitive is probably too
much: the energy cannot be negative, and nominallly putting it with
minus would result in pile-up spectral shape not a cut-off spectral
shape.

specrum ==> spectrum

SECTION5:
"the one has" ==> "the one that has"
To be honest I'm a bit lost how you estimated those systematic errors
in the end.
Have you used only the LP profile ?
For each scale i you should have alpha_i + d1alpha_i - d2alpha_i. From
the text it seems to ma that you have taken as the systematic error
(+d1alpha_syst-d2alpha_syst): d1alpha_syst = max (d1alpha_i) and
d2alpha_syst=max(d2alpha_i), independent plus and minus. It seems more
natural to take the spread of alpha_i.
Maybe a plot of alpha +/- dalpha vs energy scale would be useful here
(unless we are too scared to show it ).

If I were a nasty referee I would have asked if the claimed systematic
error of +/-0.15 in the spectral index (and maybe even more the
systematic error that Roberta gave on the curvature term in the recent
Crab paper) can lead to any additional systematic error.

SECTION6:
You should write (maybe even before) that 'E_gamma' (before called
'E') is measured in the observers frame

shouldn't (1+z)^2 be rather at E_max (i.e. broader range) ?

FIG8: are HESS and Fermi EBL results also in this plot ?

[elisa.prandini]

(from e-mail sent to authors on Feb 4th 2015)

Thank you for the nice paper! You can find below some comments.

ABSTRACT
Feb --> February

INTRO
- I suggest to change the order start with EBL definition and measurements. In this way the results you mention about the de-absorbed spectra are more comprehensible.
- You could add a table summarising MAGIC measurements
- Maybe you can skip the sentence: "Although MAGIC ... good candidate to be observed in flaring state."

4. EBL MEASUREMENTS
- Fermi-accelereted to ultrarelativistic energies (remove a)
- so the event statisticS IS large
- behaves ... asymptotic limit. (singular)
- log-parabola --> LP
- straight power law intrinsic specTrum
- you could comment on the fact that the alpha obtained is compatible with 1

5. SYSTEMATIC UNCERTAINTIES
- asses --> assess

6. DISCUSSION
- I would say that the differences become evident above 300 GeV rather than 200
- Fig 7. I suggest to use a thicker line for the data points. Also, label axis are a bit too small.
- interaction of the g-ray and the EBL photonS

[david.paneque]

Hi Adiv, Priya,
I had a look at the manuscript and I actually already discussed few things with Priya (face to face, at MPI).
I just write here my main evaluation, for bookkeeping.
In general, the paper looks very good, and it is incredibly well written (all very clear). Congrats !!!
I only have two remarks on the content:

1) you justify the usage of a single fit for all data with Figure 3, which shows that the PL index is approximately the same for all the nights (regardless of the large change in flux). However, this figure is a bit misleading because you are not showing the quality of the fits for all these nights. You might get a PL index of 2.0, but perhaps the fit quality is very bad, which means that a PL with index 2.0 is not a good representation of the data.
Do you have the Chi2/NDF (or Prob) for all these fits ? How do they look ?

Something that could be more convincing to justify the usage of a single shape would be somethign along these lines.
a) Since you want to test teh EBL, you might want to do the tests in the non-corrected spectra (which are curved)
b) Fit the "all-data" spectrum with a log-parabola ( PL is surely not a good one)
c) Fit each of the spectra for all the single nights with a log-parabola function where the parameters alpha/Beta are fixed to those of the "full data set". That is, only the normalization factor is allowed to change (to absorb the flux variations)
d) Evaluate the goodness of the fit for each of these nights (you can plot Prob vs ObsNight)

What do you think about it ?

2) When computing the systematics, based on the 15% systematic for the energy scale reported in teh performance papers, you are shifting the individual pixel size content by +/-15%. NAturally, the single PMT signals will be related to the overall size of the image, which will also be related to the overall energy estimated for that image. But these quantities are probably not "one-to-one" related, isn't it ?
In other words, a shift in the full energy estimated for an image of 15% might only be attributed with a shift in the PMT signals of 10% (because by increasing individual signals you get more pixels in the image, which were removed before) or by 20% (because besides the Size, there are other parameters that play a role in the estimation of the energy).
I never played with these quantities in MC, and hence I do not have a sense of how they actually relate to each other.
The point is that, while surely these quantities are related, it is not clear that the relation is a one-to-one, and hence our "official" systematic error in energy of 15% may not translate to 15% change in the PMT signals.

Any explanation of why you propagate the systematics using single PMT signals instead of using the reconstructed energy of the events ?

cheers,
David

[adiv.gonzalezmunoz]

Hi Adiv, Priya,
I had a look at the manuscript and I actually already discussed few things with Priya (face to face, at MPI).
I just write here my main evaluation, for bookkeeping.
In general, the paper looks very good, and it is incredibly well written (all very clear). Congrats !!!

Thanks for that!

I only have two remarks on the content:

1) you justify the usage of a single fit for all data with Figure 3, which shows that the PL index is approximately the same for all the nights (regardless of the large change in flux). However, this figure is a bit misleading because you are not showing the quality of the fits for all these nights. You might get a PL index of 2.0, but perhaps the fit quality is very bad, which means that a PL with index 2.0 is not a good representation of the data.
Do you have the Chi2/NDF (or Prob) for all these fits ? How do they look ?

Well, each individual fit had good probabilities, that one thing that we found with 1ES1011, that its intrinsic spectrum seems to be remarkably flat in our energy range. Now, if I force the fits to 2.1, all fits are good (more than 10%) except one day, where the probability is 0.4% (which still is not that bad).

Something that could be more convincing to justify the usage of a single shape would be somethign along these lines.
a) Since you want to test teh EBL, you might want to do the tests in the non-corrected spectra (which are curved)
b) Fit the "all-data" spectrum with a log-parabola ( PL is surely not a good one)
c) Fit each of the spectra for all the single nights with a log-parabola function where the parameters alpha/Beta are fixed to those of the "full data set". That is, only the normalization factor is allowed to change (to absorb the flux variations)
d) Evaluate the goodness of the fit for each of these nights (you can plot Prob vs ObsNight)

What do you think about it ?
Actually, initially that is the approach that we were using, where the normalization was the only parameter allowed to change, and also in that case the worst fit was at the level of 1 or 2% probability. However, we decided to used the de-absorbed spectra since it could be represented with a simpler function, that is, we would be using less parameters.
Regarding the plot Prob vs ObsNight, I would prefer just a table.

2) When computing the systematics, based on the 15% systematic for the energy scale reported in teh performance papers, you are shifting the individual pixel size content by +/-15%. NAturally, the single PMT signals will be related to the overall size of the image, which will also be related to the overall energy estimated for that image. But these quantities are probably not "one-to-one" related, isn't it ?
In other words, a shift in the full energy estimated for an image of 15% might only be attributed with a shift in the PMT signals of 10% (because by increasing individual signals you get more pixels in the image, which were removed before) or by 20% (because besides the Size, there are other parameters that play a role in the estimation of the energy).
I never played with these quantities in MC, and hence I do not have a sense of how they actually relate to each other.
The point is that, while surely these quantities are related, it is not clear that the relation is a one-to-one, and hence our "official" systematic error in energy of 15% may not translate to 15% change in the PMT signals.

Any explanation of why you propagate the systematics using single PMT signals instead of using the reconstructed energy of the events ?

Well, I have to say that also I'm not an expert on analysis but as far as I understood, the energy is directly proportional to the size, and actually, these approach is based in the performance paper. The way they estimated the systematic uncertainties related to the energy scale, was scaling up and down the light in the pixels in the MC. However, and this is a "novelty" in our analysis, we made the scaling on the actual data instead of the MC. In principle the approach is equivalent. Now, as you say, there are another parameters that enter in the calculation, but are stereo parameters and these should remain the same.

cheers,
David

[david.paneque]

Well, I have to say that also I'm not an expert on analysis but as far as I understood, the energy is directly proportional to the size, and actually, these approach is based in the performance paper. The way they estimated the systematic uncertainties related to the energy scale, was scaling up and down the light in the pixels in the MC. However, and this is a "novelty" in our analysis, we made the scaling on the actual data instead of the MC. In principle the approach is equivalent. Now, as you say, there are another parameters that enter in the calculation, but are stereo parameters and these should remain the same.

I do not quite understand your answer.
The systematic that is quoted in the performance paper is 15% in energy scale, not in the "Size" parameter. Surely these two are closely related, but I do not know whether one can directly convert the 15% systematic uncertainty in energy to 15% systematic uncertainty in the parameter "Size". This is not mentioned neither in the performance paper nor in yuour 1011 paper. IF that is the case, then it should be explicitly mentioned in the paper. Otherwise, it is not clear where this "15% shift comes from".

[julian.sitarek]

Hi,

I had a look in the new version which Adiv send around. Below a couple of comments, please implement them, but they are no major ones so you have my green light for the next step of refereeing.

- lambda F_lambda: in the abstract and in the end of the paper you give the value as 12.27+2.75-1.83. Obviously there are too many significant digits there, you should write 12.3+2.8-1.8 or even 12+3-2. Moreover I'm not sure how exactly you got the error. The optical depth should be more or less proportional to the flux density so I would expect that the relative error is the same. But 2.75/12.27 is 22% which is much larger even if I add statistical and systematic error on the alpha parameter in the conservative, linear way, (0.09+0.07)/1.07. Maybe there is some integration involved, but if you used some more complicated formula you should explain it in the paper.

-in the introduction when you talk about 2007 discovery of 1ES1011 and give some numbers there is a reference missing. Also the reported relative systematic error on the flux (75%) is a lot (with steep spectra large numbers are possible, but even with -4 slope I would not expect it to be more than 50%.

- in section 2 you can cite the Part I of the upgrade paper. For part II which I guess you are citing (it is hard to figure it out from the reference) you can use the DOI: doi:10.1016/j.astropartphys.2015.02.005

- in the beginning of section 3: "3219 events" - do you mean excess events ?

- the second (last) paragraph of the section 5 - I'm not so convinced of the way how you estimate the effect of the systematics. If I understand correctly you checked all those -15%, -10%, ... + 15% scalings and then took the one which has the largest errors, and those errors are interpreted as systematic error (do you subtract in quadrature the error for the "no scaling" case ?). From the systematics in energy scale you might have in principle two effects: 1) some shift in best alpha 2) increase or decrease in the statistical error. In your case you claim that dominant one is 2). From the tests that I was doing with S3 0218 1) was more important. In you case, the situation might be different, but still I think you should justify better why you take the statistical error and not the shift in the best alpha value. Probably the most conservative approach would be to take a sum (in quadrature maybe) of the shift of best alpha for the +/-15% energy shift and the statistical error of the most shallow (maybe subtracted in quadrature for the statistical error of the no scaling case).

- Fig 8 - in the caption you mention ~20 references, while in the legend there are ~40 points (e.g. Zemcov et al 2014 is missing). You should either list all of them (which is too much I think) or just mention that references are in the legend.

In the conclusions you say that the novelty is larger distance, but you should also mention that the results are more constraining than the HESS ones.

Cheers,
Julian

[julian.sitarek]

Hi David,

Except of some saturation effects, a tail due to high impact events and some logarithmic dependence due to Hmax, there is basically a scaling between the size and energy. Since we are talking about shifts of the order of 15% one can treat the shifts in light scale as shifts in energy scale, especially within the usual accuracy at which the systematic errors are given.
Moreover what we actually did in the upgrade paper (and what we were doing also in the last two performance papers) was to estimate the light scale accuracy, and based on it make a statement on the energy scale. So it is perfectly fine what Adiv et al did.

Julian

Well, I have to say that also I'm not an expert on analysis but as far as I understood, the energy is directly proportional to the size, and actually, these approach is based in the performance paper. The way they estimated the systematic uncertainties related to the energy scale, was scaling up and down the light in the pixels in the MC. However, and this is a "novelty" in our analysis, we made the scaling on the actual data instead of the MC. In principle the approach is equivalent. Now, as you say, there are another parameters that enter in the calculation, but are stereo parameters and these should remain the same.

I do not quite understand your answer.
The systematic that is quoted in the performance paper is 15% in energy scale, not in the "Size" parameter. Surely these two are closely related, but I do not know whether one can directly convert the 15% systematic uncertainty in energy to 15% systematic uncertainty in the parameter "Size". This is not mentioned neither in the performance paper nor in yuour 1011 paper. IF that is the case, then it should be explicitly mentioned in the paper. Otherwise, it is not clear where this "15% shift comes from".

[elisa.prandini]

Dear Adiv et al.,
I also went through the new paper version and I have only a couple of minor comments in addition to Julian ones, but let me first congratulate for the paper and in particular for the Introduction section, it is really well written and clear!
My first suggestion is to add the standard reference to Hauser & Dwek paper after the definition of EBL (http://adsabs.harvard.edu/abs/2001ARA%26A..39..249H). The second suggestion is to use the same font size adopted in the axis of Fig.2 (labels and titles) also for the other figures. In figures 4 and 5 you can set the background of the legends white.
There is one point that I asked you in my previous e-mail but you did not address: in the Discussion, you mention that the differences between the two scaling become evident above 200 GeV, but I would rather say at 300 GeV (the 200 GeV points are perfectly compatible). Can you comment on this?
Cheers,
Elisa

[david.paneque]

Hi David,

Except of some saturation effects, a tail due to high impact events and some logarithmic dependence due to Hmax, there is basically a scaling between the size and energy. Since we are talking about shifts of the order of 15% one can treat the shifts in light scale as shifts in energy scale, especially within the usual accuracy at which the systematic errors are given.
Moreover what we actually did in the upgrade paper (and what we were doing also in the last two performance papers) was to estimate the light scale accuracy, and based on it make a statement on the energy scale. So it is perfectly fine what Adiv et al did.

Sounds good. Then I would suggest to write 2-3 sentences about it in the paper (linking the 15% energy shift quoted in the performance paper with the 15% shift in light scale).

cheers,
David

[adiv.gonzalezmunoz]

Hi Elisa! thank you for going through the paper one more time

Dear Adiv et al.,
I also went through the new paper version and I have only a couple of minor comments in addition to Julian ones, but let me first congratulate for the paper and in particular for the Introduction section, it is really well written and clear!
Thanks!

My first suggestion is to add the standard reference to Hauser & Dwek paper after the definition of EBL (http://adsabs.harvard.edu/abs/2001ARA%26A..39..249H).
OK, included

The second suggestion is to use the same font size adopted in the axis of Fig.2 (labels and titles) also for the other figures. In figures 4 and 5 you can set the background of the legends white.
OK, done.

There is one point that I asked you in my previous e-mail but you did not address: in the Discussion, you mention that the differences between the two scaling become evident above 200 GeV, but I would rather say at 300 GeV (the 200 GeV points are perfectly compatible). Can you comment on this?
You right, the points at ~200 GeV are compatible (for sure they have the same chi2 contribution). The true is that the way we could chose the energy range is somewhat arbitrary. What HESS did was to take the range in which there was an overlap among the observations of the different AGNs. We think that we are using an conservative approach, since we could use the whole energy range of our observations. Maybe the sentence "evident" is an overstatement. It could say "start to show" instead.

Cheers,
Elisa

I'll upload a new version soon

Cheers

[adiv.gonzalezmunoz]

Hi David,

Except of some saturation effects, a tail due to high impact events and some logarithmic dependence due to Hmax, there is basically a scaling between the size and energy. Since we are talking about shifts of the order of 15% one can treat the shifts in light scale as shifts in energy scale, especially within the usual accuracy at which the systematic errors are given.
Moreover what we actually did in the upgrade paper (and what we were doing also in the last two performance papers) was to estimate the light scale accuracy, and based on it make a statement on the energy scale. So it is perfectly fine what Adiv et al did.

Sounds good. Then I would suggest to write 2-3 sentences about it in the paper (linking the 15% energy shift quoted in the performance paper with the 15% shift in light scale).

Well, I included a sentence linking to the 15% energy shift from the performance paper.

cheers,
David

I'll upload a new draft in a few minutes...

[adiv.gonzalezmunoz]

Hi

Sorry for the long delay. Please find as attachment a new version of the paper.
In the next days I'll traveling back to Mexico. If you are OK with it, we could start to circulate the paper among the collaboration and in the next two weeks while I'm off the grind, the members will have time to send comments.

Cheers

Adiv

[david.paneque]

Hi Adiv,
very good.
Before circulating the paper through magic priv, it needs to go through the internal referee process from the Pub Board members.
This will also take 2 weeks. So you should have the list of remarks by the time you are back from holidays.
cheers,
David