\documentclass[12pt]{article}
\usepackage{xcolor}
\begin{document}

\section*{MS596 ``Analysis of competitive surfing tournaments
  with generalized Bradley-Terry likelihoods" by Driver and
  Hankin: second rebuttal}

Below, the reviewer's comments are in black, and our replies to the
issues are in \textcolor{blue}{blue}.  We have indicated changes to
the manuscript where appropriate.  

Short story: we have accommodated all the comments with rewording and
clarification.  

Slightly longer story: the referee makes a wide range of observations,
all which we have engaged with.  We have overhauled much of the
writing, added descriptions of the computational and mathematical
methodology, clarified ambiguous wording, produced new statistical
tests, and improved the figures.  We have tried hard to retain the
many positive aspects of the manuscript that the referee explicitly
notes.

We have included an extended discussion of some of the points below.
The resulting document is, we believe, stronger and more scholarly
than before and we recommend it to you.


\section*{Referee's comments}

The paper does a good job of outlining the basics of surfing in terms
of competition rules and structure, and frames up the problem at
hand.  However, the paper itself feels rather light on content.  The
authors jump almost immediately to the results section and don't go
into very much detail in terms of how or why Bradley-Terry models are
used, other than the fact that they are used.

\textcolor{blue}{We have re-read our manuscript and indeed
  Bradley-Terry (BT) does make a rather abrupt entrance.  However, we
  do describe and justify and motivate the use of BT in section three.
  On reflection we believe we can address this comment by rearranging
  sections 2 and 3: this gives a gentler and more appropriate
  introduction to BT and indeed is a more appropriate structure for a
  manuscript.  In line with this, we have changed the section heading
  from ``previous related work'' to ``Previous statistical analysis of
  competitive surfing'' which is a better description, we feel.  We
  have added several new statistical tests and given brief outlines of
  their significance.}

The authors are guilty
of referring to various numerical quantities throughout the paper,
without expanding on how these values were calculated, and what
implications the values have. The paper would benefit from more
explicit equations throughout; on several occasions the authors refer
to modifying or changing their likelihood function to suit their
problem at hand, but it is difficult for the reader (or at least
myself!) to intuitively understand what this modified function would
look like, without writing it down or thinking hard about
it. Throughout the paper I found I had to do a lot of the mathemati
cal and abstract thinking myself, which would have been made much
easier with more equations and explicit definitions and derivations of
parameters, likelihood functions, and important quantities that are
referred to in Sections 5, 6, and 7.


\textcolor{blue}{These comments are entirely reasonable and, on
  re-reading the manuscript, we agree that we could have done a much
  better job describing the mathematical and statistical basis for our
  conclusions, especially the motivation for the likelihood equations.
  In the resubmission we have added a large amount of explanation that
  hopefully makes our reasoning more clear.\\ \\
  However, we observe that {\em all} our calculations are available
  for public inspection on gihub at our repo {\tt RobinHankin/hyper2}.
  In particular, file {\tt inst/surfing.Rmd} can reproduce all our
  findings.  However, the nitty-gritty detail of our R idiom is, we
  suggest, not really suitable for a JSA manuscript.  We believe that
  our enhanced reporting in the revision is a good compromise between
  not enough detail and too much.
  \begin{itemize}
  \item We have provided more detail (mathematical and computational)
    for the Fisher information argument for retaining the data with
    three surfers, as opposed to retaining just the pairs
  \item Added detail on the computational infrastructure for locating
    maximum likelihood estimates
  \item Referenced the computational method [{\tt equalp.test()}] for
    testing the null of equal competitive abilities
  \item Introduced a new table that explicitly sets out the likelihood
    function for the {\em Brazilian beachtype} reified entity
  \end{itemize}
}

  
The figures and tables are poorly presented and would benefit from
significant rework in terms of formatting.

\textcolor{blue}{We have reworked the figures:
  \begin{itemize}
  \item replaced the pie chart with a dotchart (log scale)
  \item retouched the fine-scale detail of the scatterplot of points scored 
    against maximum likelihood estimates
  \item changed the caption of the ``Profile support for the Brazilian
    wavetype reified entity" diagram to be more informative.  We
    suspect that the low information density of this graph contributed
    towards overall impression of the presentation alluded to by the
    comment above.  We would observe, however, that this particular
    diagram is worth retaining despite its plain appearance on two
    grounds: firstly, it supports our contention that the whole
    likelihood function is informative; and secondly, the smoothly
    varying curve is strong evidence that numerical optimization
    techniques are working correctly.
  \end{itemize}
  For the tables, we are happy to take specific advice, but in the
  meantime we have improved  some of the mathematical sections [using
    {\tt vphantom}] which reduces the cluttered appearance.
}

Section 7, while an interesting topic, feels like it falls more into
the realm of speculation than concrete analysis in the way it is
presented. The methodology is not very clear and it would be nice to
see a more explicit definition and derivation of pc

\textcolor{blue}{We agree that this section is more speculative than
  the remainder of the manuscript.  However, we would reiterate that
  the topic is clearly interesting and worth investigating, as both
  referees note.  Non-competitive behaviour of the sort discussed here
  is notoriously difficult to study; and in particular, formal
  statistical analysis of non-competitive behaviour is almost
  non-existant in the literature.  We are not aware of any other
  probability model that is able to even establish the existence of
  non-competitive behaviour, let alone quantify it.  We believe that
  our approach is new, and our probability model for non-competitive
  behaviour appropriate and plausible.\\ But, on re-reading the
  section, we agree that the presentation is indeed somewhat obscure.
  We have extensively re-written the section and believe it is much
  improved.  We include a new table motivating and discussing
  non-competitive behaviour using a particular example from the
  tournament.  We note that we presented a likelihood function for the
  three-competitor case in the first draft (it is now Table 3), and we
  consider this to be as clear as reasonably practicable.}

Overall, it feels like the main conclusion is ``not all surfers have
the same skill". This is not insignificant, however, it is not
particularly surprising and is one that could have been reached using
a number of different methods. Had the authors done a more complete
job of fleshing out why the Bradley-Terry approach was better for
reaching this conclusion than others, I would be more satisfied, but
as it stands it falls a little flat.

\textcolor{blue}{We regard the novelty of our computational approach
  as being access to the full likelihood function over allowable
  Bradley-Terry strengths (that is, not restricted to the MLE or point
  hypotheses, but over the entire simplex, $\left\lbrace
  p_1,\ldots,p_n\left|p_i\geq 0,\sum p_j=1\right.\right\rbrace$) and
  also the furnishing of compact and natural R idiom to allow the
  creation of such likelihood functions.  As such, there are many many
  statistical tests that could be conceived and executed using our
  method.  Our original priority was to submit a brief and compact
  manuscript but on reflection the referee seems to be saying that we
  have erred on the side of brevity.  We have added a topical
  statistical test, assessing Medina's performance at Lemoore as an
  example of the power of our method.  This test returned a
  significant p-value, a result not achievable with other methods.  We
  feel that this new analysis complements the existing analysis
  comparing Medina with Ferreira (and which also rejected the null);
  we have added a short discussion to the manuscript.  \\ \\ Reading
  between the lines of the comment, we detect a certain disappointment
  that some of the results were non-significant.  We too felt a little
  chagrined that our method returned a non-significant result for the
  compatriot analysis.  However, we would argue that it is important
  to report all findings, significant or not---especially for
  difficult or otherwise challenging hypotheses not accessible to
  conventional techniques.  We would also point out that even for
  nonsignificant results, our methods were considerably more
  informative than ``failure to reject a null'': as we stated in the
  submission, we present a novel quantification of noncompetitiveness
  (viz, $p_C$), and further, presented a nonzero maximum likelihood
  estimate for $p_C$ which, we suggest, is a worthy and interesting
  finding.  We also point out that such noncompetitiveness is a
  general phenomenon in the more general context of tournaments.  We
  present our work as the first application of a novel and very
  general method: we hope that subsequent analysis would cite this
  work.}

From a style perspective, the paper is well-written and enjoyable to
read, which makes the lack of substantial content a bit of a shame, as
the authors are clearly able to write a paper worth reading. Overall,
it sounds like the authors do have the data and analysis available to
warrant a more complete paper, however, what they have submitted feels
rushed and rather rough around the edges. While possible, it would
take a fairly significant overhaul/expansion to the current version of
the paper to get it into a publication-ready format.

\textcolor{blue}{We believe that this referee's insightful and
  constructive comments have led to a much improved submission.}

\end{document}
