Fiona M Underwood

Statistical consultancy & research

Critique of Hsiang and Sekar paper linking legal ivory sale to an increase in illegal killing of elephants

There has been considerable attention on the paper released by Solomon Hsiang and Nitin Sekar on the National Bureau of Economic Research (NBER) website that claims that the sale of ivory to China in 2008 is the cause of the increase in illegal killing of elephants and demand for ivory.

I looked at this paper with interest because I, along with others, have thought hard about how or whether you can infer the cause of observed increases in demand for ivory. It is also an emotive and controversial topic and this means that it is important to get any analysis right. After serious consideration of this paper and discussion with Bob Burn (a co-author on papers [1,2] we have written that analyse the same data that they use here) we think that both their analyses and their logic for linking what they observe to the sale of ivory to China are flawed.

The authors sent me a copy of the paper when they released it on the NBER website and asked for comments (they note that it is also with a journal to be peer reviewed – to be clear I am not involved in this peer review – but given the renewed proposals for legal sales of ivory for the coming CITES CoP, they felt it important that their manuscript become public early enough to be part of the policy discussion on this issue).  The details of my reply to the authors can be found here and I summarise some of my key points below.

The Data

Most of the analysis they present uses data from the Monitoring Illegal Killing of Elephants (MIKE) program. The data are the number of elephant carcasses found each year at a number of different MIKE sites across Africa and Asia and the proportion of these carcasses that were illegally killed – this proportion is called PIKE (Proportion of Illegally Killed Elephants).  Because PIKE are proportions they are constrained to be between zero and one.

Their Models

To illustrate their argument the authors plot a number of points showing average PIKE and a clear step change in the value of PIKE between 2007 and 2008. But the points in the graph are not raw data but model outputs.  And the model they have used is wrong. For example, although PIKE is constrained to be between zero and one their model does not constrain these values to be between zero and one. They give many reasons for doing this including that to model the data correctly is complex, they wish to choose simplicity over complexity, and if they were to use more complex methods they would need to throw away 32.1% of the data.

Methods for analysing proportions, Generalised Linear Models (GLMs), are taught at undergraduate level on statistics courses.  GLMs are actually quite intuitive, widely used and understood and not really all that complex. The authors have misunderstood the methods because you do not need to throw away 32.1% of the data – all of the data can be used. I explain what I think their misunderstanding is in my detailed response. Furthermore, it is not OK to use the simplest of methods if they are wrong and it is clearly preferable to use more complex methods if that is what is needed to correctly represent the data.

The consequences of not modelling the data correctly are that their results could be wrong and it is difficult to know how wrong it is.

Their Logic

Their argument is that in their modelling they tested whether there was evidence of a step change, or discontinuity, in the PIKE data in 2008.  That is estimates of PIKE prior to the sale (up to 2007) are significantly lower than estimates of PIKE after the sale (from 2008 onwards) They say that their model shows that estimates of PIKE from 2003 to 2007 were significantly lower than estimates of PIKE from 2008 onwards.

The authors then look for a similar discontinuity in a number of variables they have selected to measure Chinese influence and presence in elephant range states. They consider these to be other potential drivers of the trade. They don’t find the same discontinuity in these variables between 2007 and 2008. Their conclusion is that if these drivers don’t show the step change then as everything else remained constant then the only explanation for the step change is the legal sale of ivory.

There are many things wrong with this, even if we were to ignore the fact that their models are not correct. In the paper they do not:

  • provide an explanation as to their choice of potential drivers that they test
  • discuss the global financial crisis of 2008. Could this also be a reason why the discontinuity is observed?
  • talk about trends in the trade of other illegal wildlife products such as rhino horn and pangolin. These have also increased over the last few years and there have not been legal sales in these products.
  • consider trade in other goods that might play a similar role to ivory within China. How has demand in these changed over the same time period? In which case how does this match with the demand for ivory?
  • compared their models to a model which allows an increasing nonlinear trend in PIKE rather than a step change

The argument they use that a similar step change is not observed in their other potential drivers might work for a simple situation. But the illegal ivory trade is complex and dynamic with many different drivers operating on different spatial and temporal scales all along the trade chain. It is more likely that if the sale has had an effect it contributes to the increase in demand rather than being the sole reason for an increase in demand. Any analysis should therefore look at relative contribution of different drivers and how they describe changes in PIKE by modelling it in one comprehensive model.

Conclusion

Further criticisms of their approach and their modelling can be found here. To be clear, I am not commenting one way or the other about whether the sale of ivory is the, or  one, reason for the illegal ivory trade. My concern is that this analysis and the conclusions it draws is flawed and should not be used to guide future policy on elephants.

Share this via:
Email Email
Print Print

Comments

  1. Katarzyna Nowak says

    You say that there have been no legal sales in the parts of rhinos but that poaching has also increased.

    Rhinos, like elephants, were split-listed. Southern white rhinos were downlisted to Appendix II in South Africa (in 1994) and Swaziland (2004) allowing for limited permitted trade / legal sales in the form of hunting trophies and live animals. We know that legal trophy hunts created a loophole for pseudo- and proxy-hunting and that likely hundreds of trophy horns entered the illegal market mostly in Vietnam. Whether these events / pseudo-hunts precipitated / contributed to the increase in rhino poaching since 2008 remains to be tested, but cannot, for now, be refuted.

    What seems obvious however is that legal trade – whether in the form of one-off sales or trophy hunts – does not eliminate / exclude illegal, and that legal does provide a cover for illegal.

    • Fiona says

      Thanks for clarifying the situation on rhinos – that’s helpful. I guess I was thinking more about large scale sales given the context of the original paper but the situation is as you say more complex than that.
      I’m being careful here not to make an argument one way or the other about whether, or how, the presence of a legal trade could or might contribute to the illegal trade. I’m just noting that the presented analysis is flawed and therefore can’t be used to draw conclusions one way or the other.

  2. Brendan Moyle says

    I have had similar problems with the statistics. I’m particularly troubled at the apparent neglect of the GFC in 2008. This sent shipping costs plummeting (good for sending containers of tusks to Asia) and interest rates down as governments tried to stimulate spending. This led to a surge of consumption in China across a broad range of goods, from art to jade to Swiss watches etc. I cannot conceive how you could disentangle or ignore these effects in a model with the data we have.

    The paper seems to have achieved little more than identifying a discontinuity in poaching around 2008-9.

    • Fiona says

      Thanks for giving examples of commodities that have shown increases from 2008 onwards.
      It could be that there is support for either of these two ways of modelling the data
      (1) an increasing non-linear trend in poaching that starts before 2008 and continues afterwards
      (2) a discontinuity around 2008-9
      and that it is not really possible to distinguish between them.

  3. Colman O Criodain says

    With regard to the pseudo hunting argument in the case of rhinos, there is no evidence that the downlisting of the South African and Swaziland populations contributed to the poaching crisis. For one thing, trophy hunting is allowed even for Appendix I species. Secondly, the poaching crisis did not emerge until 2007, beginning in Zimbabwe (whouse rhinos are still on Appendix I) and then spreading to South Africa later, a full 15 years after the downlisting of its population.

    • Fiona says

      Thanks for your comments and providing further clarification. As you say the main increase in rhino poaching does not appear at the time of a change in the downlisting. Instead it occurs around the same time as the increase in elephant poaching – this was the reason for me mentioning rhinos in my post.

      It might be more useful to examine whether or not (or by how much if at all) the presence of a legal market contributes to increased poaching (because of providing a cover as mentioned by Katarzyna or other possible processes). This could be considered even if something else is a driving force, or triggers, increased demand and poaching. But statistically this is extremely challenging to do. The presence of a legal market would need to be considered with all other possible drivers of the trade that operate all along the trade chain. The question would then be more about the relative contribution and importance of the legal trade compared to other drivers rather than trying to attribute any changes to one particular cause. I have discussed this more here.

  4. Katarzyna Nowak says

    This primarily in response to Colman: 1) cumulative trade (total exports of wild-sourced rhinos and their parts since 1994) under the split Appendix I/II listing has been much greater than under Appendix I – and eventually, became prone to violations / loopholes with 2) pseudo hunting emerging as a threat ~ 2003 (around the same time as poaching) and continuing until 2012 with a peak 2009-2011. By 2003, South Africa had reported some 1500 trade transactions in wild-sourced rhinos and parts to CITES – perhaps this signaled sufficient promise of cover to the illegal market? What, if I may, is your evidence against a link between illegal activity in the hunting sector (and >400 horns entering the Asian market in this way) and a surge in poaching? Is it outside the realm of possibility that permitted trade under the split-listing of both rhinos (and elephants) gradually rekindled markets and helped mask illegal trade?

  5. Katarzyna Nowak says

    PS Boom-and-bust is how CITES seems to work: de-regulate when species pops recover through split-listing and permitted trade / human consumption (with legal trade often masking illegal trade) eventually followed by a bust (often due to a combination of unsustainable use and IWT) and regulation in the form of moratoria/bans upon detecting species declines; resume discussions of trade / speculative stockpiling. Surely there’s an alternative to be had and pluralism in approaches / analyses (including econometrics) needs to be welcomed.