Out for the count: some methodological questions in publications counting literature

This paper considers 'publications counting' literature, that literature which seeks to establish league tables of the published output of academics. Often justified on the grounds of benchmarking for supporting tenure and appointment decisions, publications counting literature is revealed to be methodologically flawed. This paper focuses on the major limitations of this kind of study, presenting a conceptual critique of existing 'publications counting' processes, and argues that the methods of such studies are neither reasonable nor consistent. This is particularly clear in areas such as the choice of journals to be sampled, the time frame established in sampling procedures, and various arithmetic procedures employed in calculating output measures. The paper concludes with discussion of possible alternative means of achieving goals of benchmarking hospitality scholarship while suggesting that publications counting should be abandoned as a sole means of determining research excellence.