Systematic search

A systematic search was performed for published full reports of randomised controlled trials which tested the antiemetic efficacy of prophylactic and therapeutic ondansetron compared with placebo, no treatment, or other antiemetics on nausea and vomiting after a general anaesthetic. Medline (Knowledge Finder 4.0, Silver Platter 3.25), Embase, and Biological Abstracts databases were searched, without restriction to the English language and using different search strategies14 with free term combinations (date of the last electronic search 19 September 1996). Additional reports were identified from reference lists of retrieved reports and review articles on postoperative emesis and ondansetron and by hand searching locally available anaesthesia journals. We checked our database with the database of published trials provided by the manufacturer of ondansetron. Abstracts were not considered. Unpublished trials were not sought. Authors of reports were contacted to clarify uncertainty about duplicate publications. If there was an ambiguous reply or no reply we contacted the manufacturer.

Extraction of data

Information on authors, journals (parent journal or supplement), number of patients per treatment and control arm, sponsorship, cross references to reports with similar data or preliminary reports, and anaesthetic techniques was taken from each report. We extracted efficacy data because intention to treat information was not provided in all reports.

The following definitions are used in this paper. Sponsorship by the manufacturer was assumed when a trial or a journal issue (a supplement for instance) was stated to be sponsored or when one of the coauthors was an employee of the pharmaceutical company. When we discovered matching reports of trials the full report (a multicentre trial) or the report which was published first was regarded as the original. Preliminary reports, republication of original data with or without data from other trials, or republication of part of data from an original report were regarded as duplicate. A cross reference was defined as a reference which clearly stated either the original source of the data or that the same data would be published elsewhere.

Evidence that data were duplicated was based on: the presence of a cross reference, confirmation by authors or manufacturer, or similarity of reported data.

Impact of duplication

We planned to clarify statistical significance and clinical relevance of any difference in efficacy with and without duplicated data using the dataset with the most duplication—prophylactic intravenous ondansetron 4 mg versus placebo. Efficacy was defined as prevention of postoperative emesis. Relative benefit (relative risk) was calculated with 95% confidence intervals using a random effects model,5 together with point estimates and 95% confidence intervals of the number needed to treat.678 These calculations were made for non-duplicated data, for duplicated data, and for the two sets combined—that is, as published. A positive number needed to treat indicates how many patients have to be given ondansetron to prevent vomiting in one who would have vomited on placebo. Statistical significance of any difference between numbers needed to treat was assumed if there was no overlap of the confidence intervals, and, for independent datasets, it was verified by the formula:

where NNT is the number needed to treat and SE the standard error.

Results

A total of 104 reports were considered: 16 were subsequently excluded because they were not randomised (10), the number of patients per treatment group was not mentioned (2), the observation period was not stated (2), general anaesthesia was not used (1), or they were not adequately controlled (1). We could not obtain hard copies of four citations (and neither could the manufacturer).

Impact of duplication: originals versus duplicates

Fourteen of 84 analysed randomised trials (17%) contained duplicated data. The 14 duplicates documented 4886 patients, 3335 treated with ondansetron. In all 28% (3335/11 980) of the ondansetron patient data studied in published randomised trials were duplicated.

The studies most commonly duplicated were those which compared prophylactic intravenous ondansetron 4 mg with placebo. This comparison was reported in 19 original trials,4 16 of which were not subject to duplicate publication, but data from three (trials 7a, 8a, and 9a) were subsequently duplicated in six further reports (7b, 7c, 8b, 9b-d). The three original reports which were subject to duplicate publication were large multicentre trials.

In the 16 reports which were not duplicated the number needed to treat to prevent vomiting within 24 hours with intravenous ondansetron 4 mg compared with placebo was 9.5 (95% confidence interval 6.9 to 15), relative benefit 1.33 (1.21 to 1.47) (1). In the three original reports which were subject to duplication, the number needed to treat was 3.9 (3.3 to 4.8), relative benefit 1.44 (1.33 to 1.55). The difference between the two numbers needed to treat was statistically significant (z=4.84, P<0.00001).

Treatment efficacy in 16 original trials which were not duplicated, three original trials which were duplicated, six duplicates of these three trials, and the combination of originals and duplicates. Number needed to treat (with 95% confidence intervals) to prevent vomiting up to the 24th postoperative hour with intravenous ondansetron 4 mg compared with placebo. The numbers above the symbols are the numbers of reports.

• Download figure
• Open in new tab
• Download powerpoint

In the six duplicates of those three trials the number needed to treat to prevent vomiting within 24 hours with intravenous ondansetron 4 mg compared with placebo was 3.3 (2.8 to 3.8), relative benefit 1.62 (1.43 to 1.83) (1). The combined number needed to treat with data from all three duplicated trials and their six duplicates was 3.5 (3.2 to 4.0), relative benefit 1.54 (1.46 to 1.63). This number needed to treat was significantly different from the number needed to treat from the 16 non-duplicated trials (z=6.8, P <0.00001).

Discussion

Why make a fuss about duplicate publications? There are situations where duplicate publication is justified,30 and there are laudable examples,31 in different languages and with clear cross references.32 The reason for making a fuss is covert duplication, because when duplication of patient information is unstated, estimates of treatment efficacy and safety may be biased. There are two arguments here, qualitative and quantitative.

Instead of reading nine genuine reports of the efficacy and safety of a drug the reader is presented with 23. Even thoughtful readers will fail to spot the duplication33 and give the conclusions more emphasis than they deserve,334 believing that the drug was tested in more patients than was actually the case. An exaggerated perception of both treatment efficacy and safety is the likely result.

At face value the relevant published literature contains 11 980 patients treated with ondansetron in 84 trials. In reality it was 8645 patients in 70 trials. An uncritical analysis would have overestimated the number of trials by 17% and the number of patients by 28%.

As is clear from the1, failure to exclude duplicates would have overestimated ondansetron's efficacy by 23% (improving the number needed to treat from 6.4 to 4.9), and exaggerated the accuracy of the point estimate (narrowing the confidence interval). While the exaggerated accuracy is a function of the inflated numbers, the cause of the improvement in the number needed to treat is not so obvious. It results from the fact that the trials which were subject to duplication reported much greater efficacy (number needed to treat 3.9) than the non-duplicated reports (number needed to treat 9.5) with no overlap of confidence intervals. The actual duplicates exaggerated treatment efficacy even more (number needed to treat 3.3).

The difference between originals, duplicates, and the combined data has clinical relevance. The number needed to treat of 6.4 for the original reports means that more than six patients have to be treated with ondansetron 4 mg to prevent vomiting in one who would have vomited had he or she received a placebo. For the duplicates only three patients need to be treated for one to benefit. Combining originals and duplicates—that is, combining the data as they appear in the literature, improved the 1 in 6 to 1 in 5, just within our preset definition of clinically relevant postoperative antiemetic efficacy.35

Overestimation of treatment efficacy by including duplicates is yet another methodological problem for meta-analysis.36 The effect is in the same direction and of the same degree as the effects of improper blinding or inadequate concealment of treatment allocation.37 Unwarranted assumptions about safety might also be made. No major drug related adverse effect seen in 11 980 patients sounds more persuasive than none in 8645 patients.38

Covert duplication should be uncovered by several of the processes of publication: by authors adhering to the Vancouver guidelines; by authors having to sign a declaration of unique publication when they submit a paper to a journal; and at peer review, when expert reviewers should spot the similarity of data.

The Vancouver guidelines state, “Manuscripts are reviewed for possible publication with the understanding that they are being submitted to one journal at a time and have not been published, simultaneously submitted, or already accepted for publication elsewhere.”39 All the duplicates reported here were published as full reports in journals which quote this requirement or referred to the guidelines at the time of publication. We do not know whether or not authors signed a declaration of unique publication. Half of these reports were published in journal supplements. Supplements' guides for authors were the same as their parent journals. If peer review was used the process clearly failed to detect or eliminate duplicates.

The key issue here is cross referencing. Without cross referencing duplication becomes covert. Only one preliminary report17 mentioned a future full paper, but when the full paper was published it did not mention the preliminary report. No clear cross referencing was present in the other 21 reports. Two reports referenced a matching paper but with no indication that they used the same data. These cross references were of no value, because the reader was left in the dark about the relation between the papers.

It could be argued that these covert duplicates were so obvious that they would be identified easily. However, they have not been identified. Original ondansetron data together with data from duplicate reports have been unwittingly cited in clinical investigations,40414243 in opinion leaders' articles on postoperative nausea and vomiting,44 in a widely cited review article on ondansetron,45 in a standard textbook on ambulatory surgery,46 and in a review about the ethics of antiemetic trials.33 Covert duplicate reports can be very difficult to recognise.

Examples of masking included different language or completely different authors, adding further data to duplicated material, presenting analyses as both efficacy and intention to treat, or combining the duplicated data with data from another trial and reporting a combined analysis. We detected the duplicates only because of the stringency of our systematic review process. Confirmation by the original authors had to be sought, but responses were far from comprehensive. Neither could the list of publications provided by the manufacturer resolve the matter, because the manufacturer did not distinguish between originals and duplicates.

Ironically, while writing this manuscript a report of yet another large multicentre trial of ondansetron appeared in a peer reviewed journal.47 It contained the same data as a report published two years earlier.23 Another 226 patients given ondansetron were duplicated in the published literature, and the crucial cross reference to the original paper was lacking. Caveat lector.