22 August 2014

Africa: Experts Query Stats in Africa's Science Output Report

A report on the research output of Africa produced by the African Observatory of Science, Technology and Innovation includes potentially misleading charts, SciDev.Net can reveal.

Certain charts could lead readers to form a mistaken or inaccurate picture of the output of certain African countries. The findings raise questions about the inner workings of the observatory - established by the African Union and based in Malabo, Equatorial Guinea - which has failed to respond to our queries.

In May, the observatory released a report called: Assessment of Scientific Production in the African Union, 2005-2010. Alongside this, it promoted a supplement of graphs intended to "facilitate the interpretation" of the relative strengths and weaknesses of African countries and regions in various scientific fields.

Yet experts in the scholarly publishing metrics point out that some of the graphs may be misleading. Graphs for four nations - Angola, the Democratic Republic of Congo, the Republic of the Seychelles and Sierra Leone - indicate that these countries only produced a scientific output in one field: 'biology' for the Seychelles and 'clinical medicine' for the others.

"There is no way that these countries produce [research output] in only one field," says Anastassios Pouris of the University of Pretoria in South Africa.

Edward Byers, scientific officer at the Planet Earth Institute, a UK charity working for scientific independence for Africa, also tells SciDev.Net that the apparently narrow range of scientific work conducted in the four countries struck him as odd.

Byers says that, although the methodology described in the report "seems robust", it is possible that there "may have been an error" when the graphs were created.

He adds that his own searches on Scopus, a database of research articles that the observatory report cites as its data source, returned significant numbers of results listed under scientific disciplines not shown in the graphs for the four nations.

SciDev.Net confirmed this by repeating the Scopus searches. For example, a search for authors affiliated to Angola publishing between 2005 and 2010 returned 91 results filed under 'medicine' (which the country's graph represents as a large bubble). But it also returned 41 in 'agricultural and biological sciences', 22 in 'engineering' and 17 in 'social sciences'. None of these fields are displayed on the graph. Similar results were obtained for the other three nations.

SciDev.Net's attempts to contact the observatory using email and Twitter received no response. No phone number is listed on its website.

The report notes that a Canadian firm, Science-Metrix, was contracted to conduct data analysis for the report.

Éric Archambault, Science-Metrix's CEO, tells SciDev.Net that, although his company did this analysis, the observatory produced the final documents, including the supplementary graphs.

He also says that graphs sometimes have a cut-off point, below which data is not included, and this might be why fields with small numbers of publications were not displayed.

Byers agrees that this might be a plausible explanation in principle. Yet the supplement does not state that this is the case. All it says (in its short preamble) is that the size of data points in each graph is "proportional to the number of publications produced by the country/region in the corresponding scientific fields".

He also notes that the main report itself gives Angola reasonably high ratings in the field of natural sciences, and that it would make sense for this to be represented in some way in the supplement's graph for the country.

Link to the report

Link to the graphical supplement

Ads by Google

Copyright © 2014 SciDev.Net. All rights reserved. Distributed by AllAfrica Global Media (allAfrica.com). To contact the copyright holder directly for corrections — or for permission to republish or make other authorized use of this material, click here.

AllAfrica publishes around 2,000 reports a day from more than 130 news organizations and over 200 other institutions and individuals, representing a diversity of positions on every topic. We publish news and views ranging from vigorous opponents of governments to government publications and spokespersons. Publishers named above each report are responsible for their own content, which AllAfrica does not have the legal right to edit or correct.

Articles and commentaries that identify allAfrica.com as the publisher are produced or commissioned by AllAfrica. To address comments or complaints, please Contact us.