Geneva, 28th Nov (TWN): There are significant concerns that WHO’s study on public health and socio economic impact of substandard and falsified medical products to be released during a press conference today afternoon in Geneva, is based on questionable data, lacking in rigorous analysis.A virtual press conference (VPC) is being organised at CET 15to launch a report on the Global Surveillance and Monitoring System (GSMS) and a study on substandard and falsified medical products.
The first report compiles 1500 real cases from all WHO regions, giving estimates on the products most affected and regional prevalence. The GSMS was established in 2013, with financial support from the European Union and some of its Member States. The cases however were reported to the GSMS between 2013 and 2016 and during this time, the GSMS was using the following categories to capture the data from various member states: falsified medical products, suspected falsified medical products, substandard medical products, suspected substandard medical products, intentionally manufactured substandard medical products, diverted medical products, stolen medical products and unlicensed medical products. However the Secretariat has never provided breakdown of the data captured by the GSMS, according to the terms used by GSMS.
In 2016, the World Health Assembly determined that compromised medical products should be categorized as “substandard” and “falsified” and adopted working definitions on the same. Given this development, it is rather concerning that the GSMS cases are automatically categorized as being “substandard” or “falsified”.
The second report is titled ““Public health and socioeconomic impact of substandard and falsified products”. This report is the brainchild of, and supported by the US.
The Member State Mechanism (MSM) has however decided that it would not formally adopt the study.
Discussions with several member states familiar with the process leading to the study revealed that many have concerns with the findings of the Study especially the estimation of prevalence of substandard and falsified medical products in the low and middle-income countries using an unscientific methodology.
TWN has learned that the study examined 100 published papers and one publicly available database between 1st January 2007 to 31st December 2016 covering 48,218 samples. These samples are grouped into low income countries (11156), middle income countries ( 36884) and high income countries ( 178). Based on the number of failed samples in low-income countries (1166) and middle income countries (3906), the report finds the aggregate failure rate of tested samples of substandard and falsified medicines in low and middle-income countries is approximately 10.5%. The study then asserts that as the market size for low and middle income countries is nearly $300 billion, and given the 10.5% failure rate, the amount countries spend on substandard and falsified and medicines is $30 billion.
The methodology used by the Study is rather flawed. First, the Study is based on published papers that use heterogeneous terms and definitions. The terms and definitions also differ from the working definitions of “substandard” and “falsified” adopted by the 2017 World Health Assembly.
Secondly there are significant divergences in terms of methodology and sample size of the various papers relied on by the study, varying from 10 samples to 15,000 per study. The findings of the papers are thus not comparable. Thirdly, the samples substantially belong to two types of medicines i.e. antimalarial (18764) and antibiotics and anti-infectives (12375).
These flaws calls into question the data and analysis presented in the Study and its extrapolation of the failure rate to the total medicines market.
The study gives the impression that 10.5% products valued at USD 30 billion sold in the low and middle-income countries are substandard and falsified. The compilation of the 48218 samples from 100 published papers with a divergent number of samples and different parameters (definition) for the categorisation of substandard and falsified medical products cannot be used to measure the prevalence of substandard medicines in the entire pharmaceutical market of low and mididle income countries.
The inherent limitations of data led to the experts commissioned to develop the study to reject the idea of calculating the economic value of spending on substandard and falsified medicines. The first draft report submitted to the Steering Committee meeting in March 2016 did not contain any figures with regard to the purchase of the substandard and falsified medicine. The first draft report estimated the prevalence based on the sample size combining “data from 107 published field studies that met quality criteria for inclusion, totalling almost 73,000 samples. The mean prevalence rate of substandard medicines obtained from pooling the samples – the proportion of sampled medicines that failed laboratory testing based on specified reference standards – was 8.7%”.
Further it stated, “Most of the studies were conducted in Africa and Asia, and many focused on anti-malarial and antibiotic medicines. Because of this sampling bias, it is not possible to use the 8.7% figure as a global estimate of the prevalence of substandard medicines or as an estimate for all therapeutic categories”.
Then it concluded: “Empirical data on the public health and socioeconomic impact of substandard and falsified medicines is also very limited, particularly from studies where it is possible to track how the authors of published papers have developed their estimates. All that can be said at the moment is that the consequences of substandard medicines for public health, and for patient and health system costs, can be large in certain cases but as yet the data do not allow a global estimate of the cost that would be considered methodologically sound”.
The finding of the first version was changed without any valid explanation in the second draft circulated in November 2016.
The second draft estimated the economic impact of substandard and falsified medicine to the value of USD 31.1 billion. The second draft states: “The approach that is therefore adopted is ascribing the 10.5% ?6.2 prevalence rate to the pharmaceutical market of low-income countries ($135.2 billion), which produces a projected economic impact of $14.2 billion (with a 95% confidence interval of ?8.4 billion) in low income countries as a result of substandard and falsified medicines. For middle-income countries, the data used is based on a subset of field studies that sample the quality of a basket of medicines; 14 such field studies examine various therapeutic categories, selected on the basis of both availability and essentiality. These studies not only cover a wide range of therapeutic products, but 13 of the 14 studies were conducted in at least one middle-income country. The prevalence figure from this subset of 14 field studies is 6.8% ?5.6. If we apply this figure to the market size of middle-income countries ($249.2 billion), the estimated cost would be $16.9 billion (with a 95% confidence interval of ?13.9 billion). Combining these estimates, we arrive at a potential economic impact of $31.1 billion to LMICs.”
The third draft provided in March 2017 for comments by the expert advisory group and Member States continues the same approach and estimated the cost to be 28.6 billion.
“The approach that has therefore been adopted is ascribing the mean prevalence rate (across 2196 samples) of 10.8% to the pharmaceutical market of LICs ($112.4 billion), which produces a 2197 projected substandard and falsified medicines economic impact of $12.1 billion in LICs. For 2198 MICs, the data used is based on a subset of field studies that sample the quality of a basket of 2199 medicines; 24 such field studies examine various therapeutic categories, selected on the basis 2200 of both availability and essentiality. These studies not only cover a wide range of therapeutic products, but 22 of the 24 studies (92%) included at least one MICs. The prevalence figure from this subset of 22 field studies is 6.8%. If this prevalence figure is applied to the market size of MIC ($242.9 billion), the estimated cost would be $16.5 billion. Combining these two estimates yields a projected economic impact of $28.6 billion to LMICs due to substandard and falsified medicines”.
The final version of the study, according to sources, avoids all explanations, but continues using the flawed methodology.
The study also does not provide a break down of numbers that are substandard and falsified from the total of 4072 failed samples.
[The 2017 World Health Assembly adopted working definitions of substandard and falsified medicines. The former category generally refers to “out of specification” medicines produced by licensed manufactures, while the latter category would generally include activities of unlicensed manufacturers. ]
The study also overlooks the nuances involved in the health implications of substandard medicines. Generally, substandard medicines are grouped into minor, major and critical deviations. Major and critical deviations from the specifications are usually of a concern from a public health perspective, requiring regulatory actions. Given the study fails to provide such details, one cannot conclude that all the failed samples pose a threat to health. For instance a study published by WHO titled
“Survey of the Quality of Anti-Tuberculosis Medicines in Circulating in Newly Independent States of the Former Soviet Union” (http://apps.who.int/prequal/info_applicants/qclabs/monitoring_documents/TBQuality-Survey_Nov2011.pdf) concludes:
“A survey of quality of selected anti-TB medicines in six NIS identified 11.3% samples failing to meet the specifications set for this survey. Focusing only on extreme deviations as defined in this report, which are likely associated with direct health implications, the failure rate reached 1.0%. No sample was suspected to be of spurious, falsely-labelled, falsified or counterfeit product”
The Secretariat is proposing to widely disseminate this study as part of the prioritised activities of MSM for the 2018-19 biennium. The proposed prioritised activities F states: “Disseminate and use the study on the public health and socioeconomic impact of substandard and falsified medical products and the report on the WHO Global Surveillance and Monitoring System for substandard and falsified medical products as a tool to increase awareness and advocate at the highest policy levels nationally, regionally and globally for support and resources to prevent, detect and respond to substandard and falsified medical products”.
A major concern with this action is the wide dissemination of a study rooted in unscientific flawed methodology and misleading information, which does not reflect the on-the ground reality and results in unwarranted policy response. – Third World Network
