Keywords

Clinical trials; Discontinued drugs; Approved drugs; Protocol criteria; Drug development strategy

Introduction

Highly significant progress has been made in the promotion of patient health through pharmaceutical research. Though this contribution to human health is undeniable, the global decline in the seeding of drug creation has made pharmaceutical development more challenging and has stiffened international competition [1]. At least ten years and more than 1.5 billion dollars are generally required for the development of a new drug, and the cost is increasing due to safety concerns of regulatory authorities throughout the world [2,3]. In the meantime, the number of approvals by the US, the EU and Japan has been recovering in recent years, but it has not improved significantly for the long term [4-7], suggesting that recent scientific progress might not necessarily spur the development of new blockbuster drugs [8- 10]. The success rate of investigational drugs has remained very low and has been reported to be ca. 30% for Phase2 (Ph2) and ca. 60% for Phase3 (Ph3), with the cumulative rate from Phase1 (Ph1) to approval ranging between 10% and 15% [11]. In addition to the search for new seeds, the development of more effective means to streamline Research and Development to improve success rate is an urgent issue. Improving development strategy and the conduct of clinical studies are imperative. One goal is to increase the success rate for late-stage studies in which the investment is significant because of the requirement for large numbers of subjects [12-14].

Many examples of successful investigational drugs have been reported; however, few examples of discontinued compounds have been published. Khanna [15] investigated the trend in drug discovery following failures in Ph1-Ph3 over the last 20 years. Arrowsmith et al. [16] reported rates and causes of Ph2 and Ph3 attrition. DiMasi [17] outlined reasons for failures in 2000- 2010 by development phase and therapeutic area. However, no investigator has conducted studies on discontinued products at the level of detailed study protocols in comparison with corresponding approved products.

Recent trends toward disclosure of information have prompted the publication of clinical study results [18]; however, not all information has been made available and few attempts have focused on discontinued drugs. To obtain possible insight into the efficiency of the pharmaceutical drug development, we therefore collected accessible, published, detailed data and protocol information on products discontinued in the late phase of development, by making comparisons with products approved during the same survey period.

Methods

i. Target discontinued and approved products

Target discontinued products were selected based on the commercial database (Pharmaprojects^®) [19]. This was used to identify those products discontinued after the initiation of Ph3 during the five years between 2011 and 2015. A total of 820 products were discontinued during the survey period. Of these, 12% (95 products), excluding diagnostic agents and medical devices discontinued during or after Ph3, were reviewed. Ph3 studies were found for 18 of these 95 products with 2 indications each and for 2 products with 3 and 5 indications each, which tallied up to 119 projects (Figure 1 and Table S1). As a reference, target approved products were selected based on the same database providing information about products which had obtained first approval for the initial indication by any country's regulatory authorities during the survey period. A total of 544 projects were reviewed.

Figure 1: Inclusion flowchart of target products and projects. Left flowchart: Discontinued products and projects. Right flowchart: Approved products and projects.

ii. Study procedure

We used the commercial database (Pharmaprojects^®) [19], materials released by pharma companies, publicly available data from public clinical trial registries [20-22] and public regulatory authority websites [23-25] as the sources of information on target projects. These projects were classified by disease and therapeutic area and data were collected to examine patterns in the target projects. Discontinuation reasons by sponsors were examined and re-evaluated based on related information and classified into efficacy, safety and strategy. Protocol differences between Ph2 and Ph3 studies were reviewed with respect to the following five items (endpoint, target subject, control, evaluation period, and design, hereinafter referred to as the "major protocol criteria"), taking their impact on study results into consideration. Target studies were classified as either (1) Protocol changed, (2) Not changed, or (3) No Ph2. Definition of endpoint was Ph3 primary endpoint and each study endpoint was compared with one of endpoints in the respective Ph2. A study was classified as "No Ph2" if the subjects in Ph2 differed completely from those in Ph3. The impact of change in the major protocol criteria on study results was presumed to be mainly on efficacy. Therefore, the projects discontinued for efficacy reasons were compared with those discontinued for strategy reasons (E/S), to investigate the impact of changed major protocol criteria. Differentiation points of the major protocol criteria for discontinued projects were compared with those of approved projects, which were shown by D/A (ratio of percentages of discontinued/approved projects).

Results

i. Background of discontinued and approved projects

Target discontinued projects were classified into therapeutic areas (Table 1). Oncology is the predominant area. Target approved projects were also classified into therapeutic areas. Infection is the predominant area; clearly there is a significant difference between Oncology and Infection in terms of challenges for drug development. Target discontinued projects were classified into product types. ‘New chemical entity’ (NCE) is the predominant type. Target approved projects were also classified into product types. Formulation changes to existing medicines such as reformulation and fix dose account for nearly half of projects. NCE comprised 29%, which is considered to have a high degree of difficulty of drug development.

Table 1: Discontinued and approved projects on therapeutic area basis and product type basis.

ii. Discontinuation reasons of Ph3 projects and details

Most projects were discontinued due to failure in achieving the primary endpoint (Figure 2). Discontinuation due to "Strategy" was more than that due to "Safety" and relatively common even for Ph3 studies, suggesting that numbers of discontinued projects could be reduced by discreetly assessing whether to proceed to Ph3.

Figure 2: Reason for projects discontinued in Ph3 during 2011-2015.

The discontinuation reasons were classified into therapeutic areas (Figure 3). Discontinuation of Oncology and Cardiovascular areas was mainly due to "Efficacy" and infrequently due to "Strategy". "Efficacy" was a less common reason for Endocrine/Metabolism, whereas "Safety" and "Strategy" were more common reasons. There was no "Safety" reason for Infection projects, but "Strategy" was a common reason. We reviewed each therapeutic area by using E/S ratio (number of discontinuation due to "Efficacy"/ number of discontinuation due to "Strategy") for discontinuation trends as shown in Figure 3. Oncology, Cardiovascular and Autoimmune represented relatively high scores, indicating "Efficacy" was a principal reason for discontinuations. Endocrine/ Metabolism and Gastroenterology showed lower scores, indicating that these projects were mainly discontinued due to "Strategy".

Figure 3: Details for discontinued projects on therapeutic area basis. Discontinuation reasons were classified into therapeutic areas. The percentages represent breakdown of discontinuation reason by respective therapeutic area. ^aCNS stands for Central Nervous System. ^bEM stands for Endocrine/Metabolism. ^cE/S represents the ratio of number of failures due to lack of efficacy to number of failures due to strategy.

iii. Changes of the major protocol criteria from Ph2 to Ph3

Some changes were made to the five major protocol criteria of Ph2 and Ph3 protocols in 75 of 119 projects (Figure 4). Only 16% of projects were free of protocol changes. Ph2 itself was not conducted in 18%. On the other hand, some changes were made to the five major protocol criteria in 181 of 544 approved projects. Only 12% of these projects were free of protocol changes. Neither Ph2 nor Ph3 was conducted in 41%. We used discontinued/approved ratios (D/A) to compare trends for discontinued and approved projects as noted in Figure 4. The highest D/A score indicated that discontinued projects were more common in "Protocol changed" comparing to approved projects. The lowest D/A score showed that approved projects were more common in "No Ph2" where only one pivotal Ph2 or Ph3 study was conducted.

Figure 4: Ph2/Ph3 Relationship between differences in number of projects with changed protocol criteria and reasons Target projects were divided according to differences in major protocol criteria and reasons for discontinuation for respective changed protocol categories. ( ) represents percentages of Total number for discontinued and approved projects. ^aNo Ph2 category include not only the case of Ph3 studies without Ph2 studies but also Ph2 studies without Ph3 studies. ^bD/A represents the ratio of percentage of discontinued projects to percentage of approved projects.

Figure 5 showed numbers of differences and E/S scores in the major protocol criteria between Ph2 and Ph3 in discontinued projects. Endpoints were changed in 48 projects. Of these, 32 required outcome measures such as overall survival of Oncology projects in Ph3 from imaging data or tests/biomarkers in Ph2. The E/S score was 13.5 which was the highest number. The evaluation period was prolonged in the transition from Ph2 to Ph3, which was the highest number among the five items.

Figure 5: Numbers and E/S scores of changed major protocol criteria between Ph2 and Ph3 in discontinued projects.
More than one item was selected for the number of changes in major protocol criteria. ^aEndpoint (Outcome measure) : Primary endpoints in Ph2 such as evaluation scale, imaging data, laboratory tests and biomarkers were changed to outcome measures in Ph3. ^bEvaluation period+endpoint: Both of evaluation period and primary endpoint were changed. ^cEvaluation period－endpoint: Evaluation period was changed but primary endpoint was not changed. ^dEndpoint (Evaluation scale,etc): Primary endpoints in Ph2 such as evaluation scale, imaging data, laboratory tests and biomarkers were changed to different measures other than Outcome. ^eDesign means open design of Ph2 changed to double blind design of Ph3.

Overall, the percentages of discontinued projects which changed protocols in all major criteria from Ph2 to Ph3 were more than that of approved projects (Table 2). In particular, D/A scores of endpoint and target subjects are relatively high; indicating that discontinued projects rather than approved projects changed these criteria. In addition, the high D/A trends of every major protocol criteria are clearly found in the case of NCEs and Biologics and are less evident in Reformulations.

Table 2: Comparison between discontinued and approved "Protocol change" projects on product type for differentiation points of major protocol criteria between Ph2 and Ph3.

Changes in major protocol criteria between Ph2 and Ph3 are further stratified by therapeutic areas. Of 40 Oncology projects, 34 changed major protocol criteria. These changes took place mostly in five therapeutic areas. The percentage of "Not changed" projects is highest in Infection (40%). In the same way for approved projects by therapeutic area, all therapeutic areas of approved projects undergo less change compared to those of discontinued projects (Tables S2 and S3).

Discussion

i. Discontinuation reasons of Ph3 projects and details

The discontinuation reason was lack of efficacy in most projects (Figure 2), suggesting that Ph3 could be designed to enable appropriate evaluation of efficacy. Discontinuation due to "Safety" was relatively rare compared to previous studies [26,27]. Drug safety rather than efficacy might have been evaluated prior to Ph3 in many projects. This may be the result of an emphasis on safety assessment due to increasing recent safety concerns of regulatory authorities. A significant number of projects were discontinued due to "Strategy", which was a higher ratio compared to previous studies [26,27]. It is considered that the Ph3 results for these projects did not allow a distinct judgment of unmet medical needs due to stiff competitive circumstances in recent years. Ringel et al. [28] reported indicators of good judgment such as decision making contributed to R&D productivity. Careful checking of unmet medical needs at a stage transition from Ph2 to Ph3 would provide a chance to reduce the incidence of discontinuation by "Strategy".

Oncology is the area with the highest discontinuation rate (Figure 3), possibly because of the complex etiology of the disease and the severe competitiveness of new drug development. Lack of efficacy was the principal reason for discontinuation. It is assumed that it is relatively easy to evaluate the primary endpoint of efficacy compared with other therapeutic areas. CNS ranked second because of the diversity of diseases and the many candidate compounds in development. Hay et al. [11] pointed out a relatively low success rate both for Oncology and CNS diseases. In comparison among approved projects, Infection is the most common, whereas Oncology and CNS areas are less common, supporting the low success rate in Oncology and CNS. The results of CNS and Infection confirmed a consistency with the R&D productivity analysis [28] that Neuroscience is a relatively difficult therapeutic area; by contrast Infection is an easy therapeutic area. Discontinuation due to "Efficacy" was relatively rare and other reasons were notable in Endocrine/Metabolism; this is possibly because, in most studies, the target diseases were diabetes and its complications, and investigational drugs would be required to show some differentiation from established competitors. The high number of "Efficacy" discontinuations in Cardiovascular could be explained as follows: acute coronary syndrome, for which it is considered to be difficult to achieve the expected endpoint, was the more frequent target disease. Success rates for the Cardiovascular were reported to have decreased over time, and this has been one of the therapeutic areas with the lowest success rate [2,29]. Considering that one of the most approved projects in the Cardiovascular was hypertension. This trend may stem from the shift in focus from hypertension to heart diseases. In infection, the most frequent indication was hepatitis-C virus. Since a variety of drugs are available, differentiation between new and established drugs has been vital and has caused a high rate of "Strategy". Previous research on discontinued projects in 2007-2010 showed a similar ranking of the above five therapeutic areas [27].

The higher E/S score levels for Oncology and Cardiovascular indicates discontinuation reasons due to "Efficacy" rather than "Strategy" (Figure 3). The lower score levels for Endocrine/ Metabolism, Infection, and Gastroenterology suggest that distinct judgments for unmet medical needs were not demonstrated because "Strategy" was the main reason. Thus, the discontinuation reasons in each therapeutic area differed; furthermore, from the fact as described in the Cardiovascular case, the trend in discontinuation reasons varied over time. Therefore, the E/S metrics would be a useful parameter to indicate that high score areas should be better designed to enable appropriate evaluation of efficacy and low score areas should carefully assess the external environments including medical needs.

ii. Changes of the major protocol criteria from Ph2 to Ph3

In comparison between discontinued and approved projects, D/A is highest in the "Protocol changed", indicating that the major criteria for discontinued projects were changed more than those of approved ones (Figure 4). D/A in "No Ph2" ("No Ph2 or Ph3 for approved projects) is the lowest. This is because NCE in approved projects were fewer, and formulation changes were in the majority. More concretely, all products apart from NCE were easily approved by conducting only one pivotal study in Ph2 or Ph3. D/A for "Protocol changed" is higher than for "Not changed", suggesting that protocol changes produced negative impacts on success rate.

When the outcome measure endpoint is focused on "Protocol changed", "Efficacy" is higher than "Strategy" (Figure 5). The difference in discontinuation reasons suggests that such a change in a challenging direction could exert the greatest impact on efficacy and would result in a lower success rate. In terms of changes to target subjects, when the study population was narrowed to be based on evidence from Ph2, this would lead to a failure in reproduction of the Ph2 data in some cases. In other cases, Ph3 was started although sufficient evidence might not have been obtained in Ph2. Because the evaluation period of Ph3 was longer than that of Ph2 in most cases, the effect might become weaker and adverse reactions might develop. Lastly, control and design were changed in Ph3, which was not associated with a high rate of "Efficacy", but "Strategy". This was due to discontinuation for strategy reasons such as market competitiveness based on Ph3 results where protocols were changed more objectively, for example by addition of control drugs.

As shown in Figure 5, a change in the endpoint exerted the greatest impact, followed by evaluation period, target subjects, control, and design, in that order. In particular, E/S of the endpoint which was changed to outcome measure in Ph3 is the highest. Of the endpoint changes for approved projects, 19 cases out of 46 (41%) had an endpoint converted to outcome measure, which is a small number compared to discontinued projects (67%). In the case of Oncology, which would be expected to change an endpoint to an outcome measure, these projects should use a Ph2 design that allows data extrapolation for Ph3 by appropriate surrogate endpoints or biomarkers. The 40 studies differing in both endpoint and evaluation period had an E/S of 6.4, indicating that changing these two conditions had a high risk of impacting on the study result. E/S is 3.0 in 18 studies in which only the evaluation period was changed without any changes in the endpoint, suggesting a greater impact of changing endpoints than evaluation period. The impact of changes of control and design were not as great as expected. In addition, other projects were discontinued in the absence of changes in the major protocol criteria. In these projects, Ph2 data may not have been reproduced in Ph3, or initiation of Ph3 may have been based on insufficient Ph2 data.

Even though there were differences in indications between discontinued and approved projects, the latter resulted in fewer changes to the protocol (Table 2). It is noted that there were big differences in endpoints and target subjects, but few differences in evaluation period and control. Evaluation period and control were even changed in most approved projects, which did not significantly impact on the reproducibility at Ph3 of the Ph2 result.

Each change of the major criteria was assumed to be magnified by each E/S score, which then had an impact on the results of Ph3. For instance, although E/S of endpoints and evaluation periods were substantially similar, the D/A of endpoints was greater than that of evaluation periods, and thus the endpoint changes had a greater impact on the outcome of Ph3. In addition, there were obvious differences in endpoints and target subjects in innovative products such as NCEs and Biologics as opposed to Reformulations. Of the 46 cases for approved projects, NCE were 10 cases (22%), while 42% of discontinued projects where endpoint was changed to outcome measure were NCE. It follows from the above result that NCE should be evaluated in Ph3 by the same study design as that of Ph2.

Then we assessed the impact of protocol changes for studies in the top five therapeutic areas (Tables S2 and S3). Of the Oncology projects, 63% changed endpoints, and most of these changed outcome measures. Prolongation of the evaluation period was further reason for the lowered success rate. The percentage of Ph3 with the "No Ph2" component is highest in the Cardiovascular since some target subjects differed between Ph2 and Ph3. "Not changed" is observed in most Infection projects, possibly because the endpoints were clear and a change in Ph3 was not required. By contrast, numbers of changes for approved projects are low compared to discontinued projects. However, the trend such as Oncology and Infection for discontinued projects is similar to approved projects. It could be concluded protocol changes between Ph2 and Ph3 negatively impacted success rates across therapeutic areas.

Conclusion

This research has revealed a number of findings regarding efficiency in development of investigational drugs from studies of discontinued projects. We propose the E/S metrics and D/A that provides insight into why drugs fail during late-stage development and why the rates of failure are different for different therapeutic areas. Drug developers should guard against late-stage protocol changes and should be particularly careful in setting their strategy in therapeutic areas. Access to results of discontinued projects has increased in recent years [18,30], but still remains incomplete. Such information can be used for planning subsequent studies and preventing repetition of past failures. A system of data sharing for discontinued products can be one of the key solutions to enhancement of pharmaceutical drug development.

Acknowledgements

The authors would like to thank Dr. Haruko Takeyama (Department of Life Science and Medical Bioscience, Waseda University) and Dr. Yasuo Ikeda (Department of Life Science and Medical Bioscience, Waseda University) for their helpful advice.

Conflict of Interest/Disclosure

The authors have declared that no competing interests exist.

References

1. Munos B (2009) Lessons from 60 years of pharmaceutical innovation. Nat Rev Drug Discov 8: 959-968.
2. Mestre-Ferrandiz J, Sussex J, Towse A (2012) The R&D Cost of a New Medicine. Office of Health Economics.
3. Paul SM, Mytelka DS, Dunwiddie CT, Persinger CC, Munos BH, et al. (2010) How to improve R&D productivity: The pharmaceutical industry's grand challenge. Nature Reviews Drug Discovery 9: 203-214.
4. Garnier J (2008) Rebuilding the R&D engine in big pharma. Harvard Bus Rev 86: 68-76.
5. DiMasi J, Faden L (2009) Factors associated with multiple FDA review cycles and approval phase times. Drug Inform J 43: 201-225.
6. Schulze U, Baedeker M, Chen YT, Greber D (2014) R&D productivity: On the comeback trail. Nature Rev. Drug Discov 13: 331-332.
7. Mullard A (2016) FDA drug approvals. Nature Reviews Drug Discovery 15: 73-76.
8. Pammolli F, Magazzine L, Riccaboni M (2011) The productivity crisis in pharmaceutical R&D. Nature Rev Drug Discov 10: 428-438.
9. Bungage E (2011) Gettimg Pharmaceutical R&D back on target. Nature Chem Biology 7: 335-339.
10. Goodman M (2008) Pharma industry performance metrics: 2007-2012E. Nature Rev Drug Discov 7: 795.
11. Hay M, Thomas D, Craighead J, Economides C, Rosenthal J (2014) Clinical development success rates for investigational drugs. Nature Biotechnology 32: 40-51.
12. Williams R (2011) Discontinued drugs in 2010: Oncology drugs. Expert Opin investing Drug 20: 1479-1496.
13. Kola I, Landis J (2004) Can the pharmaceutical industry reduce attrition rates? Nat Rev Drug Discov 3: 711-715.
14. Reichert JM, Wenger JB (2007) Development trends for new cancer therapeutics and vaccine. Drug Discov Today 13: 30-37.
15. Khanna I (2012) Drug discovery in pharmaceutical industry: Productivity challenges and trends. Drug Discov Today 17: 1088-1102.
16. Arrowsmith J, Miller P (2013) Phase II and phase III attrition rates 2011-2012. Nat Rev Drug Discov 8: 569.
17. DiMasi JA (2013) Causes of clinical failures vary widely by therapeutic class, phase of study. Tufts CSDD Impact Report 15: 1-4.
18. International Federation of Pharmaceutical Manufacturers & Associations. (2009) Joint Position on the Disclosure of Clinical Trial Information via Clinical Trial Registries and Databases.
19. https://citeline.com/products/pharmaprojects/
20. https://clinicaltrials.gov/
21. https://www.clinicaltrialsregister.eu/ctr-search/search
22. https://database.japic.or.jp/is/top/index.jsp
23. https://www.fda.gov/default.htm
24. https://www.ema.europa.eu/ema/index.jsp
25. https://www.pmda.go.jp/
26. Arrowsmith J (2011) PhaseⅡfailure: 2008-2010. Nat Rev Drug Discov 10: 328.
27. Arrowsmith J (2011) PhaseⅢ and submission failures: 2007-2010. Nat Rev Drug Discov 10: 87.
28. Ringel M, Tollman P, Hersch G, Schulze U (2013) Does size matter in R&D productivity? If not, what does? Nature Rev Drug Discov 12: 901-902.
29. Christopher F (2015) Cardiovascular Drug Development. J Am Coll Cardiol 65: 1567-1582.
30. HHS (US Department of Health and Human Services) and NIH (US National Institutes of Health) Announce New Initiatives to Enhance Availability of Clinical Trial Information: 2016 September 16.