•Abstract
The Oklahoma City Veterans Affairs Medical Center restructured its formulary system in 2003, creating a system of restrictive
criteria for certain drugs to improve pharmaceutical cost effectiveness by enforcing evidence-based indications in the prescription
system. The new criteria-restriction system required the use of formulary drugs as first-line therapy; prescriptions for nonpreferred,
criteria-restricted drugs had to be justified by the prescriber at the time of prescription in accordance with the criteria
established by the pharmacy and therapeutics (P&T) committee. To determine the effects of the system on cost avoidance and
prescriber opinion, we analyzed drug utilization statistics and surveyed prescribers. Analysis demonstrated that the criteria-restriction
system was an effective cost-avoidance tool because inappropriate prescriptions for restricted, high-cost drugs were curtailed
in favor of lower-cost formulary alternatives. The total cost avoidance observed after placing 15 outpatient drugs in the
criteria-restriction system at the Oklahoma City Veterans Affairs Medical Center pharmacy was $1.23 million to $1.54 million
yearly, equaling 4% to 5% of the total outpatient pharmacy budget. Prescribers generally felt that the system was a "necessary
nuisance," but many expressed frustration concerning the unwieldy interface of the system. (Formulary. 2006;41:657–668.)
The development and operation of effective formulary systems have come under increased scrutiny in recent years because of
the rapid rise in pharmaceutical costs, which have increased 8% to 13% a year nationwide in each of the past 5 years.1,2 Healthcare systems often enact general formulary restrictions based on cost effectiveness, and several studies have measured
the various effects of such restrictions.3–12 Some healthcare systems have opted to specifically restrict prescriptions of certain drugs to the indication(s) for which
usage has been approved. Studies of this type of restriction are scarce: 1 recent study of meperidine restriction and several
related studies of antibiotic restriction are the extent of the available literature.13–23 This study will examine the restriction of 15 nonantibiotic pharmaceuticals by indication and the subsequent effect on pharmacy
cost avoidance and prescriber opinion at the Oklahoma City Veterans Affairs Medical Center (OKCVAMC).
The OKCVAMC is a tertiary-care, 169-bed facility that serves more than 200,000 veterans from Oklahoma and Texas. The OKCVAMC
is run by the Veterans Health Administration, and it operates on a fixed budget that is appropriated annually. At the time
of the study, the OKCVAMC was 1 of 146 VAMCs nationwide. VAMCs are grouped geographically into Veteran Integrated Service
Networks (VISNs), with 23 VISNs at the time of the study. The OKCVAMC is a member of VISN 16, along with 9 other VAMCs throughout
the southern United States. 
Table 1 Drug utilization evaluation comments for selected drugs before criteria restriction
|
Since its establishment in 1950, the OKCVAMC has operated a closed formulary in an effort to control inventory and costs.
The pharmacy and therapeutics (P&T) committee, charged with the creation and oversight of the pharmacy and formulary system,
includes doctors from several subspecialties within the medical center, along with representatives from nursing, pharmacy,
dietetics, and various other disciplines. The authority to approve or deny nonformulary requests is vested with 3 physicians:
the chairperson of the P&T committee, the assistant chief of ambulatory care, and the chief of staff.
Table 2 Source data used to develop criteria restrictions, by drug
|
A number of drug utilization evaluations performed for the P&T committee in 2002 and 2003 indicated that several high-cost
drugs were being prescribed in ways contrary to established guidelines (Table 1). Using national guidelines and input from
local VA doctors, the P&T committee drafted usage criteria for a number of these drugs in an attempt to alter prescribing
patterns. This criteria-restriction system was established to enforce the use of therapeutically equivalent formulary drugs
as first-line therapy and to limit the use of restricted drugs solely to their respective evidence-based indication(s) and
regimens. In October 2003, the P&T committee voted to incorporate those criteria into the computerized prescription system
for the entire OKCVAMC, creating 3 categories of drugs: formulary, nonformulary, and criteria restricted. If a drug is determined
to be criteria restricted, committee members must approve the indication(s) for each drug. If the national VA Pharmacy Benefits
Management and Medical Advisory Panel have released suggested criteria, those criteria are reviewed by doctors in the specialties
that use the drug (Table 2), and after their review and adjustments, the criteria are approved. If no national studies exist,
doctors from relevant specialties work with the P&T committee to create a list of approved indications, relying on scientific
literature and clinical experience.
Figure 1 Outpatient pharmacy options for prescribers
|
Prescription system. The OKCVAMC operates on the Computerized Patient Record System, an electronic system that replaces paper charts, progress
notes, and paper prescriptions for each patient with an analogous computer file. A computerized prescription system allows
for many options in the way that medications are ordered and processed.
Figure 2 Example of approved indications for a criteria-restricted drug
|
When prescribers at the OKCVAMC access the outpatient pharmacy on the Computerized Patient Record System, they first must
choose to order a formulary, nonformulary, or criteria-restricted drug (Figure 1). If the prescribers select a criteria-restricted
drug, a second screen prompts the prescribers to justify their selection of the drug; as Figure 2 shows, prescribers must
provide evidence that the indication approved for this medication is applicable to their patients. For instance, prescribers
may justify the selection of some criteria-restricted drugs that are approved as second-line therapy if first-line formulary
alternatives are not effective (eg, gabapentin is approved for patients in whom amitriptyline [or other formulary alternatives]
has been ineffective or is contraindicated).
Figure 3 Example of a completed prescription
|
Once these steps are completed (Figure 3), requests are forwarded electronically to a clinical pharmacist for review. The
clinical pharmacist checks the patient records to ensure that requests are justified and documented according to the indication(s)
approved for that drug; if so, the prescriptions are approved. If the clinical pharmacist feels that requests are not complete,
prescribers are asked for further justification. If, after review of the further justification, the clinical pharmacist still
feels that the criteria are not met, requests are denied and prescribers are asked to use a formulary alternative instead.
When a request is denied, the prescriber has the option of appealing to the physician who chairs the P&T committee.
This study analyzed the first 15 drugs entered into the criteria-restriction system at OKCVAMC: amlodipine, losartan, valsartan,
carvedilol, metoprolol SA, clopidogrel, gabapentin, insulin glargine, montelukast, nitroglycerin aerosol, olanzapine, rosiglitazone,
pioglitazone, sevelamer, and sildenafil.
METHODS
Cost avoidance. The cost avoidance associated with the criteria-restriction system was measured in 2 ways. For restricted drugs with only
1 primary indication and a limited number of preferred alternative agents with the same primary indication (eg, nitroglycerin
aerosol, amlodipine), cost avoidance was measured by the difference in the average cost per patient per year (PPPY) for a
patient receiving either a criteria-restricted drug or 1 of its preferred alternative agents. This approach allowed for the
consideration that cost avoidance from decreased utilization of the restricted drug was partially offset by increased expenses
for the unrestricted alternative.
Restricted drugs with no preferred alternative agent (sildenafil) or with multiple indications and therefore many preferred
alternative agents (eg, clopidogrel, gabapentin) required a different approach. Only the change in use of the restricted drug
could be analyzed, producing a cost-avoidance estimate that accounted for the savings due to the change from restricted to
formulary drug but not for any other changes in prescription tendencies.
Both types of cost-avoidance analysis were converted into yearly savings in 2 ways. The first savings analysis, designated
as "OKL," assumed that, without criteria restriction, the percentage of all patients on the drug and its preferred agents
at the OKCVAMC would have stayed constant. This analysis was performed by comparing the current average cost PPPY at the OKCVAMC
with the same cost measure before the drug was placed on criteria restriction. The second savings analysis, designated as
"OKL versus VISN," assumed that, without criteria restriction, prescription patterns at the OKCVAMC would have followed trends
reflected in usage across VISN 16. This analysis was performed by comparing the change in the average cost PPPY at the OKCVAMC
after the introduction of criteria restriction with the change across VISN 16 during the same period. Drug utilization information,
measured by the percentage of patients with a prescription for that drug, was combined with drug cost information to determine
average cost PPPY at the OKCVAMC and across VISN 16. For both analyses, the change in cost PPPY was multiplied by the number
of current patients at the OKCVAMC receiving either the restricted drug or the preferred alternative to produce the estimated
yearly cost avoidance.
Prescriber opinion. To assess prescriber opinion, a number of individual interviews were conducted with prescribers from many different specialties
throughout the OKCVAMC. To quantify the opinions found in the interviews and to produce a statistical record of prescriber
opinion, a 10-question paper survey was given to every active prescriber (N=122) in the VAMC. The survey questioned prescribers
in several different ways about their experience with the criteria-restriction system, including their opinions about the
overall effect of the system and how it should be changed.
RESULTS
Figure 4 ARB prescriptions, OKCVAMC and VISN 16
|
Cost avoidance. As an example of how the enforcement of restrictive criteria changes prescription patterns, prescriptions for angiotensin
receptor blockers (ARBs) and angiotensin-converting enzyme (ACE) inhibitors at the OKCVAMC and across VISN 16 can be compared.
In this study, drug utilization and cost data have been analyzed by class (ARB/ACE inhibitor) rather than individually, when
possible. The OKCVAMC pharmacy stocks 2 ARBs (losartan and valsartan) and 4 ACE inhibitors (captopril, fosinopril, lisinopril,
and enalaprilat). Both ARBs and ACE inhibitors are used to lower blood pressure, but the pharmacy cost for an ARB prescription
($153 PPPY) is much higher than the cost for an ACE inhibitor prescription ($47 PPPY). Upon review of the 2 drug classes,
the P&T committee found no compelling literature indicating that the higher-priced ARBs were more effective than ACE inhibitors.
The P&T committee placed ARBs in the criteria-restriction system in October 2003, approving ARB use only for patients with
proteinuria and for patients in whom ACE inhibitors, designated as on formulary, had been ineffective or were contraindicated. 
Figure 5 ACE inhibitor prescriptions, OKCVAMC and VISN 16
|
As seen in Figure 4, before ARBs were placed on criteria restriction, the percentage of patients prescribed ARBs had been
increasing at a steady rate, both at the OKCVAMC and across VISN 16. The addition of ARBs to the criteria-restriction system
caused the percentage of OKCVAMC patients who received ARBs to stabilize, but the percentage of patients receiving ARBs across
VISN 16 continued to increase (Figure 4). The criteria for ARBs emphasized the use of an ACE inhibitor as a first-line agent
instead of an ARB, and the resulting change in ACE inhibitor utilization can be seen in Figure 5; the use of ACE inhibitors
increased greatly after ARBs were added to the criteria-restriction system.
The OKL analysis considered the difference in average cost at the OKCVAMC before and after the introduction of criteria restriction.
The stabilization of ARB use along with the increased utilization of ACE inhibitors resulted in an average cost for patients
on either type of drug at the OKCVAMC of $56.22 PPPY in the last measured quarter (second quarter of fiscal year 2005) compared
with an average cost of $56.79 PPPY in the quarter before ARBs were added to the criteria-restriction system, a decrease of
$0.57 PPPY. Applying this decrease to all 11,649 patients receiving ARBs or ACE inhibitors at the OKCVAMC, an estimated cost
avoidance of $6,639 annually was achieved.
Figure 6 Average cost for ARB/ACE inhibitor patients, OKCVAMC and VISN 16
|
During the period in which the average cost of a patient receiving either an ARB or ACE inhibitor dropped $0.57 PPPY at the
OKCVAMC, across VISN 16 the same average patient cost rose from $59.13 to $61.61 PPPY, an increase of $2.48 (Figure 6). The
OKL versus VISN analysis considered the difference in average cost change between the OKCVAMC and VISN 16; in this instance,
the difference between a decrease of $0.57 and an increase of $2.48 was $3.05 PPPY. Applying this $3.05 change in average
cost to all patients on ARBs or ACE inhibitors at the OKCVAMC demonstrated an estimated cost savings of $35,530 yearly.
Using the changes in average patient cost to calculate cost avoidance allowed 2 different types of savings to be included
in the analysis. One source of savings came from prescriptions for ARBs that were found to be inappropriate and were switched
to ACE inhibitors. The second source of savings came from the change in prescription tendencies after the introduction of
the criteria-restriction system. Inappropriate requests for ARBs were denied or never submitted, and ACE inhibitors were prescribed
instead. In this case, the cost of the nonapproved ARB prescription was never incurred.
Table 3 Cost avoidance resulting from criteria restriction, per drug
|
Analysis of the rates of change in the utilization of the 15 drugs that were placed on the criteria-restriction system and
their preferred formulary agents demonstrated a total cost avoidance of $1.23 million (OKL analysis) to $1.54 million (OKL
vs VISN analysis) per year for the outpatient pharmacy at the OKCVAMC (Table 3), an average savings of $80,000 to $100,000
per drug.
Prescriber opinion. Of 122 surveys distributed, 92 were returned, a 75.4% completion rate. Survey results demonstrated the perception among prescribers
that prescribing a criteria-restricted drug required, on average, 5 to 8 more minutes than prescribing a formulary drug, and
the majority of prescribers believed that the criteria for drug use were "somewhat" correct. When asked how the criteria-restriction
system affected patient care on a scale of 1 to 10 (1, a negative effect; 5.5, no effect; 10, a positive effect), the prescribers'
response was, on average, 3.89. As the frequency of the criteria-restriction system use increased, prescribers perceived a
greater negative effect on patient care. Among prescribers who used the criteria-restriction system more than 3 times a day
(n=7), the perception of the effect of the criteria-restriction system on patient care was, on average, 3.28.
Interviews with prescribers demonstrated similar results. A majority expressed recognition that the criteria-restriction system
is a "necessary nuisance" for a medical system operating on a fixed budget and did not disagree with the concept itself. Other
studies have demonstrated similar results, reporting that prescribers understand the need to effectively manage material resources
and believe that economic data should be incorporated in a medical system.24,25 However, the prescribers in this study expressed dissatisfaction with the interface of the criteria-restriction system,
and some prescribers disagreed with the criteria used for evaluating prescriptions.
Costs incurred. The true economic effect of the criteria-restriction system on the OKCVAMC cannot be measured by the 2 savings analyses alone,
because the system also generates costs. One measurable source of costs is the personnel time used to create and maintain
the criteria-restriction system. These resources include the time spent on the criteria-restriction system by pharmacists
who process the prescriptions, the administrators who set up and maintain the system, and the formulary ambassador who assists
prescribers in navigating the system. An analysis of the number of pharmacy personnel hours spent creating and maintaining
the criteria-restriction system determined an estimated cost of $160,000 per year. This cost has been included in the calculations
for the total cost avoidance.
Another source of costs resulting from the criteria-restriction system is the possible increase in usage of VA medical services
because of less effective medicating of patients. Studies have demonstrated that a lower-priced drug may, in fact, cost more
for the medical center because of higher utilization of other medical services.26,27 For example, 1 of the drugs restricted at the OKCVAMC was insulin glargine; NPH insulin was emphasized as first-line therapy
instead. For both type 1 and type 2 diabetes mellitus patients, NPH insulin produces nearly the same effect on HbA1c levels as insulin glargine, but NPH insulin is associated with a higher incidence of hypoglycemia.28–30 Patients receiving NPH insulin who experienced hypoglycemia were switched to insulin glargine; however, the initial restriction
of insulin glargine may have resulted in a higher use of VA medical services, consuming time and resources that may have offset
the savings incurred by the pharmacy. Studying this source of costs was beyond the scope of this study and warrants further
inquiry.
The criteria-restriction system may also result in increased costs as related to the time invested by prescribers who utilize
the system. As mentioned previously, prescribers perceived that ordering a criteria-restricted drug takes a significantly
longer time than ordering a formulary drug because restricted prescriptions require a written justification, unlike formulary
prescriptions. The extra time spent by prescribers when using the criteria-restriction system represents a cost to the system.
Calculating this source of costs was beyond the scope of this study.
DISCUSSION
The criteria-restriction system has proven to be a powerful cost-savings tool for the OKCVAMC; the effectiveness of the system
can be quantified by using the 2 different analyses developed for this project. The OKL analysis, which isolated the OKCVAMC,
assumed that the percentage of patients receiving a particular drug would have stayed constant at the level present before
the criteria restriction was implemented. Of course, prescription rates often change. Therefore, the OKL versus VISN analysis
assumed that the prescription rates at the OKCVAMC would have changed in the same way that they changed across VISN 16. Used
together, the 2 estimates provided excellent parameters for the true cost avoidance associated with the criteria-restriction
system, which likely fell somewhere in between. Final calculations indicated a yearly cost avoidance of $1.23 million from
the OKL analysis and $1.54 million from the OKL versus VISN analysis. The OKCVAMC outpatient pharmacy has a current annual
budget of $29.9 million, and savings from criteria restriction represented 4% to 5% of that budget.
The cost avoidance measured in this study likely overstated the true cost avoidance to the OKCVAMC because some costs incurred
by the system, in the form of increased usage of medical services and lost time by the prescribers, were not studied and therefore
were not included in cost-avoidance calculations. However, these costs are incurred by the medical services of the OKCVAMC,
whereas the cost avoidance due to changing prescription rates is realized by the OKCVAMC pharmacy. Therefore, the calculated
cost avoidance should accurately reflect the savings for the OKCVAMC pharmacy. Further study is merited to determine the true
costs of the criteria-restriction system for the medical services at the OKCVAMC.
A limitation of the criteria-restriction system was demonstrated through interviews with prescribers, who indicated that it
is quite easy to falsify the electronic patient record to circumvent the system. A prescriber who wants approval for a criteria-restricted
prescription needs only to record that the patient has a condition covered by the approved indication(s). This potential for
falsification of patient records could not only prevent the system from saving costs but could also damage the relationship
between the clinician and the healthcare system. In an effort to limit the occurrence of this type of situation, the P&T committee
now actively solicits prescribers for input when new criteria are constructed, and a committee of the medical staff approves
final criteria for all restricted drugs.
The pharmacists charged with reviewing requests for criteria-restricted drugs found themselves in the position of reviewing
the "correctness" of prescriptions written by doctors. Some pharmacists, when interviewed, expressed that they felt like the
"police" of the criteria-restriction system and that prescribers blamed pharmacists for denied requests. Attempts are being
made to rectify this problem by reinforcing to prescribers that all criteria are determined by a committee of medical staff
and that any denied request can be appealed to a clinician, the chairperson of the P&T committee.
Another potential negative aspect of the criteria-restriction system is the formulation of acceptable criteria for drug use.
Many prescribers feel that restricting the use of drugs to the indication(s) supported by available studies or convincing
clinical evidence unduly restrains their prescribing ability.31–33 This finding is not surprising; other studies have demonstrated that prescribers tend to place a high value on personal
experience when making prescribing decisions, while clinical pharmacists and P&T committee members tend to assign a lower
value to personal experience during the decision-making process.34 Prescribing a restricted drug for a non-FDA-approved indication is prohibited. Although future studies may demonstrate such
use to be effective, the studies conversely may demonstrate such use to be ineffective or even harmful. The criteria-restriction
system is by nature evidence based and therefore tends to be conservative regarding potential new indications for drugs.35
A criteria-restriction system is by no means specific to the OKCVAMC. It can be implemented in any formulary system, although
it would be much more difficult to implement in a system without electronic medical records and a computerized prescription
system. The system does require a large initial investment in time and resources to establish criteria and to create the system;
therefore, the savings of the system may not surpass initial costs for some time. Also, this particular system targeted mainly
outpatient drugs, a recurring and consistent cost. It remains to be seen how effective a criteria-restriction system may be
in cutting inpatient pharmacy costs. Finally, the introduction of this system, like any other change in a healthcare system,
is accomplished much more easily if the hospital is staffed by salaried, full-time prescribers.
Mr Moore is a BA candidate with a Plan II Honors and Biology double major at the University of Texas, Austin, Tex.
Acknowledgments: The author would like to acknowledge Steven Orwig, MD; Martin Jarshaw, PharmD; EM Hampton, PharmD; Elizabeth Schmidt, BSP,
DPh; and Ken Lawson, PhD, for their assistance with this paper.
Disclosure Information: The author reports no financial disclosures as related to the content of this article. The opinions and conclusions in this
article are those of the author and do not necessarily reflect the views of the Department of Veterans Affairs.
REFERENCES
1. Hoffman J, Shah N, Vermeulen, L, et al. Projecting future drug expenditures-2006. Am J Health Syst Pharm. 2006;63:123–138.
2. US Department of Health and Human Services Centers for Medicare and Medicaid Services. National Health Expenditure Data.
Available at: http:// http://www.cms.gov/statistics/nhe/. Accessed November 27, 2006.
3. Abourjaily P, Gouveia W, Selker H, Zucker D. Evaluating the nondrug costs of formulary coverage restrictions. Manag Care. 2005;14:50–57.
4. Soumerai S, Ross-Degnan D, Avorn J, McLaughlin T, Choodnovskiy I. Effects of Medicaid drug-payment limits on admission
to hospitals and nursing homes. N Engl J Med. 1991;325:1072–1077.
5. Tamblyn R, Laprise R, Hanley JA, et al. Adverse events associated with prescription drug cost-sharing among poor and elderly
persons. JAMA. 2001;285:421–429.
6. Huskamp HA, Deverka PA, Epstein AM, Epstein RS, McGuigan KA, Frank RG. The effect of incentive-based formularies on prescription-drug
utilization and spending. N Engl J Med. 2003;349: 2224–2232.
7. Motheral BR, Henderson R. The effect of a closed formulary on prescription drug use and costs. Inquiry. 1999;36:481–491.
8. Hanson EC, Shepherd M. Formulary restrictiveness in health maintenance organizations. J Soc Adm Pharm. 1994;11:54–56.
9. Huskamp HA, Epstein AM, Blumenthal D. The impact of a national prescription drug formulary on prices, market share, and
spending: lessons for Medicare? Health Aff (Millwood). 2003;22:149–158.
10. Meissner B, Dickson M, Shinogle J, Reeder CE, Belzi D, Senevirante V. Drug and medical cost effects of a drug formulary
change with therapeutic interchange for statin drugs in a multistate managed Medicaid organization. J Manag Care Pharm. 2006;12:331–340.
11. Horn SD, Sharkey PD, Gassaway J. Managed Care Outcomes Project: study design, baseline patient characteristics, and outcome
measures. Am J Man Care. 1996;2:237–247.
12. Lyles A, Palumbo FB. The effect of managed care on prescription drug costs and benefits. Pharmacoeconomics. 1999;15:129–140.
13. O'Connor AB, Lang VJ, Quill TE. Eliminating analgesic meperidine use with a supported formulary restriction. Am J Med. 2005;118:885–889.
14. Ozkurt Z, Erol S, Kadanali A, Ertek M, Ozden K, Tasyaran MA. Changes in antibiotic use, cost and consumption after an
antibiotic restriction policy applied by infectious disease specialists. Jpn J Infect Dis. 2005;58:338–343.
15. Ruttimann S, Keck B, Hartmeier C, Maetzel A, Bucher HC. Long-term antibiotic cost savings from a comprehensive intervention
program in a medical department of a university-affiliated teaching hospital. Clin Infect Dis. 2004;38:348–356.
16. Bassetti M, Di Biagio A, Rebesco B, Amalfitnao ME, Topal J, Bassetti D. The effect of formulary restriction in the use
of antibiotics in an Italian hospital, Eur J Clin Pharmacol. 2001;57:529–534.
17. Climo MW, Israel DS, Wong ES, Williams D, Coudron P, Markowitz SM. Hospital-wide restriction of clindamycin effect of
the incidence of Clostridium difficile-associated diarrhea and cost. Ann Intern Med. 1998;128:989–995.
18. Morgan AS, Brennan PJ, Fishman NO. Impact of a vancomycin restriction policy on use and cost of vancomycin and incidence
of vancomycin-resistant Enterococcus. Ann Pharmacother. 1997;31: 970–973.
19. Pestotnik SL, Classen DC, Evans RS, Burke JP. Implementing antibiotic practice guidelines through computer-assisted decision
support: clinical and financial outcomes. Ann Intern Med. 1996;124:884–890.
20. Maswoswe JJ, Okpara AU. Enforcing a policy for restricting antimicrobial drug use. Am J Health Syst Pharm. 1995;52:1433–1435.
21. Crowe HM, Quintiliani R. Antimicrobial formulary selection. Med Clin North Am. 1995; 79:463–476.
22. Suwangool P, Moola-Or P, Waiwatana A, Sitthi-Amorn C, Israsena S, Hanvanich M. Effect of a selective restriction policy
on antibiotic expenditure and use: an institutional model. J Med Assoc Thai. 1991;74:272–275.
23. DeVito JM, John JF Jr. Effect of formulary restriction of cefotaxime usage. Arch Intern Med. 1985; 145:1053–1056.
24. Wu O, Knill-Jones R, Wilson P, Craig N. The impact of economic information on medical decision making in primary care.
J Eval Clin Pract. 2004;10:407–411.
25. Glassman PA, Good CB, Kelley ME, Bradley M, Valentino M. Physician satisfaction with formulary policies: is it access
to formulary or nonformulary drugs that matters most? Am J Manag Care. 2004;10:209–216.
26. Sheehan, DV, Wright-Etter PJ. Impact of formulary restrictions on the cost-effectiveness of antidepressant treatment.
Manag Care Q. 2002;10: 21–31.
27. Horn SD, Sharkey PD, Tracy DM, Horn CE, James B, Goodwin F. Intended and unintended consequences of HMO cost-containment
strategies: results from the Managed Care Outcomes Project. Am J Manag Care. 1996;2:253–264.
28. Garg SK, Paul JM, Karsten JI, Menditto L, Gottlieb PA. Reduced severe hypoglycemia with insulin glargine in intensively
treated adults with type 1 diabetes. Diabetes Technol Ther. 2004;6:589–595.
29. Rosenstock J, Dailey G, Massi-Benedetti M, Fritsche A, Lin Z, Salzman A. Reduced hypoglycemia risk with insulin glargine:
a meta-analysis comparing insulin glargine with human NPH insulin in type 2 diabetes. Diabetes Care. 2005; 28:950–955.
30. Maxion-Bergemann S, Huppertz E, Jacobs LD, Muller F, Walleser S. Improved glycemic control with decreased hypoglycemia
prevents long-term complications in type 2 diabetes patients: long-term simulation analysis using the "diabetes mellitus model."
Int J Clin Pharmacol Ther. 2005; 43:271–281.
31. Goodwin FK. Impact of formularies on clinical innovation. J Clin Psychiatry. 2003;64 (suppl 17):11–14.
32. Rees H, Sipos A, Spence M, Harrison G. Attitudes of psychiatrists to evidence-based guidelines. Psychiatric Bulletin. 2002;26:421–424.
33. Tracy CS, Dantas GC, Upshur RE. Evidence-based medicine in primary care: qualitative study of family physicians. BMC Fam Pract. 2003;4:6.
34. Schumock GT, Walton SM, Park HY, et al. Factors that influence prescribing decisions. Ann Pharmacother. 2004;38:557–562.
35. Eddy, D. Evidence-based medicine: what it is, why to use it, and how to incorporate it into decision-making. Formulary. 2002;37:525–530.
Resources: 
Formulary archives are now available on ModernMedicine.com, a new online resource designed to meet the evolving needs of physicians.