The use of automation in detecting drug diversion is an exciting new area. As one who has developed diversion programs at institutions and been responsible for overseeing the audit process, I know all too well how time intensive and demanding the process can be. It takes days per investigation when a deep dive is needed. The proactive monthly audits can take weeks, depending on the size of the facility. Additionally, performing a thorough audit and knowing what to audit takes skills, which one develops over time. The staff member performing the audit must have a certain aptitude for it, and ideally, a mentor who can work with them on what to look for and how to look for it. Not many facilities have a Human Resources Department (HR) that will allow administration of a Myers Briggs type test to determine aptitude, and most don’t have the luxury of having a skilled investigator to come alongside a new investigator to mentor them. The value of the diversion audits are dependent on time and skill. As a consequence, facilities do the best they can and largely depend on front line staff to make them aware of a potential problem and a need for a deeper investigation.
With automation, those days are over! The automation in still in its early stages, so we need to be patient and willing to work for that automation. This article will review the various drug diversion software products. Vendor, as well as client, interviews have been performed for FairWarning, HelioMetrics, Invistics’ Flowlytics, Kit Check’s BlueSight, Omnicell Analytics and Protenus. BD HealthSight declined to participate. The results of those interviews are in the tables below. I do not have firsthand knowledge of all the products other than seeing demos. I originally planned to ask each vendor how many risk points their product takes into consideration, but I found the answer to that was not decisive enough to put in the comparison. Each product takes hundreds of data points into consideration when running the analytics, and the number of points looked at are increasing all the time as more data history becomes available.
There are four main points I would like to emphasize. First, a vendor may say their product can or does perform in a particular area; however, having the capability is different than a proven track record. Consider asking them specifically how many current customers are live in that particular area of coverage. You may find that the data can be extracted, but currently they do not have enough historical data for the artificial intelligence to make conclusions. Therefore, that area of coverage has not truly gone live. It will come in time, but understand what is possible now versus down the line (short term and long term). Along these lines, remember that your facility will only get the benefit of a feature if there is a way to get the data to the software monitoring system. For example, if you expect infusion pump data integration but your nursing staff bolus out of pumps, chart in various locations within the Electronic Medical Record (EMR), or chart on paper flow sheets, you will most likely be frustrated. Though infusion pump integration was listed as offered, your facility won’t be in a position to utilize it.
My second point is related to the first. Since these drug diversion software monitoring systems are still in early states, there most likely will be growing pains with your vendor. They may be in the infancy stage with a particular coverage area, and they will learn from you. They may not have encountered all the data feed sources for a particular EMR or Automated Dispensing Machine (ADM), so installation may be delayed. If you currently have a home-grown or manual drug diversion monitoring system, you may have to adjust to looking at data or visual reports differently. This may lead to the vendor modifying or developing new types of reports for you. This is a good thing because you have a vendor who will work with you. It can also lead to some frustration, because it will require time and patience while working through the process of improvement.
Third, each of these drug diversion monitoring software programs require feedback from the facilities to improve the analytics and move toward more accurate alerts. It is extremely important that all staff involved in responding to the alerts are consistent in the feedback they provide to the software. For example, let’s say the software sends an alert for a healthcare worker (HCW) who has missing waste. The investigator does a deeper dive and the HCW states that they inadvertently threw the waste away without logging the waste in the ADM. Does the investigator provide feedback to the software that it identified a true positive (possible diversion or poor practice issue), or does the investigator provide feedback that it was a false positive because the HCW stated they actually did waste it so this is just a documentation matter? The feedback will be determined by the investigator and if multiple people are involved in investigations, then the feedback may be inconsistent. Definitions need to be clearly defined from the onset because remember, in order to decrease the false negatives or false positives, the analytics need to get “smarter.” The only way to do that is with consistent, accurate feedback. For the above example, the correct feedback is the software identified a true positive. The HCW may have indeed neglected to document waste in the ADM, but the fact remains the software correctly detected a missing item and you want the software to keep letting you know when situations like these arise.
My fourth and final point for consideration is having one of these drug diversion software programs does not preclude the facility from having a designated person with oversight to the diversion program. These software programs will generate alerts, but someone needs to devote the time to review the alerts, investigate the details, pull in the Risk, Quality, and HR departments as well as unit managers. This person will also need to respond to any call to action which may be needed, whether that means working toward improved practice, or reporting suspected diversion. Also, there is more to a diversion program that happens outside of the diversion monitoring software. Policies and procedures need to be solid and in place for chain of custody. Gaps in drug security should be evaluated regularly, and continuous improvements made. There is a need for continually educating staff on the topic of diversion and their responsibility to report anything suspicious. Hospital leadership should be kept informed of metrics surrounding diversion. Facilities that have a designated drug diversion point person have more successful programs and less risk to their facility, staff and patients.
In conclusion, there is extreme value in the drug diversion monitoring tools. The tables below will give you an overview and a place to start with your review. Consider costs in both directions: the cost of purchasing a software package versus the cost of not purchasing a software package. The cost to purchase is more straightforward. However, the cost of not purchasing, or worse yet, not having a drug diversion program of any kind, may in the end be a much higher cost, not only in dollars but in human life.
Vendor Table (for cells with numbers 1-7, see description below)
Questions | Invistics Flowlytics | Kit Check Bluesight | Heliometrics | Protenus | FairWarning | Omnicell Analytics |
---|---|---|---|---|---|---|
Year of first implementation (roughly) | 2017 | 2017 | 2016 | 2018 | 2019 | 2016 |
Average facility IT manpower hours needed for implementation | 40-60 | <10 | 20-40 | 16 (vendor on-site at facility) | 2-5 | N/A as analytics already on Servers |
Average time to complete implementation | 13 weeks is goal | 6-10 weeks | 6 weeks | 6 weeks | 6-8 weeks | As soon as license activated |
Average training time included with implementation | weekly calls | weekly calls | weekly calls, and as needed | weekly calls and ad hoc as needed | weekly calls | 1/2 day training on use |
Support offered after deployment | M-F business hours EST | 24/7/365 | M-F 0830-1700 EST, response within 8 hours. | "24/7/365 online/email 7am-9pm EST phone" | "IT support 24/7 Audit support business hours"1 | Each customer has a different support plan |
Stages of medication pathway monitored for diversion | Wholesaler to waste | Wholesaler to Reverse Distributor | Wholesaler to Reverse Distributor | CS vault to waste | CS vault to waste | Activity surrounding ADM only |
Electronic Medical Records (EHR) supported | All major | All major | All major | All major | All | Epic & Cerner |
Automated Dispensing Machines (ADM) supported | All major | All major | All major | "All major (no experience with Accudose)" | All major | Omnicell only |
Type of analysis | Artificial Intelligence | Artificial Intelligence | Statistical analysis and behavioral patterns | Artificial Intelligence | Artificial Intelligence | Trend analysis and algorithms. |
Real time data feed available | ✓ | ✓ | ✓ | |||
Data feed off archived data is standard (archive time varies, in all cases no more than 24 hrs) | ✓ | ✓ | ✓ | ✓ | ✓ | |
System in place for notification if feed from EHR or ADM stops transferring | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
Ability to search for staff across multiple facilities (float staff) | ✓ | ✓ | ✓ | ✓ | ✓ | 1 |
Software has current performance in all patient care area types including general nursing, perioperative, outpatient clinics, etc | ✓ | ✓ | ✓ | 2 | 3 | 4 |
Reports available at the system level as well as individual facility level | ✓ | ✓ | ✓ | ✓ | ✓ | 1 |
Customizable System | ✓ | ✓ | ✓ | ✓ | ✓ | |
Weighted Risk Scores | ✓ | ✓ | 5 | ✓ | 6 | ✓ |
Capable of monitoring removals of other CNS depressants along with CS meds | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
Capable of monitoring cancelled removal transaction of non-CS in conjunction with CS removals | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
Integration with time/attendance offered | ✓ | ✓ | ✓ | ✓ | ✓ | |
Integration with infusion pump systems | ✓ | ✓ | ✓ | |||
Capable or monitoring pain score anomalies | ✓ | ✓ | ✓ | ✓ | ✓ | |
Frequency of automated software/patching updates | every 8 weeks unless urgent | every 2-3 weeks | "as needed - upgrades that may require downtime occur approximately quarterly" | every 2 weeks | monthly | every 6 months |
Ability to set up alerts for notification of suspicious alerts | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
Deployment includes back testing (confirming identification of past known diversion cases) | ✓ | ✓ | ✓ | ✓ | ✓ | |
Cloud based | ✓ | ✓ | ✓ | ✓ | ||
Access from any PC (with appropriate security in place) | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
Length of backup storage | As BAA states | indefinitely | Unless specified in contract, stored until end of subscription plus 30 days | per customer's retention policy | per customer's retention policy | Customer responsible for backups |
Automated offsite backups | ✓ | ✓ | ✓ | ✓ | 7 | |
HIPAA Compliant | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
Product differentiator (according to vendor) | NIH grant used for extensive testing of AI capabilities. All facilities get the AI benefits of each other without sharing sensitive information. | Started in the operating room which is a difficult area and they monitor this area very well. They focus on collaboration. This problem can't be solved by pharmacy exclusively. They don't build reports the customer has to spend all day reviewing. | Entire pathway utilizes statistical analysis and behavioral patterns | Approach monitoring from a compliance/risk perspective. They also bring a clear process with their "care flow" pathway for investigation. The platform is customizable to fit the hospital's P&P for investigation | Have a certified solutions for every EHR and ADM. Full managed solution provider, meaning they review P&P, assist with education of employees regarding product and expectations, FairWarning staff review all alerts and notify client if further review needed. | "We know our data best so we can provide the best experience around that feed" |
2. Immature in OR area
3. Has ability, no current customer for all of these areas
4. Yes, if ADM in all the areas
5. Normalized, not weighted
6. Live person at FairWarning providing a weighted review
7. Customer responsible for backups
Client Review Table
Questions | Invistics Flowlytics | Kit Check BlueSight | HelioMetrics | Protenus | FairWarning | Omnicell Analytics |
---|---|---|---|---|---|---|
Size of facility | 1000+ beds, inpatient only | 206 beds, 4 urgent care, 2 ambulatory | 3 facilities. Beds 225, 100, and 75 plus OP Center | 305 beds | No client available for interview. Will update as one becomes available | No client provided for interview but see below for comments from previous clients |
Electronic Medical Record (EMR) | Epic | Cerner | Epic | Cerner | ||
Automated Dispensing Machine (ADM) | Pyxis | Pyxis | Pyxis | Omnicell | ||
What was being done to monitor for diversion prior to software purchase? | RxAuditor | Manual Audits | Nothing formal | Pandora and Omnicell Analytics | ||
"Overall how satisfied are you with the product? 1=unsatisfied 5=very satisfied" | 4 | 4 | 4.75 | 5 | ||
Would you purchase again? | Yes | Would not rule out, but would evaluate other products | Yes | yes | ||
Approximately how long was the learning curve (X work weeks)? | Learning curve was learning how to work with Invistics. Invistics did not have established roll out process. This has improved. | Extremely easy to understand and use. | Two education calls to learn how to use | Couple hours. User friendly. | ||
Was there a percentage increase in diversion cases found after implementation? | Too soon to tell. A facility that had no previous monitoring process, has had diversion identified with Flowlytics | During installation, staff member self reported & retroactively software identified the concern. Software has not identified anyone else. Has identified poor practice. | No formal program prior. In first 2.5 years of using software, roughly 1 diversion/month identified. | None found yet (live beginning of June = 3.5 months at time of interview) but several significant practice issues identified. | ||
Did you evaluate other solutions? If so, what was the reason you selected the one you did? | Yes, all of them & spoke with clients. Selected Flowlytics because at the time they were the ones who could merge the EHR data with ADM data well. | No - was approached to be an early adopter & participate in the design. | Selected because at the time only company who could connect the ADM to EHR. | Decision made by prior pharmacy manager. | ||
How many IT hours would you estimate were invested in implementation? | Hard to estimate, but recommends the facility have an IT point person who thoroughly understands pharmacy workflow. | Hard to estimate due to being a beta site | Hard to say but no issues on the part of the vendor. Total implementation time was 6 weeks. | Unsure but met all target deadlines for go live. | ||
How would you rate the system response time? | Good | Fast | Good. System did go down once due to vendor losing power in a storm. | Fast | ||
Did hospital do any testing to see if they would have caught diversion you have found prior to implementing their solution? | Yes, Flowlytics did alert on all known past diversion cases anywhere from 1 month to 1 year before previous process would have prompted a needed response | Confirmed the one who self reported during installation | During implementation software did confirm known diversion cases | During timeframe past data was imported, there were no past confirmed cases of diversion | ||
Approximate hours it has saved you in terms of monitoring | Time spent on investigations has decreased by more than 50% | Prior a tech spent 3-4 hrs/day on anesthesia, now spends 2-3 hrs/week reviewing alerts & responding. | No previous process, but currently 4-6 hrs/week | Prior would review 5-10 cases/month, each would take 4-5 hrs of investigation. Now 5-10 cases/week reviewed requiring less than 30 min, many only 5 min. | ||
How are you alerted to possible diversion? | Currently logs on to dashboard to review alerts. | Logs on to dashboard daily. Has been talk about being notified via email of an alert as an enhancement. | Reviews daily, weekly and monthly.accessed by logging on to software. Reports can be downloaded in PDF to share with others. | Chosen to be notified via email by Protenus of 5-10 alerts/week. Protenus monitors & sends 1-2 cases/day for review. Client can define that number. Protenus is working on a way to designate a priority to assist client with prioritization. | ||
Anything you would like to see or change? | Vendor has worked to make all improvements suggested. | Would like to be able to do everything electronically, including emailing provider if poor practice has been identified | "When setting a new filter would like the ability to choose all filter criteria at one time then run the data. Would also like set reports to be emailed ather than needing to log on to view. Would like a new grouping of non-CS CNS depressant meds. " | """The amount of change based on feedback has been astronomical"" Currently can only view activity of staff who presented with an alert. Have requested the ability to view any user regardless of alert history." | ||
Would you recommend the product? | Yes | yes | yes | yes | ||
Customer service needs met? | Yes, very responsive to requests. | During implementation vendor was very responsive, however client's project lead has left the company & client is no longer getting the support needed. | When I ask for something they are good about getting on it". When reaching out, client declares how urgent the matter is & vendor usually answers same day and certainly within 24 hours. | yes | ||
What was done post implementation to assure your success with product? | Available when needed | Calls twice a week early on | Available & provided education calls on how to use product. | Calls every other week | ||
What areas of medication pathway are included in the solution you have? | Wholesaler to waste. Working on data validation to incorporate all CII Safe activity. | Currently just around anesthesia machine/waste activity and a few med stations in PACU. Does not include the CII Safe. | Wholesaler to Reverse Distributor. Just launching the OR portion because until now EPIC/IT could not figure out how to get the intra-op flowsheet feed. | Only activity surrounding the ADM on the units. Working on adding the Omnicell Vault. They currently do not have Omnicell in OR so no monitoring there. | ||
Other words from the client | "These solutions are not going to tell you who is diverting. It will tell you who to investigate. Has the potential to ferret out possible diversion & non-compliance. The software gives one the ability to drill down to employee & transaction level to see if problem is for the unit as a whole (practice issue) or staff member (possible diversion). Every facility is so different so there will never be a turn key solution and anyone that says it is turnkey is not realistic. Questions any vendor that says they can do infusion pumps. ""You want a system that can make data meaningful.""" | "Great product, but continue to be issues which the vendor has told client is a Cerner problem. Client is trying to be the go between, but facility resources are slim, & client is not getting vendor support needed. Frustrated with the struggle but happy with the product and thinks will be satisfied once the technical problems are worked out. ""I have hope."" ""Product is great, I wish I could get it to work for everything. We need to get it working for nursing.""" | "Advice- get buy in from the top down. Have a dedicated diversion specialist to oversee program as well as software data. Software has really helped identify poor practice. Simple dashboard. Green, yellow, red. Client can set up what defines green etc. Has used the data to create naloxone/opioid reports." | This customer was not already a customer of Protenus' patient privacy compliance service. | "Current Protenus customer stated: Pandora and Omnicell Analytics give different data. Analytics missed a huge diversion they had of oral CS. They have potential but they no longer seem to be interested in their product. Have given suggestions but seen no changes. (3 clients have made this comment)" |